PLAN INTERNATIONAL USA Inc. d/b/a CHILDREACH _____________________________________ FOR OUR MOTHERS AND CHILDREN: Scaling Success to the District Level PLAN Nepal Child Survival XIII Cost Extension FAO-A-00-97-00042-00 30 September 2001 – 29 September 2006 Final Evaluation August 2006 Submission Date: December 31st, 2006 Report Prepared By Sandra Wilcox, Consultant Reviewed and Edited by: Pierre-Marie Metangmo, Laban Tsuma and Plan Nepal Team 2 ACKNOWLEDGEMENTS The principal author of this report would like to express her appreciation to the Plan staff who gave so generously of their time and effort to improve the situation of the women and children of Bara District in Nepal. The author is especially grateful to those who supported the Final Evaluation team during the evaluation process and made the work possible. The leadership and support of Prem Shukla, Nepal Country Director, Subhakar Baidya, Program Support Manager, Dr. Chandra Kumar Sen, District Program Coordinator, Dr. Kedar Baral, Plan Nepal National Health Advisor, and Bhagawan Das Shrestha, CS Project Coordinator facilitated this work tremendously. Additional thanks goes to those who conducted the final surveys and quantitative tools found in the FE: Kedar Baral, Bhagawan Das Shrestha, and the entire CS team. Thanks also goes to Dr. Pierre Marie Metangmo, Plan’s Health Programs Coordinator, whose facilitation skills and long experience with the Nepal CS project greatly enhanced the Final Evaluation process. A special thanks also goes to Nitu Acharya for all her hard work in organizing the final evaluation workshop and the associated logistics for the evaluation activities. 3 LIST OF ACRONYMS ANM Auxiliary Nurse Midwife ARI Acute Respiratory Infection BASICS Basic Support for Institutionalizing Child Survival BCC Behavior Change Communication CB-IMCI Community Based Integrated Management of Childhood Illness CDD Control of Diarrheal Disease CDP Community Drug Program CG Child Group CHDK Clean Home Delivery Kit CMR Child Mortality Rate CPR Contraceptive Prevalence Rate CS Child Survival CSGP Child Survival Grants Program CSHGP Child Survival Health Grants Program CSSA Child Survival Sustainability Assessment CSTS Child Survival Technical Support CWC Community Welfare Center (NGO Partner) CWS Child Welfare Society (NGO partner) DDC District Development Committee DHO District Health Office DHS Demographic and Health Survey DIP Detailed Implementation Plan EOC Emergency Obstetric Care EPI Expanded Program on Immunization FCHV Female Community Health Volunteer FE Final Evaluation FHD Family Health Division (MOH) FP Family Planning HF Health Facility HFA Health Facility Assessment HFSC Health Facility Support Committee HIS Health Information System HMG/N His Majesty’s Government/ Nepal HP Health Post IEC Information, Education, Communication IMCI Integrated Management of Childhood Illness IMR Infant Mortality Rate INGO International Non-Government Organization IUD Intrauterine Device JSI John Snow International LMD Logistic Management Division LQAS Lot Quality Assurance Sampling MCHW Maternal and Child Health Worker MMR Maternal Mortality Ratio 4 MNC Maternal Neonatal (or Newborn) Care MOH Ministry of Health MOT Mode of Transmission NFHP National Family Health Program NFHS National Family Health Survey NSMPA National Safe Motherhood Plan of Action OR Operations Research ORS Oral Rehydration Solution OSM Operations Support Manager PC Project Coordinator PCM Pneumonia Case Management PHA Public Health Administrator PHC Primary Health Care PUM Program Unit Manager PWG Pregnant Women’s Group RHF Recommended Home Fluid SCM Standard Case Management SHP Sub-Health Post SM Safe Motherhood STI Sexually Transmitted Infection TA Technical Assistance TBA Traditional Birth Attendant TFR Total Fertility Rate TOT Training of Trainers UNICEF United Nations Children’s Fund UNFPA United Nations Population Fund USAID United States Agency for International Development VDC Village Development Committee VHW Village Health Worker WHO World Health Organization 5 TABLE OF CONTENTS A. EXECUTIVE SUMMARY............................................................... 6 B. ASSESSMENT OF RESULTS/ IMPACT OF THE PROGRAM..... 9 1. RESULTS: SUMMARY CHART...............................................................................9 2. RESULTS: TECHNICAL APPROACH..................................................................12 3. RESULTS: CROSS-CUTTINGAPPROACHES………………………………..15 a. Community Mobilization………………………………………………….…15 b. Communication for Behavior Change……………………………………….17 c. Capacity Building Approach…………………………………………………18 d. Sustainability Strategy……………………………………………………….22 C. PROGRAM MANAGEMENT....................................................... 23 1. PLANNING…………….. ......................................................................24 2. STAFF TRAINING...............................................................................24 3. SUPERVISION OF PROGRAM STAFF..............................................25 4. HUMAN RESOURCES AND STAFF MANAGEMENT.......................25 5. FINANCIAL MANAGEMENT .............................................................26 6. LOGISTICS……………. .......................................................................26 7. INFORMATION MANAGEMENT.......................................................27 8. TECHNICAL AND ADMINISTRATIVE SUPPORT ............................27 9. MISSION COLLABORATION .............................................................28 10. MANAGEMENT LESSONS LEARNED...............................................28 D. OTHER ISSUES IDENTIFIED BY TEAM .................................... 29 E. CONCLUSIONS AND RECOMMENDATIONS ........................... 29 F. RESULTS HIGHLIGHT................................................................ 32 ATTACHMENTS............................................................................... 33 A. Evaluation Team Members and their titles B. Evaluation Assessment Methodology C. List of Persons Interviewed and Contacted D. Training completed by Project E. Interview questionnaires F. LQAS Results – Comparison of coverage for 8 time periods G. Special Study by Project (Pregnant Women Groups Study) H. Project Data Sheet Form 6 A. Summary The Child Survival XIII Cost Extension project is located in the Bara District of the Narayani Zone of the Central Development Region of Nepal. The district is in the Terai, a lowland area that extends along the Indian border. The project area includes 98 Village Development Committees (VDCs). Bara has a total population of 600,000 inhabitants. Approximately 80,000 children under five years of age and 110,000 women of reproductive age are the target beneficiaries. The goal of the project is to assist the Ministry of Health to improve the health status of children under five and of women of reproductive age (15-49) in Bara district. The end￾of-project objectives are: Behavioral: Women of reproductive age and mothers of children under five years of age will be practicing healthy behaviors and seeking medical care from trained providers. Increased access to services: Communities and families will have increased access to health education, quality care and essential medicines. Quality of care: Ministry of Health personnel, community volunteers and other service providers will be practicing appropriate integrated management of sick children (particularly pneumonia and diarrhea case management). Practitioners and volunteers will also deliver quality family planning and maternal and newborn preventive care. Institutional strengthening: Community-based organizations, local non-governmental organizations, and Ministry of Health facilities in the district will be developed and strengthened to support and implement activities that enhance child survival. Project interventions include: Control of diarrhea, Pneumonia Case Management, Maternal and Newborn Care, and Child Spacing. Project partners are implementing two complementary strategies to reach project objectives. First, the project set up a community based health education, services and monitoring system that is linked to and supports the Ministry of Health services in the district facilities. Second, partners are implementing the Integrated Management of Childhood Illness (IMCI) approach at the facility level and expanding the existing integrated community level approach (for management of diarrhea and pneumonia) throughout the district. According to the final surveys, the project has either met or surpassed most of its objectives. The baseline lot quality assurance sampling (LQAS) conducted in October 2001, found that 16 percent of children with diarrhea in the past two weeks received oral rehydration therapy. The final survey found that 60 percent received ORS, the end-of￾project target is 50 percent. The proportion of mothers who know at least three signs of pneumonia has increased from 15 percent to 98 percent (the project target is 50 percent). The health facility assessment conducted in 2002 found that 69 percent of children with pneumonia who visited a health facility received an appropriate antibiotic. In 2006 it was 100 percent. The HFA also found at baseline that the proportion of children with simple 7 diarrhea who received ORS/RHF was 29 percent, whereas at the Final evaluation that percentage had risen to 75. During the baseline lot quality assessment 32 percent of mothers reported that a trained provider attended their last delivery; the proportion increased to 77 percent at the final evaluation. The end-of-project target is 65 percent. The proportion of mothers who reported having received two doses of tetanus toxoid increased from 13 percent at baseline to 63 percent at the final (the end-of-project target is 50 percent). At baseline 24 percent of mothers who do not desire children in the next two years reported using a modern method of contraception and in June 2006, 66 percent did. These successful results have occurred despite the fact that Bara District has had its services interrupted by the Maoist insurgency. The project was able to employ innovative strategies to address limitations imposed by the insurgency and still mobilize communities as well as reach them with the BCC messages. The project was also able to successfully work with the District Health Office (DHO) to provide regular training and updates to district staff. Regular coordination with MoHP and DoHS at National level. Another strategy contributing to the project’s success is that since the mid-term the project has managed to install functioning CDPs in 94 percent of the district’s VDCs. Some of the priority conclusions and recommendations include: ƒ In sum the key lessons learned from this activity have been: 1) the importance of regular monthly review meetings at the district and sub-district (Ilaka level) health facilities to keep the staff motivated and maintain updated health status records; 2) Using LQAS as a monitoring tool every 6 months which helps the DHO review progress and make local health decisions and; 3) the effectiveness of targeted group education such as the pregnant women’s groups, mother's group and child groups to mobilize women and children to care for their own and the communities’ health. ƒ Some of the major achievements and strengths of the project include the strong relationship that the project has developed with the government of Nepal, particularly the Child Health Division and other key offices at the MOH such as Nutrition, Family Health and Logistic Management Division (LMD). At the district level this has translated into an active and powerful partnership with the District Health Office and the district field staff, allowing the project to strengthen staff skills as well local health structures and systems including local financing through community drug programs. The CS project has also strengthened Plan’s position in Nepal allowing it to have influence in the policy process in Nepal. ƒ Another strength is the strengthening of Bara health facilities. At the beginning of the project most health facilities were not functioning but now 98 percent are operational including community drug programs. Recommendations: 8 ƒ It is very important that the monthly review meetings that have been installed in Bara district at the sub-district level continue and be supported by the DHO. Currently the DHO is supporting the review meetings. Plan intends to continue using its staff to provide on going support for the CDP activities after the project ends. Although the CDPs are present and functioning in 94% of the district, some are new and need continued support to assure success. Plan may also want to consider scaling up this sub-district level review meeting model to its other districts. ƒ The review meetings, in which facility service statistics are analyzed together with the records from FCHVs and the VHW registers, also need to incorporate LQAS monitoring data. The LQAS data should continue to be collected every 6 months. This will allow district personnel to understand where the service gaps and health needs are. ƒ Group approach. The project has had success in addressing project objectives by forming specific groups to address them. These include the Pregnant Women’s Groups and Child Groups. These groups have been successful in motivating women to comply with the recommended visits for antenatal and postnatal care as well as improve immunization levels for children under 5. Given the continued concerns about neonatal mortality, it would be useful to work with these existing groups on issues of newborn care or form pilot groups of mothers with newborns to assist them with breastfeeding, early child care and immunizations. ƒ Turnover of government MOH staff continues to be an issue. At the beginning of the project t it was about 20 percent (10 percent is the norm). Now it is 10 percent so the project has had a positive influence on this. However, Plan and the DHO will need to continue to work on maintaining good morale and motivating staff through on going training updates, sub-district and district level review meetings etc. The DHO stated that they needed to focus on regular field supervision now that the insurgency is over and it will be important as this is one of the few indicators that was not met by the project. It would be advisable for the project to work with the DHO to develop a plan for regular DHO field supervision before the close of project. 9 B. Assessments of Results and Impact of the Program 1. Results: Summary Chart Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jun'06 Confidence Interval ± CI BREAST FEEDING AND CHILD NUTRITION INDICATORS Percent of children aged 0-11 months who are breastfed with in the first hour after birth 9 4.87 66 8.04 Percent of infants aged 0-5 months who were fed breastfed milk only in the last 24 hours 62 8.26 100 Percent of infant aged 6-9 months who received breast milk and solid foods in the last 24 hours 73 15.82 96 4.81 Percent of children aged 20-23 months who are still breast feeding 77 14.72 85 12.23 Percent of Children aged 6-23 months who received a vitamin A does in the last six months 91 4.17 99 1.47 CHILDHOOD IMMUNIZATION INDICATORS Percent of Children aged 12-23 months who have a Vaccination Card 19 6.64 74 7.41 Percent of children aged 12-23 months who received DPT 1 16 6.20 71 7.68 Percent of children aged 12-23 months who received measles vaccine 11 5.22 72 7.62 Percent of drop out- rates between DPT1 and DPT 3 14 14.97 3 3.38 Percent of children aged 12-23 months who received BCG, DPT3, OPV3 and measles vaccines before the first birthday 10 5.05 67 8.00 Percent of children aged 12-23 months who received OPV 3 14 5.95 72 7.62 SICK CHILD Percent of mothers of children aged 0-23 months who know at least THREE signs of childhood illness that indicate the need for treatment 73 5.34 98 1.78 DIARRHEA INDICATORS Percent of children aged 0-23 months with diarrhea in the last two weeks 20 4.83 21 4.87 Percent of children aged 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/ or recommended home fluids (RHF) 16 9.78 60 12.95 Percent of children aged 0-23 who received breastfed same amount or more during diarrhea in last two weeks. 62 12.84 95 6.00 Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more drink / fluid during the illness 24 11.23 93 6.86 Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more food during the illness 27 11.77 95 6.00 10 Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jun'06 Confidence Interval ± CI Percent of Children aged 0-23 months with diarrhea in the last two weeks whose mothers Sought outside advice or treatment for the illness 76 11.23 91 7.60 Percent of mothers who can correctly prepare ORS 34 8.60 84 6.20 Percent of mothers who usually wash their hands with soap or ash before food preparation. 23 7.10 92 4.68 Percent of mothers who usually wash their hands with soap or ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated. 5 3.80 78 7.02 ARI IDICATOR Percent of Children aged 0-23 months with cough and fast / difficult breathing in the last two weeks who were taken to a health facility or received treatment. 79 9.99 80 14.31 PRENATAL CARE INDICATORS Percent of mothers with a maternal card (Card-confirmed) for the youngest child less than 12 months of age 17 6.43 65 8.12 Percent of mothers who received at least TWO tetanus toxoid injections (Card confirmed) before the birth of the youngest child less than 12 months of age. 13 5.67 63 8.20 Percent of mothers who had at least ONE prenatal visit prior to the birth of her youngest child less than 12 months of age 45 8.46 78 7.02 Percent of mothers who received /brought iron supplements while pregnant with the youngest child less than 12 months of age. 36 13.30 93 4.27 PLACE OF DELIVERY AND DELIVERY ATTENDED Percent of children aged 0-11 months whose delivery was attended by a skilled health personal upto TBA level 32 7.95 77 7.19 Percent of children aged 0-11 months whose delivery was attended by a skilled health personal upto MCHW level 42 8.39 Percent of children aged 0-11 months whose delivery involved use of a clean birth kit or whose cord was cut with a new razor 96 3.23 Percent of children aged 0-11 months whose delivery involved use of a clean birth kit 75 7.34 Percent of children aged 0-11 months who were immediately breastfed with the mother immediately after birth. 2 2.52 Percent of children aged 0-11 months who were placed with the mother immediately after birth 25 7.34 82 6.54 POSTPARTUM CARE Percent of mother who had at least ONE postpartum check-up 11 5.38 63 8.20 Percent of mothers able to report at least TWO known maternal danger signs during the postpartum period 41 8.37 Percent of mothers able to report at least THREE known neonatal danger signs 98 2.07 Percent of mothers able to report at least TWO known neonatal danger signs 71 7.68 98 2.07 Percent of mothers who received a Vitamin A dose during the first i k ft d li 16 6.20 80 6.74 11 Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jun'06 Confidence Interval ± CI six weeks after delivery Percent of mothers who received at least 1 month iron tablets during the first two months after delivery 10 5.05 75 7.34 CHILD SPACING Percent of non pregnant mothers who desire no more children in the next two years or are not sure, who are using a modern method of child spacing 24 7.54 66 8.04 Percent of mothers who report at least one place where she can obtain a method of child spacing 54 8.47 100 Percent of children aged 0-23 months who were born at least 24 months after the previous surviving child 58 11.10 69 12.90 Percent of children aged 0-23 months who were born at least 36 months after the previous surviving child 12 7.26 14 9.80 KNOWLEDGE OF DANGER SIGNS DURING PREGNANCY, POSTNATAL AND NEW BORN CHILD Percent of mothers (15-49 years) who know at least TWO danger signs/symptoms during pregnancy 29 7.68 100 Percent of mothers (15-49 years) who know at least THREE danger signs/symptoms during pregnancy 26 7.48 99 1.47 Percent of mothers who knows at least TWO danger signs/ symptoms of after delivery 41 8.37 98 2.07 Percent of mothers who knows at least THREE danger signs/ symptoms of after delivery 11 5.22 95 3.80 Percent of mothers who know at least TWO danger sign of new born 71 7.68 98 2.07 Percent of mothers who know at least THREE danger sign of new born 37 8.20 98 2.07 DANGER SIGNS OF PNEUMONIA AND DIARRHEA Percent of mothers who know at least THREE danger signs/ symptoms of pneumonia 15 6.07 98 2.52 Percent of mothers who know at least THREE danger sign of diarrhea / dysentery 14 5.95 92 4.48 KNOWLEDGE ON HIV/AIDS/STD Percent of mothers who knows at least ONE HIV/AIDS and STD transmission (MOT) 14 5.95 68 7.95 Percent of mothers who knows at least ONE HIV/AIDS and STD prevention (MOT) 14 5.95 69 7.85 12 B.2. Results: Technical Approach A) Project Overview The project is located in the Bara District of the Narayani Zone of the Central Development Region. The district is in the Terai, a lowland area along the Indian border. The project area includes all 98 Village Development Committees --VDC administrative divisions -- of the district. This project followed a previous Child Survival project, which was located in 33 VDCs in Rautahat district and 17 VDCs in Bara District. Bara has a total population of 600,000 inhabitants. Approximately 80,000 children under five years of age and 110,000 women of reproductive age are the target beneficiaries. The goal of the project is to assist the Ministry of Health to improve the health status of children under five and of women of reproductive age in Bara district. The end-of-project objectives are as follows: ƒ Behavioral: Women of reproductive age and mothers of children under five years of age will be practicing healthy behaviors and seeking medical care from trained providers. ƒ Increased access to services: Communities and families will have increased access to health education, quality care and essential medicines. ƒ Quality of care: Ministry of Health personnel, community volunteers and other service providers will be practicing appropriate integrated management of sick children (particularly pneumonia and diarrhea case management). Practitioners and volunteers will also deliver quality family planning and maternal and newborn preventive care. ƒ Institutional strengthening: Community-based organizations, local non￾governmental organizations, and Ministry of Health facilities in the district will be developed and strengthened to support and implement activities that enhance child survival. Project interventions include: Control of diarrhea, Pneumonia Case Management, Maternal and Newborn Care, and Child Spacing. Project partners are implementing two complementary strategies to reach project objectives. First, the project set up a community based health education, services and monitoring system that is linked to and supports the Ministry of Health services in the district facilities. Second, partners are implementing the Integrated Management of Childhood Illness (IMCI) approach at the 13 facility level and expanding the existing integrated community level approach (for management of diarrhea and pneumonia) throughout the district. b) Progress Report by Intervention Area Control of Diarrheal Disease and Pneumonia Case Management Activities: These two interventions have been implemented together through the IMCI approach. Training was given to Ministry of Health staff at the health facilities and to female community health volunteers to address community based approaches. Staff from the project and the Ministry of Health are jointly training, supervising and monitoring the volunteers and birth attendants. They also procure and supply oral rehydration solution packets and cotrimoxazole tablets to health facilities and to the volunteers and birth attendants. Results: The baseline lot quality assurance sampling assessment -- conducted in October 2001 -- found that 16 percent of children with diarrhea in the past two weeks received oral rehydration therapy. The mid-term assessment (January 2004) showed that 35 percent had received it; the final survey found that 60 percent received ORS, the end-of￾project target is 50 percent. The proportion of mothers who know at least three signs of pneumonia has increased from 15 percent to 98 percent (the project target is 50 percent). The percent of mothers seeking care when a child shows pneumonia danger signs only changed from 79 percent at baseline to 80 percent at the final. The target was 90 but this may have been unrealistic. The previous project had an intense focus on community PCM and achieved a high level of compliance. The fact that the project maintained this high level is significant. Data for additional indicators are presented in the Attachments. The health facility assessment conducted in 2002 found that 69 percent of children with pneumonia who visited a health facility received an appropriate antibiotic. In 2006 it was 100 percent. Female community health volunteers also administer cotrimoxazole tablets to children with pneumonia. The HFA also found at baseline that the proportion of children with simple diarrhea who received ORS/RHF was 29 percent, whereas at the Final evaluation that percentage had risen to 75. This may have been facilitated by the availability of commodities through the Community Drug Programs. The mid-term evaluation team found that health facilities and community-based health workers had shortages of oral rehydration solution packets. However, at the final evaluation these shortages were no longer evidenced, mainly because of the Community Drug programs. As with some other commodities (vitamin A for administration to women after delivery or when bringing children for BCG vaccine), shortages were attributed to increased demand resulting from promotional activities carried out by the project in the communities. At mid-term it was discovered that demand was reportedly outstripping the capacity of Ministry of Health to keep facilities and workers supplied. The project and DHOs prioritization of the community drug programs has corrected this situation. 14 Maternal and newborn care and child spacing Activities: The project is training female community health volunteers and traditional birth attendants in maternal and newborn care and child spacing. Birth attendants find training on safe and clean delivery practices valuable. The volunteers educate pregnant women about the need for prenatal and postnatal care, tetanus toxoid vaccination, and iron and folic acid supplementation through pregnant women's groups. Volunteers and birth attendants distribute contraceptives, iron and folic acid tablets, and clean home delivery kits in the communities where they work. The project has also developed a flipchart on newborn care and distributed it to volunteers for use in educating mothers. Results: During the baseline lot quality assurance sampling 32 percent of mothers reported that a trained provider attended their last delivery; the proportion increased to 68 percent at the midterm and 77 percent at the final evaluations. The end-of-project target is 65 percent. The proportion of mothers who reported having received two doses of tetanus toxoid increased from 13 percent at baseline to 20 at mid-term to 63 percent at the final (the end-of-project target is 50 percent). At baseline 24 percent of mothers who do not desire children in the next two years or are not sure reported using a modern method of contraception; in January 2004, 43 percent did so and in June 2006, 66 percent did. See the Attachments for data about other indicators. Although the project has worked hard to discourage community members from treating simple diarrhea with antibiotics, and although the percentage of villagers doing so dropped significantly from 75 percent to 17, it was short of their 10 percent target. During the first Child Survival project, community members of Nijgadh (in the northern part of Bara District) established a maternity home with support from Plan. The facility was initially located in a rented building. Now it has its own building and no longer receives funding from Plan. However, Plan is anticipating that it will support some EOC training at the MOH training facility in the future for the nurse midwives working there. Auxiliary nurse midwives admit women with high risk pregnancies and provide skilled care to them during the final weeks of pregnancy. They also offer routine prenatal and postnatal check-ups, family planning services and child care. The home maintains and operates two ambulances that transport patients to referral hospitals. The ambulance service generates enough revenue to meet its costs and a portion of the maternity home expenses. Special Outcomes The project conducts two surveys a year to monitor progress in achieving targets. These surveys employ the Lot Quality Assurance Sampling technique and generate information from the seven field areas that the district is divided into by the project. Project partners review results, discuss findings, identify problems and recommend subsequent action. The Plan project was one of the first to successfully use LQAS not only for program monitoring but also for baseline and final surveys. Many organizations in Nepal, including Save the children and CARE, have received assistance from the project staff for initiating LQAS surveys. 15 The project staff have documented its use of the LQAS technique in several publications. An issue of the CSTS newsletter highlighted the project’s monitoring strategy (Espeut D., ‘Effective Monitoring with Efficient Methods: Plan/Nepal’s Experience with LQAS in Project Monitoring’ Child Survival Connections, Volume 1, Issue 2, Calverton MD: Child Survival Technical Support Project and the CORE Group). A chapter in a book on community based health care also highlights the project’s LQAS experience (Valadez, J and Devkota BR. 2002. ‘Decentralized Supervision of Community Health Programs: Using LQAS in two districts of Southern Nepal’ Community Based Health Care: Lessons from Bangladesh to Boston, Edited by Rohode J and Wyon J, Boston, MA: Management Sciences for Health). B3. Cross-cutting Approaches a) Community Mobilization A few months before the mid-term after reviewing the project monitoring data, staff realized that despite their work with mother’s groups, they were still not reaching their target group of pregnant women. So in coordination with District Health Office staff, the project initiated the formation of pregnant women's groups. 430 groups are currently active. Each group consists of an average of 8-12 pregnant women. Female community health volunteers facilitate the groups and meetings are held once a month. The groups serve as a forum for discussion of health problems and for education of members and other women about pregnancy care and other maternal and child health topics. The women enjoy meeting and discussing pregnancy related concerns. They also develop community maps with their homes and dots indicating the services they’ve received, which they update at the meetings. Members receive prenatal care at local health facilities. Since the creation of the groups, health facility staff report increases of antenatal check-ups, iron and folic acid supplementation among pregnant and lactating women, postnatal vitamin A supplementation, prenatal tetanus toxoid vaccinations, and immunizations among children aged 12-23 months. The project also works with child clubs in the district. Plan supports over 200 child clubs in the district, 56 of which are in non Plan communities. Project partners have trained club members in staging dramas in their villages to deliver health messages. Club members participate in activities aimed at raising awareness in their communities about health problems, child marriage, bride trafficking and have helped with polio vaccination campaigns. The clubs originally evolved from efforts by Plan to address child rights. The CS project in Bara has been able to further provide basic health training and assistance with organizing meetings. Most of the clubs collect dues and have an annual plan of action. Many of the clubs target girls’ enrollment in schools. The project helps the Ministry of Health with national campaigns to deliver vitamin A, deworming medication (albendazole), and polio vaccine to children. Female community health volunteers inform community members about campaign dates and locations, administer the drug or vaccine, and keep track of the number of children who participate. 16 The project assists the District Health Office with procurement and distribution of supplies. The Maoist insurgency has contributed to inadequate program support particularly in the district interior due to reduced security. Calling meetings with key district authorities has also been hampered by shifting district priorities occasioned by the present conflict. This notwithstanding, as mentioned at the beginning of Section B "Assessment of Progress", most activities in the communities have been conducted as scheduled. Project staff attribute this success to "neutrality, transparency, and good relations". They welcome participation by all in discussions about project activities in the villages -- without regard to political affiliation; they describe project progress and the management of project resources in an open manner; and involve community-based volunteers in efforts to build positive relationships with communities. In a sense the project has been able to use the restrictions imposed by the insurgency to strengthen local organization of services. Because the area health workers were not able to come to regular district meetings and DHO staff were not able to travel to the field to conduct supervision and meetings, the DHO in collaboration with the CS staff decided to institute regular monthly meetings at the sub-district (Ilaka) levels. Now the sub-district In Charge arranges monthly meetings with field staff from the facility and surrounding communities (FCHVs). Previous to the meeting, the FCHVs meet with mother’s groups in their respective communities and compile local health data. This information is then presented at the sub-district meeting along with data from the other area communities. All statistics in accord with MOH reporting categories are recorded on a white board that is displayed at the sub-district office (Health Post or Primary Health Care Center). This activity has served as an inspiration for all the health workers. When interviewed the health workers and health management committee members discussed how they had not understood the importance of the data before and now they have a better understanding of the importance of their work. According to the In-Charges interviewed, these regular meetings have greatly contributed to a sense of ownership of the local health process by local health workers. Most of the community mobilization objectives were met or surpassed. As of June 2006, 98 percent of the VDCs had at least 3 mothers groups with demonstrated health promotion activities and plans for BCC activities (DIP target 80 percent). By the final evaluation, 29 percent of HFMCs or mother’s groups are planning and monitoring local health activities (4 activities: 1) regular meetings, 2) establishment of health savings fund, 3) PWG formation and safer motherhood service and 4) support to out reach clinics). As of June 2006 43 percent of the FCHVs have established community health funds (25 percent were the target). 92 percent of FCHVs were involved in education and community mobilization efforts 8 times in the last 12 months (target is 70 percent). 67 percent of VDCs have committed some financial support to CS activities at the community level (target 60 percent). At the time of the FE 94 percent of health facilities have established community drug programs and community drug management committees. 17 The evaluation team believes that with all these accomplishments that the activities will be maintained by the communities and health facility staffs. Also despite the fact that the CS project is ending, Plan will continue to operate in Bara at a reduced level but a proportion of CS staff will continue working with the district health staff to monitor activities at facilities and in communities and will provide guidance and support where needed. In sum the real lessons learned from this activity have been: 1) the importance of regular monthly review meetings at the sub-district (Ilaka) and district health facilities to keep the staff motivated and maintain updated health status records; 2) Using LQAS as a monitoring tool every 6 months which helps the DHO review progress and make local health decisions and; 3) the effectiveness of targeted group education such as the pregnant women’s groups and child groups. 3b) Communication for Behavior Change The project employs a variety of methods including the use of role plays and street dramas to change behaviors among community members and health facility staff. Pregnant women's groups and child clubs provide project partners the opportunity to deliver messages on maternal and child health in a group setting. Meetings of women's groups often attract non-members (such as mothers and mothers-in-law). As mothers-in￾law have important roles in household decision making, their participation is encouraged. Members of child clubs take what they learn in their groups into the community (families, schools etc.). The evaluation team visited a club that has been very active in community affairs over the last three years. The children persuaded community elders to stop several child marriages -- of girl club members 12 or 13 years of age. The club members in this community also claimed that at this time all girls are enrolled in school thanks to their efforts. Project staffs provide interpersonal communication skills and coaching training to District Health staff to improve the way they interact with patients who visit health facilities. In addition, health staffs participate in the training of community volunteers and in outreach activities in the communities. Having served as trainers and because of interactions with community members outside health facilities, it was anticipated that they would provide better care to patients who are referred by volunteers and to others who visit their facilities. This seems to have been the case as according to community interviews, community members are taking a stronger role in managing their care working in coordination with health facility staff. Project staff have used printed materials, posters, pamphlets and banners developed under the previous project. Many of them are for non-literate audiences and consist of diagrams that were carefully pre-tested and have proved very useful in communicating health messages. The project also makes use of Ministry of Health documents to develop training and education materials. The resulting printed materials are consistent with Ministry recommendations. To supplement these materials, project staff have prepared a flip chart and pictorial cards on maternal and newborn care. The project learned that in 18 training community health workers they should employ the same messages as are used in the key messages in the materials. So the BCC materials and FCHV training use the same words. (In March 2006 two project staffs attended a Behave Framework workshop in Bangladesh to horn their skills in defining activities that best address key factors.) They have also placed billboards with health messages in the project area as well as carried out radio broadcasts to reach the population. The project support regular radio spots between July 2005 through June 2006 that aired through a local Birgunj FM station between 7:15 and 7:30 every Monday morning in the local language, Bhojpuri. Key CS messages were disseminated such as danger signs of CDD, ARI, pregnancy and post partum periods as well as the importance of iron, family planning and MNC. Literacy levels are low in the district (both among community members and community-based volunteers) and pictorial materials assume great importance in educational activities. As a result of the mid-term recommendation, the project contracted Mithila artists from Janakpur to develop wall paintings on CS health themes. While quite lovely, the art work was also costly and does not appear to have had the same relevance to Bara residents as it does to residents in Janakpur and districts farther east. According to mothers groups, health management committees and the facility staffs, the new behaviors will be maintained because the facility will continue to monitor the community activities conducted by the FCHVs and TBAs as well as the outreach conducted by VHWs, MCHWs and other staff. When indicators show in the monthly reporting that there is a problem in certain communities, the staff will address it as it has been doing during the past years. The evaluation team believed that with the decentralization of services and the fact that many VDCs are now supporting health services in some form together with the Community Drug Programs that are now installed in 94 percent of the VDCs, that there is good likelihood that these behaviors will be maintained. Also Plan will continue to work with the DHO in Bara. It will be cutting back its level of involvement but it will monitor health activities and provide support where needed. 3c) Capacity Building Approach i: Strengthening the PVO Organization Child Survival programming has had an important impact on Plan as an organization. As discussed above, its experience with LQAS has been significant. Plan Nepal’s experience has allowed it to be successfully employed in other Plan countries such as Cameroon, Kenya, Mali, Burkina Faso, Senegal and others. As noted in the last section, the Plan Nepal CS staff have written a number of papers and abstracts about their experiences that have been presented at International meetings and journals such as the CSTS Connection that highlighted Nepal’s LQAS experience. Plan headquarters is also using the experience the CS project conducted with the Child Survival Sustainability Assessment (CSSA) tool in other Plan countries and comparing the experience with Nepal’s. 19 Currently Plan as an organization is undergoing a restructuring process. One result is a decision that has made most central level technical area managers redundant. After conducting an extensive survey among program offices, they found that most technical oversight was more effective at regional levels. Child Survival was an exception. The survey found that Country Offices unanimously agreed that the headquarters support and technical tools provided through the CS program have been invaluable and useful for other country programs. The CS headquarters backstop believes that the success of the Nepal experience has helped maintain the headquarters office. ii. Strengthening Local Partner Organization Plan has invested a lot in strengthening 2 local partner organizations, the Community Welfare Center (CWC) and the Child Welfare Society (CWS), who have primarily been responsible for training support for the project. Particular support has been given to management and administration areas including accounting, bookkeeping, office management and information systems. Originally the two agencies worked together but after the mid-term, on a recommendation of the evaluation, the two agencies separated. Each has taken responsibility for different training support activities. For example, CWC provides HMIS training to District staff. They also provide Community Drug Program training at district health facilities (HPs and SHPs) and assist with setting up CDP systems in accord with MOH guidelines. They have provided CDP training in 96 VDCs. They have also organized FCHV refresher training with district health staff, as well as refresher IMCI training for new district health staff. CWS provided training to the FCHVs and other community workers in community mobilization and the formation of PWGs. They also work with the DHO to monitor and improve the EPI program in the 98 VDCs. The NGO partners believe that their experience with the CS project has strengthened their ability to manage health programs and they feel competent to manage child survival projects. In addition to financial management they have developed networking, program planning and monitoring skills. Most valuable has been what they’ve learned about working with the National Health system and coordinating with the DHO operating system. They’ve learned operational skills such as record keeping, accounting, inventory management etc. During the CDP training in communities, they learned the importance of doing work planning with the communities and setting monthly goals. Perhaps the biggest success has been in working with the district health office and the district health staff and facilities. The project has supported staff at the DHO to monitor IMCI and the district has now taken over the position. The CS project also used its influence to recruit the new district Public Health Officer who previously worked in the National Child Health Division, and is actively reviewing and strengthening the district health systems. The project has fortified all the district health facilities assuring all have adequate supplies and equipment. The project has also worked with district staff to assure that all facility and community personnel have received updated training in IMCI, CDD, 20 PCM, family planning and maternal-newborn care (see training section below for more detail). The District has become very involved in managing data collection efforts at all facility and community levels and monitoring progress. The DHO believes that the project has had a significant impact on reducing IMR. Previous to the project according to the MOH records, Bara District ranked 62 out of 75 in terms of health status. In 2005 it was ranked 48 and district officials expect it to rank in the 30s in 2006. While the indicators include all health areas covered by the MOH and not all of these are addressed by the CS project, there have still been significant improvements. iii. Health Facility Strengthening The final HFA shows that health staffs skills have improved dramatically since the beginning of the project, with most clinical exam indicators for project intervention areas ranking in the 90 percent range. The HFA showed that 80 percent of workers had received supervision in the past 6 -12 months and received supportive feedback. It also showed that 94 to 100 percent of sick children seen at the facility were receiving correct home care and caretakers knew 2 signs indicating a need to return to the facility for care if the child’s condition worsened. No facilities had experienced ORS stock outs during the previous month and 100 percent of facilities had up to date immunization and patient registers. At the beginning of the project only 44 percent of registers were up to date. It is expected that once the project ends that the DHO will be able to maintain updated systems and training for HF staff and therefore maintain the gains in health facility care that have been made. The evaluators were able to observe an MCHV training course at the DHO during the evaluation that was part of the regular DHO training program. iv. Strengthening Health Worker Performance The project has built capacity to deliver and track health care services at the community level and among health facility, partner NGO, and Plan staff. The strongest capacity building efforts are at the health facility and community levels -- formation of pregnant women's groups and child clubs and training of traditional birth attendants and female community health volunteers. The project has greatly strengthened the capacities of its main partner, the DHO and district health facility staffs. Although Ministry of Health staff participate in leading training activities, they depend on Plan staff and NGO partners for community mobilization, monitoring, and other project-related activities. As mentioned above by project-end, 80 percent of village development committees are expected to have at least three pregnant women's groups (or other mothers' groups) with demonstrated health promotion activities. In June 2006, 98 percent of the village development committees were reported to have such groups. Compared to an end-of￾project target of 25 percent, 29 percent of female community health volunteers had established community health funds and mechanisms for cost recovery by June 2006. In October 2002, 10% of the volunteers reported conducting community mobilization and education activities eight times in the last 12 months. In June 2006, the proportion had increased to 92 percent (the final target is 70%). Progress has also been slow in 21 improving the frequency of supervisory visits by Ministry of Health staff. At baseline 7 percent of health workers reported at least one occasion in the last three months when a Ministry of Health supervisor visited them. District Health Office records show that in June 2006, the proportion had increased to 52 percent; the final target is 75 percent. The DHO personnel explained the unattained target by the fact that they had not been able to complete the programmed supervisions and field visits because of the insurgency. They also stated that they hoped to greatly increase field visits in the near future now that there is a peace agreement. The regular review meetings at sub-district levels have really had a major impact on health worker performance. This really got going after the mid-term when the Plan Health officer had a meeting with the DHO to see what could be done to encourage regular staff supervision and review of activities and data collected. They also decided to reduce the number of indicators for monitoring to 13. Previously they had measured a large number of indicators that were too numerous for project monitoring. v. Training Project partners have conducted 50 training events since the beginning of the project (for staff members from Plan, partner non-governmental organizations, Ministry of Health staff and community-based volunteers). The project training summary is presented in the Attachments. Training has included: • Transformation for health for project staff (training based on Paulo Freire's philosophy and designed to help trainees gain skills in solving problems in communities in a participatory way) • General training of trainers (a course on training methods and facilitation techniques) • Integrated Management of Childhood Illness for health facility staff and CB￾IMCI for community volunteers. • Control of diarrheal disease and pneumonia case management • Maternal and newborn care and family planning • Clean home delivery (training of traditional birth attendants) • Infection control (training of peons -- support staff -- from health facilities) • Drama (training of child club members in techniques for staging plays on health issues) • Community drug program management and organization for communities • Lot Quality Assurance Sampling training for district personnel • Health facility assessment methodology • Community Drug Program TOT for HF staff and training for SHP staff • Behave Framework Training for 2 Plan staff in Dhaka, Bangladesh Participation in Meetings: In addition to attending formal training events and workshops in the project area, Plan staff (from Bara, Kathmandu, and Washington, DC) participate in national and international meetings and workgroups organized by other agencies (such as the Ministry of Health, Global Health Council, the Child Survival 22 Technical Support Project, and the Child Survival Collaborations and Resources Group). In May 2002, Babu Ram Devkota, the Project Coordinator, participated in Global Health Council's annual meeting in Washington D.C. In September 2002 he presented a paper at the Data for Action Workshop organized by the Child Survival Technical Support Project and the Child Survival Collaborations and Resources Group ("Using Biannual LQAS Data to Improve Child Survival and Safe Motherhood in Nepal: Program Results from 1999 to 2001"). Since the mid-term, Babu Ram, who has moved out of Nepal continues to work under contract for Plan providing LQAS training to other projects. Supportive Supervision: In order to strengthen capacity for delivery of health services in Bara, the project supplements training events with review meetings and supportive supervision. Trainees are encouraged to report problems they face in their daily work during supervisory visits and they receive support in solving problems from their supervisors. Plan has designed several checklists with input from the Ministry of Health. These checklists help supervisors remember key issues when they observe project activities in health facilities and communities. Effect of Capacity Building Efforts on Facilities and Community-based Volunteers: The team that conducted the baseline health facility assessment in January 2002 was able to visit only a third of the facilities (the rest were closed). All the facilities visited by the final evaluation team were open and project staff report that health posts are now operational year-round. Facility staff members stated, during interviews by the team, that their ability to manage diarrhea, pneumonia, and other childhood illnesses has improved after attending project-sponsored training events. They were also pleased with the training provided to peons (on infection control). Sanitation at health posts has improved. At the district hospital, the chief of the District Health Office personally leads an effort to clean the premises every month. During interviews with the evaluation team, traditional birth attendants reported that training in clean home deliveries has been very useful. However health facility staff indicated that although they appreciated all the training, the most useful for them has been the IMCI training that has revolutionized the way they view and treat patients. They now look at the patient from an integrated standpoint, no longer only focusing on the immediate problem or task but examining the child as a whole and providing counseling to mothers about all areas of child health and nutrition. After the project it is expected that some training will continue to be conducted by the Child Health Division of the MOH, such as the regular IMCI annual reviews that they currently have. The DHO will take over training activities and Plan will continue to help them with planning. 3d. SUSTAINABILITY STRATEGY Although there were no sustainability objectives there were the institutional objectives mentioned above in which the project has either met or surpassed its targets achieving them at between 80 and 100 percent. 23 Several activities conducted by the project point to the long term sustainability of CS activities. The maternity home in Nijgadh, which was begun during the first CS project, has been operating without Plan support since 6 months before the 2004 mid-term evaluation. This year Plan will support EOC training for the nursing staff at the center. Plan is arranging the training with the MOH at the Narayanghat Hospital training center. The phase over plan of responsibilities to the DHO is on schedule and the project has been scaling back since July 2005. As noted above local financing of the CDPs is good, with 94 percent of health facilities having them. In addition, 42 percent of FCHVs have community revolving funds with mothers groups to support re-supply of commodities. Also 67 percent of VDCs have agreed to provide some support to health facilities and the government has recently increased the amount budgeted for health that is obligated to the VDCs. Previously it was a half million rupees that should go to health now it is 1 million. One of the obvious conclusions is that the various community groups that have been mobilized through the project such as the health management committees, the mothers groups, PWGs and the CGs have built increasing demand for services and quality care. In July 2006, the project and the District health staff conducted a Child Survival Sustainability Assessment (CSSA) workshop and applied it to Bara District. Findings showed that the community capacity index scored the highest at 76.6, the health outcome index and health services index both scored 63. The organizational capacity index scored 67 and the organizational viability index scored 51. The environmental index scored the lowest at 32. The environmental index includes many factors beyond the immediate control of the project, such as education and female literacy levels, percent of landless populations as well as decentralization and implementation status. Although these factors are not in the immediate project area, they are in areas that Plan is trying to address through its other programs. The CSSA has proved a useful tool for Plan to assess its long term impact not only in Nepal but also in other countries. C. Program Management Project personnel in Bara are highly skilled managerially and technically and are led by a very knowledgeable and experienced Project Coordinator, Bhagawan Das Shrestha. They receive good support from Dr. Kedar Baral, the Kathmandu-based National Health Advisor and from Dr. Pierre Marie Metangmo and Dr. Laban Tsuma (from Plan's Washington, DC office), and Dr. Sun Lal Thapa (Child Health Division, Ministry of Health, Kathmandu). Project team members have forged excellent working relationships with each other and with staff from partner organizations. Many of them have worked together for years (during the previous Child Survival project and the current project). During the final evaluation, representatives of local non-governmental organizations, District Health Office, the Child Health Division and Plan's Kathmandu and US offices commended project staff for their good teamwork. 24 C.1 Planning Bara district is divided into seven field areas. Field teams consisting of a Community Health Officer and 1-3 Assistant Community Health Officers conduct much of the planning and decision-making for project management in each area. The teams work closely with female community health volunteers, traditional birth attendants, pregnant women's groups, and health facility staff. Two advisory groups meet at the district level every two months (the Reproductive Health Coordination Committee and the Quality Control Committee). Project management issues are discussed in these committees. However, some supply and staffing issues are still negotiated at the national level. The project is having more and more success with that and having more influence on the types of staff assigned to Bara district. The project has an annual plan based on the work plan and LQAS findings. After the annual plan is developed, each field area develops its annual plan. At monthly planning meetings, the staff present their activities, discuss constraints, lessons learned and then based on this develop the next months plan. Each field area has a monthly work plan with daily activities. If there are problems in an area then visits by project and DHO staff are organized to follow-up. Overall the project found the DIP work plan useful. The annual work plans are based on the DIP. There were a few strategies that weren’t for seen in the DIP that the project had to develop such as the sub-district level monthly review meetings, but in general the DIP plan was the basis. The DIP provided the overall direction for the project. They sometimes had problems due to the conflict in the area so the project prepared strategies to work around it such as the review meetings, PWGs and CGs which according to project staff were the major reasons they achieved results. C2. Staff Training As described in Section B.3c (Cross-cutting Approaches: Capacity Building Approach), project personnel have participated in a number of training events during the life of project. Trainees are tested before and after the events to measure changes in knowledge levels. New knowledge and skills are also monitored and reinforced through review meetings and supportive supervision. In addition to receiving the training, staff are also facilitators for CB-IMCI, logistics management and other activities. The main lessons learned for staff training include that it is key to have fixed interval refresher training for community and facility staff. Also training needs to be followed with supportive supervision. The staff saw a need for further training needs to develop negotiation skills and learn how to handle difficult people, clients, health workers etc. 25 C3. Supervision Project staff visit health facilities and community health volunteers to monitor their work, reinforce knowledge and skills gained during training workshops, and to help solve problems. They encourage District Health Office staff to accompany them during their visits. As described in Section B.3.c (Cross-cutting Approaches: Capacity Building Approach), supervision by district office staff has increased over the course of the project. However, even now supervisors from the district office are able to visit only 23% of health workers over a three-month period. They lack adequate transportation and until recently felt unsafe traveling to villages because of the Maoist insurgency. All program staff participate in monthly review meetings and based on findings the project staff find out what the needs are for TA and visit the program sites as required. The monthly review meetings will continue after the project ends and will be run by the DHO. The DHO is already carrying out some supervisory activities like the monthly review meetings with district staff which were previously run by the project. The DHO has recently brought in a new Public Health Administrator (PHA) who is actively recruiting new staff and changing old staff in the district that are not producing. CS staff believe that a result of these on going efforts the district will have very strong staff in place. C4. Human Resources and Staff Management Ministry of health staff members at the district office and health posts are frequently transferred. This disrupts service relationships with community members and programmatic decision making. It is hoped with the new PHA being so active in the district that he will be able to negotiate with central authorities to reduce staff transfers. The Plan team in Bara has been a remarkably stable group with little turnover. The original Project Coordinator did leave after the mid-term but was replaced 4 months later. Plan managers in Bara and Kathmandu (Project Coordinator, National Health Advisor, Human Resource Manager, and Country Director) have been aware of the need to plan ahead in order to find jobs for project staff after funding ends in 2006. They are planning to maintain two thirds of the current staff in Bara to continue working on CS activities, which demonstrates Plan’s commitment to continuing CS in Bara. They will also be moving some personnel into positions with other Plan projects in Bara and other districts of Nepal. Essential personnel policies are in place and CS staff, although hired specifically for the project, do receive most of the same benefits of regular Plan staff. Staff morale is high especially since a decision was made to maintain the CS staff at Plan. Currently 3 CS staff have taken jobs in other districts. The Plan Health Coordinator is trying to implement health programming that follows the principles of the CS project, though there is not as much staff in the other districts. 26 C5. Financial Management The Finance Officer provides monthly financial reports to the Project Coordinator, who consolidates the information and submits quarterly reports to the Operations Support Manager via the Program Unit Manager. The reports are then passed on to Plan's U.S. office where they are reviewed for compliance with USAID requirements and then submitted to USAID. Funding transfers are made on a monthly basis from the US office to the project to reimburse project expenditures. The NGO partners send expenditure reports to the project accountant who then send them to the Operations Support Manager who includes it with the rest of the project accounts and forwards it on to Plan’s US accounts office. In 2005 for USAID, Price Waterhouse audited the Plan Nepal CS project accounts and there was “no comment.” The Health Coordinator spent a lot of time with the auditors explaining the project and partner activities. The project expects to have spent all the allocated funds by the end of the project, September 29, 2006. There were funds left over from the first CS project that were spent during the first year of this project. Plan has also provided complementary resources for activities. The project has had enough outside technical assistance. The headquarters staff, Laban Tsuma and Pierre Marie Metangmo, worked with the CS project to apply the CSSA framework to Bara district. The DHO and NGO project partners participated in the in the September 2005 workshop. C6. Logistics The mid-term report indicated that the project health facilities and community-based health workers were facing shortages of some medicines and other commodities (such as BCG vaccine, oral rehydration solution packets, and vitamin A for administration to women after delivery). District officials pointed to insufficient supplies from the national level and stated that they quickly move commodities to health facilities and workers, with help from Plan and the Nepal Family Health Program, after they are received from central supply units. They wanted the Ministry of Health to step up supplies of these commodities to keep pace with the increased demand that resulted from promotional activities carried out by the project in the communities. Since the mid-term, the CDP program has really evolved to fill the gaps from the MOH central supply. Each sub-health post is given 25,000 rupees worth of drugs which is inadequate for all the drugs needed but with the CDP program in operation they are able to maintain supplies. 98 percent of VDCs have operating CDPs. Also the MOH drug supply system has improved so drugs can be requested when supplies are low. There is a basic charge for medicines but essential drugs such a cotrimoxazole, vitamin A and iron are distributed gratis. 27 C7. Information Management A strong community monitoring system is in place. Mothers groups and Pregnant women's groups prepare village maps that provide a snapshot of preventive services received by their members. Female community health workers compile data on the status of community families and how up to date mothers and children under 5 are in receiving preventive facility and outreach services. This data is share with health facility and project staff at regular monthly meetings. The project also conducts a survey every six months to monitor progress in changing health knowledge and practices of caregivers in Bara. The lot quality assurance sampling technique is used to identify 19 households in each field area. Project staff conduct interviews with women aged 15-49 years and mothers with children 0-23 months of age. They then summarize survey results, discuss problems, and suggest solutions for each field area. They also combine findings for the seven field areas to generate results for the entire district. As a result of mid-term recommendations, the project reduced the number of indicators used for monitoring from 50 to 15. The Health Information System Coordinator has been competently managing the information system for 8 years (began with previous CS project). Plan is currently looking for a Monitoring and Evaluation training update for him to attend in order to keep his skills current. Dr. Thapa, Director of the Child Health Division who is the National Coordinator for IMCI likes the LQAS system and is currently using it for national immunization programming (the Plan Health Coordinator trained the Immunization Focal person in use of LQAS). LQAS is also being used in selected MOH districts as a monitoring tool. C8. Technical and Administrative Support Several organizations and individual consultants have given technical assistance to the project. Basic Support for Institutionalizing Child Survival II Project (BASICS II) helped train health facility staff in improving the quality and coverage of routine immunization. Trainers from the National Pediatric Society of Nepal conducted basic and follow-up training in Integrated Management of Childhood Illness. Individual consultants were hired to help develop the detailed implementation plan, to train staff in "transformation for health", and to evaluate the information, education, and communication strategy. Dr. Pierre-Marie Metangmo (Plan - USA) helped with the project start-off workshop and he, along with Dr. Laban Tsuma and other colleagues from his office, conducted the CSSA Workshop with project and DHO staff, edited annual reports, answered technical and administrative questions, and shared technical materials with staff in Bara. Project staff conducted an operational research study to assess the impact of pregnant women's groups in improving care-seeking behaviors. A comparison between PWG members and non-members showed 50 to 60 percent reductions in IMR, NMR, ENMR, PMR and MMR between those attending the PWGs and those pregnant women who did not. Project staff have also benefited from training in the Windows version of EpiInfo (an 28 epidemiological and statistical software package designed by a team from the Centers for Disease Control and Prevention, Atlanta, Georgia). C9. Mission Collaboration According to interviews with USAID Mission personnel, it was clear that they really appreciate Plan’s child survival accomplishments and related activities such as the LQAS training. USAID believes that the CS project is directly in line with the Mission Strategic Objective (SO) that is to reduce child mortality. A secondary objective is to reduce maternal mortality and increase use of family planning services. The CS project also targets both of these objectives through its interventions. In other areas, the strengthening of logistic and community supply systems also are in line with mission objectives. Plan has also collaborated with the Mission to help with specific activities. For example when the Mission was having difficulty purchasing zinc for its projects, Plan was able to purchase it for them. In Bara, the Child Health Officer participated in 3 meetings to strengthen health post level review meetings, determining Plan’s and the DHO’s roles. The mission and Plan staffs work together on several national working groups: FCHV, child health, nutrition, IMCI, etc. In addition, the Plan Health Coordinator is in weekly contact with the Mission Child Health Officer. Whenever there are low health indicators in Bara district, the CS project is contacted to respond and often USAID is involved. There is also good collaboration between the CSP and the mission funded National Family Health Project (NFHP). For example, at the IMCI training update held in Bara this year, NFHP sent the facilitator and the CS project supported logistics for the meeting. Also an NFHP representative participated in the FE. C10. Management Lessons Learned According to project staff, the key management lessons learned are: ƒ The monthly review meetings with project staff have been a key management strategy. The project directors learned early on that in addition to the technical planning and evaluation tasks at the monthly meetings, it was key to spend time interacting with staff to learn their concerns and discuss the difficulties they were dealing with. The monthly meetings became a forum for sharing concerns, problem solving and strengthening the bonds between staff members. It seems to have worked well as there has been very little turnover among staff in the five years of project activity despite the tremendous difficulties of working in a conflict zone. The few who have left have moved on to more responsibility with Plan or other health projects. ƒ The management structure of the project did not change during the project period. Because of the well experienced staff and stable structure and procedures, the project was able to maintain its operations without interruption during the 6 29 months period after the first project director left and a new one was hired. An interim management team of 3 project managers was formed who, with support from Plan’s Health Coordinator, were able to maintain day-to-day operations. D. Other Issues Identified by the Team There were no other issues identified by the team. E. Conclusions and Recommendations As can be seen from the Results Summary Chart in Section 1B the project has either met or surpassed almost all its goals and objectives. These outcomes are the result of the innovative strategies involving capacity building of DHO partner staff, community mobilization and a forthright and well monitored technical approach for reducing maternal and child morbidity and mortality in Bara District. Some of the major achievements and strengths of the project include the strong relationship that the project has developed with the government of Nepal, particularly the Child Health Division and other key offices at the MOH such as Nutrition, Family Health and Logistic Management Division. At the district level this has translated into an active and powerful partnership with the District Health Office and the district field staff, allowing the project to strengthen staff skills as well local health structures and systems including local financing through community drug programs. The CS project has also strengthened Plan’s position in Nepal allowing it to have influence in the policy process in Nepal. Another strength is the strengthening of Bara health facilities. At the beginning of the project most health facilities were not functioning but now 98 percent are operational including community drug programs. The project has dramatically strengthened the Bara district communities despite the civil disruption caused by the Maoists insurgency throughout the project period. However because of innovative strategies such as holding monthly review of community data at local sub-district levels (since district staff could not travel to central offices for such meetings), the development of Pregnant Women’s Groups to directly address issues of this target group, Child Groups to take advantage of an active mobilizing force in the communities, the project has either reached or surpassed most of its objectives. Although the project is ending, Plan will continue to support health activities in Bara as well as its other programs. Since July 2005, the project began turning responsibilities over to the DHO. At this point, Plan is not as involved as it was but will continue to monitor district health activities building on its close relationship with the DHO. When needed Plan will work with the District to address problems and concerns. The new PHA has requested that Plan house one of it’s staff in his office. Plan will also continue to build its health programming in the other 7 districts where it works in Nepal. Although Plan employs a more reduced level of effort in these districts than has been expended 30 with the Bara CS project, it has applied the lessons learned from the CS project and worked closely with the MOH to implement successful strategies. Although the CS project is ending, the evaluation team has a few recommendations for the remainder of the current project and for continuing child survival activities in the district. ƒ It is very important the monthly review meetings that have been installed in Bara district at the sub-district level continue and be supported by the DHO. Efforts should be made by the project to assure that the review meeting system is in place at the DHO before the close of the project. Currently the DHO is supporting the review meetings. Plan intends to continue using its staff to provide on going support for the CDP activities after the project ends. Although the CDPs are present and functioning in 94% of the district, some are new and need continued support to assure success. Plan may also want to consider scaling up this sub￾district level review meeting model to its other districts. ƒ The review meetings in which facility service statistics are analyzed together with the records from FCHVs and the VHW registers, also need to incorporate LQAS monitoring data. The LQAS data should continue to be collected every 6 months. This will allow district personnel to understand where the service gaps and health needs are. ƒ Group approach. The project has had success in addressing project objectives by forming specific groups to address them. These include the Pregnant Women’s Groups and Child Groups. These groups have been successful in motivating women to comply with the recommended visits for antenatal and postnatal care as well as improve immunization levels for children under 5. Given the continued concerns about neonatal mortality, it would be useful to work with these existing groups on issues of newborn care or form pilot groups of mothers with newborns to assist them with breastfeeding, early child care and immunizations. ƒ Turnover of government MOH staff continues to be an issue. At the beginning of the project it was about 20 percent (10 percent is the norm). Now it is 10 percent so the project has had a positive influence on this. However, Plan and the DHO will need to continue to work on maintaining good morale and motivating staff through on going training updates, sub-district and district level review meetings etc. The DHO stated that they needed to focus on regular field supervision now that the insurgency is over and it will be important, as this is one of the few indicators that was not met by the project. It would be advisable for the project to work with the DHO to develop a plan for regular DHO field supervision before the close of project. ƒ Presently there is a peace accord between the government and the Maoists. However if that changes and the insurgency returns, the DHO will need assistance 31 in development of strategies and negotiation skills to continue to provide community care. ƒ It would be useful in order to support future activity with the MOH, for Plan to provide negotiation skills training for Plan field staff. It is not always easy to work with the different personalities and convince government workers of the importance of learning new ways of providing care, acquiring new skills and doing accurate and timely reporting. Negotiation skills will also be important in dealing with the Maoists and other rural community leaders present in many of the villages as well. ƒ Currently the communications officer is writing a script for a video film that documents the community mobilization strategies used by the project including the formation of PWGs and child clubs. The MOH Director of Child Health and IMCI would like to use this as a training tool for MOH health workers to show how it is done. It is recommended that Plan complete this project for use by the MOH and others in Nepal who may benefit from the community mobilization lessons learned by the CS project. The evaluation team recognizes that this may not be completed before the end of the project but if Plan has the resources to finish it later, it would be a very useful tool for any future work it plans to conduct with the MOH. It would also be important to send this to the CSHGP office as a reference for other CS projects. ƒ The CS project has achieved and documented some important successes during the life of project. One of these is the results of the Pregnant Women’s Group Impact Study. It would be useful to present these results at international meetings and/or publish them in relevant international health journals. It might also be useful to streamline the methodology and results of the Sustainability Assessment Framework Workshop (2006) and use it as an example of a tool that could be applied to similar projects in Nepal. It could serve as a tool for examining what sustainable development means in Nepal and for highlighting what factors need to be addressed for improving sustainability. In the case of this assessment, the weakest areas were in the environmental index, which are mostly areas beyond the scope of the project, but within several domains that Plan addresses in its broader community projects. In its future activities, Plan may want to consider applying this framework to assess the feasibility of sustaining its projects within the broader context of its activities. 32 F. Results Highlight The Plan CS XIII cost extension project in Bara has perfected the use of biannual LQAS monitoring for project indicators. During these surveys DHO staff, NGO partner staff and Bara Plan staff involve themselves through the entire process including the analysis. Thus the project has institutionalized LQAS monitoring process with the District Health System. In fact LQAS is now used by the Child Health Division of the MOH to assess national Immunization coverage. This was made possible through training offered to the National Immunization M&E focal point by Plan. Forming and working with Pregnant Women Groups has been another project highlight. Each pregnant women's group consists of an average of 8-12 pregnant women. Female community health volunteers facilitate the groups and meetings are held once a month. The groups serve as a forum for discussion of health problems and for education of members and other women about pregnancy care and other maternal and child health topics. The women enjoy meeting and discussing pregnancy related concerns. They also develop community maps with their homes and dots indicating the services they have received, which they update at the meetings. Members receive prenatal care at local health facilities. Since the creation of the groups, health facility staff report increases of antenatal check-ups, iron and folic acid supplementation among pregnant and lactating women, postnatal vitamin A supplementation, prenatal tetanus toxoid vaccinations, and immunizations among children aged 12-23 months. Plan Nepal also made a calculation of the number of lives saved between 2001 and 2006 (Lives Saved Analysis) in the Bara project area by use of the Bellagio calculator. The Bellagio Group on Child Survival represents many technical agencies. It published the 2003 Child Survival and 2005 Neonatal Survival articles in the Lancet. The Bellagio group estimated the percentage of deaths attributable to the six most common causes of child mortality (diarrhea, pneumonia, measles, malaria, HIV/AIDS and neonatal causes) in 42 high mortality countries including Nepal. It reviewed the literature for interventions supported by the evidence as effective against these six main causes and estimated the effect size of these interventions from the literature. The calculator estimates the number of deaths attributable to the six main causes and accounts for both the effects of malnutrition on mortality and the effect of interventions on multiple conditions (e.g. Vitamin A reduces measles, diarrhea and malaria deaths). It also avoids double counts. Lives saved for each intervention is given by the product of baseline number of deaths for that cause, intervention effectiveness and change of coverage. The project was able to save 1302 lives in the course of its 5 years which approximates to under $1000 per life saved. In the last year of the project alone it is estimated that it saved 259 lives from diarrhea, 162 lives from pneumonia, 97 lives from neonatal causes and 3 lives from measles for a total of 521 lives saved. This makes the project a highly efficient program worth emulation in the region. 33 ATTACHMENTS A. Evaluation Team Members and their titles B. Evaluation Assessment Methodology C. List of Persons Interviewed and Contacted D. Training completed by Project E Pregnant Women’s Group Impact Study F. Interview questionnaires G. LQAS Results – Comparison of coverage for 8 time periods H. Integrated Health Facility Assessment: Comparison of Key Indicators between 2002 and 2006 I. Project Data Sheet Form 34 A. EVALUATION TEAM MEMBERS List of Final Evaluation participants SN Name Position & Organization 1 Dr. Pierre-Marie Metangmo Health Coordinator, Plan IH 2 Dr. Sun Lal Thapa National IMCI Coordinator, Child Health Division, Ministry of Health and Population 3 Dr. Ram Lakhan Shah District Health Officer, Bara 4 Mr. Parsuram Shrestha District Public Health Officer, Bara 5 Dr. Chandra Kumar Sen District Program Coordinator – Plan Nepal Rautahat/Bara PU 6 Arjun Nepal Coordinator, NFHP, Bara 7 Dr. Kedar Baral National Health Coordinator, Plan Nepal Country Office 8 Bhagawan Das Shrestha CS Project Coordinator 9 Shusil Joshi Research Officer, Plan Nepal Country Office 10 Subarna Raj Gurung Training Coordinator, Plan Nepal, CS Project 11 Dipak Dahal Health Information System Coordinator, Plan Nepal, CS Office 12 Lava Raj Timilsina CS Project Coordinator, Care Nepal, Doti 13 Sandra Wilcox External Consultant 35 B. EVALUATION ASSESSMENT METHODOLOGY On the first day the consultant and the headquarters Child Survival Coordinator, Dr Pierre Marie Metangmo, met with the Plan Health Program Coordinator (Dr Kedar Baral) in Kathmandu to review schedules and information and plan activities for the evaluation in Bara District. Following this, during a one day workshop in Bara core team members worked with staff from Plan, Ministry of Health, and other partners to plan the evaluation, develop data collection instruments, and discuss pertinent issues. Dr. Kedar Baral presented findings of a quantitative assessment of project progress through June 2006 (based on lot quality surveys conducted twice every year). Additional information was presented by Bhagawan Das Shrestha, the Project Coordinator. Workshop participants then regrouped into three field teams (led by Dr. Kedar Baral, Dr. Ram Lakhan Shah, and Dr. Sun Lal Thapa) to visit health facilities, interview health personnel, and carry out focus group discussions with women and children in project communities over a two-day period. Findings were compiled in the evenings and shared with the consultant. The consultant conducted a group discussion with Plan Community Health Officers and Assistant Community Health Officers to understand the challenges they face in their work and hear their recommendations and suggestions for the future. In addition, she interviewed staff from Plan, Ministry of Health, and other partners in the district and in Kathmandu. Kedar Baral, Pierre Marie Metangmo and the consultant conducted two debriefing sessions, one for Plan and another one for USAID Nepal. In both these meetings, the consultant also received input from Plan and USAID regarding concerns and recommendations for future CS related activity in Nepal. 36 C. List of Persons Interviewed and Contacted Bagwan SHP– 2 August 2006 1. Sazad Ansari – Sub Health post Incharge 2. Mrs Kabita Aryal (teacher), Nijgadh – 2 August 2006 Community Based Maternity Home committee members 1) Kishor Nepal – Vice Chairperson 2) Pravat Sigdel – Joint Secretary 3) Bishnu Lama – Member FCHV (6) : Shreemaya Thada Magar Mother’s groups’ chairperson: Tara Bhatta VDC: Kuduwa – 3 August 2006 1. Mrs Radhika Devi Yadav – FCHV 2. Gita kumari Sahani – MCHV 3. Ram.K .Yadav – AHW 4. Mahavir Mukhiya - community key informant 5. Ganga Dev Yadav - community key informant Simara PHC – 3 August 2006 1) Naina Pathak (Staff Nurse) – Acting Incharge 2) Ashok Jaiswal – Health Assistant Dumarwana SHP – 3 August 2006 1) Dev Prasad Pyakurel (AHW) – In-charge 2) Sabitri Adhikari – VHW 3) Mina Begam – FCHV, Ward # 8 FCHV (6) – Bhawani Pokharel Mother’s Group’s chairperson & TBA (6) – Kamala Adhikari Fattepur – 9 (Amadar Tole) – 3 August 2006 1) Chairperson of Child Club (Jan Sewa Bal Club) – Yosodha 2) Secretary – Akur VDC: Basatpur – 4 August 2006 1. Shaily Devi Saha, FCHV 2. Laxman Prasad Yadav – VHW 3. Anil Kumar Pandey - AHW 4. Bacha Singh - health facility committee member 5. Manjo Kumar Yadav - health facility committee member 6. Atiullah Ansari - community key informant 37 Rampurwa HP – 4 August 2006 Incharge – Md. Sabir Health Management Committee chairperson – Ram Prasad Chaudhary FCHV (6) – Rupi Chaudhary Mother’s groups’ chairperson – Sabita Chaudhary Uttar Jhitkaiya SHP – 4 August 2006 1. Birendra Kumar Mahato – VHW 2. Paspat Kunwar – AHW 3. Surendra Ram - community key informant VDC: Bhaluhibharbhaliya – 4 August 2006 1. Sona Devi Baitha – CHV 2. Bidhya Laxmi Maharjan – MCHW 3. Ram Bidhaya Prasad Kusuwaha SHP Incharge 4. Mr. Ram Chandra Pd. Kanu – key informant Sapahi SHP – 4 August 2006 1. Hari Maya Tamang - FCHV 2. Krishna Kumari Ray - MCHW 3. Parasnath Chaudhary – AHW 4. Mr. Krishna Chandra Neupnae – key informant 38 D. Training completed by Project Plan Nepal Rautahat/Bara Program Unit Child Survival Project Kalaiya Major training and workshop conducted by Child Survival Project from 2001 to 2006 (Health Facilities and community) Sn T/W Name of Event Level Duration # of Batch # of Participants Date 1 T GTOT HF staff 5 Days 5 95 Feb - May 02 2 WT FP/MNC Training HF staff 4 Days 1 18 Mar-02 3 T FP/MNC Training SHP staff 7 days 5 105 Apr-May 02 4 T FP/MNC Training TBAs 12 Days 9 187 Apr - May 02 5 T IMCI Training PHC/HP/SHP 10 days 9 149 Apr - Jul 02 6 T FP/MNC Training FCHVs 7 Days 39 725 May - June 02 7 T FP/MNC Training VHW/MCHW 7 days 5 126 May - June 02 8 T IMCI Training Non Technical 5 Days 1 16 July - Aug 02 9 T Transformation for Health Training HF staff 5 days 1 3 Oct-02 10 T IMCI Training Follow up PHC/HP/SHP 2 Days 98 149 Nov- Dec 02 11 T FP/MNC Training FCHV/TBAs 2 Days 48 1142 Feb - March 03 12 T IMCI Training VHW/MCHW 7 Days 6 168 Aug -Sep 03 13 T FP/MNC Refresher Training FCHV/TBAs 2 Days 47 1018 Sep-03 14 T TBA Basic Training (New) TBAs 13 days 1 23 Sep-03 15 T Infection Control Training Helper 3 Days 4 64 Oct-03 16 T IMCI Refresher Training FCHV 3 Days 47 884 Nov -Dec 03 17 T Street drama Training Child Club Member 3 day 4 109 Dec-03 18 T CDP TOT HF staff 4 Days 1 23 Mar-04 39 Sn T/W Name of Event Level Duration # of Batch # of Participants Date 19 T CDP Training SHP staff 4 Days 7 146 Mar - Apr 04 20 T HMIS Training Clerk 4 Days 1 12 Mar-04 21 T IMCI Refresher Training FCHVs 2 Days 47 884 May - June 04 22 T COFP counseling training MCHW 4 Days 2 57 Dec-04 23 T Healthy school Promotion Training School Children 3 Days 4 360 Dec 03- Jan 04 24 T ROC Training Child Club Member 5 Days 18 602 Dec 03 - Feb 04 25 T Healthy school Promotion Training School Children 3 Days 22 662 Jan-05 26 W Rational Use of Drug Drug Retailers 1 Day 1 28 Jan-05 27 T CDP Training Mgmt Committee 3 Day 49 1076 Feb - March 05 28 T FP/MNC Training FCHV/TBAs 2 Days 48 1048 Mar-05 29 T IMCI Refresher Training FCHVs 2 Days 47 884 May - June 05 30 T LQAS Training District Supervisor 9 Days 1 10 Jul-05 31 T IMCI Refresher Training FCHVs 2 Days 48 884 Sep-05 32 W Iodized salt Promotion Orientation Shop Keeper 1 Day 8 209 Sep-05 33 T ROC Training Child Club Member 5 Days 13 396 Nov - Dec 05 34 T FP/MNC Training FCHV/TBAs 2 Days 47 1048 Dec-05 35 W Iron Intensification Workshop HF staff 1 day 2 36 Dec-05 36 TW School Health Promotion Workshop School Children 3 days 19 573 Dec-05 37 T FCHV Refresher training on New Approach (DTOT) HF staff 7 days 1 22 Jan-06 38 T Iron Intensification Training FCHVs 2 Days 15 884 Jan- Mar 06 39 T Iron Intensification Training VHW/MCHW 2 Days 15 170 Jan-Mar 06 40 T FP/MNC Training FCHV/TBAs 2 Days 47 1048 Feb - Mar 06 41 T IMCI Training (Transferred staff) HF staff 3 Days 2 42 Mar-06 42 T FCHV Basic Training FCHVs 9 days 5 121 Apr - May 06 40 Sn T/W Name of Event Level Duration # of Batch # of Participants Date 43 T FCHV Refresher New Approach FCHVs 5 Days 44 766 Apr-06 44 T HMIS Training SHP Incharge 3 Days 6 138 April - June 06 45 T FCHV Refresher New Approach FCHVs 5 days 44 766 May -June 06 46 T LQAS Training District Supervisor 9 Days 1 10 Jun-06 47 T ROC Refresher Child Club Member 2 Days 30 900 Nov 05 - May 06 48 T HMIS Training HF staff 5 Days 1 22 Dec 05- Jan 06 49 W Iron Intensification Workshop SHP staff 1 Day 8 114 Jan-06 50 W CSSA workshop HF staff / Community 1 Day 1 20 Jul-06 51 W Behave Training Project Staff 5 Days 1 2 March 2006 T: Training HMIS: Health Management Information System W: Workshop HF: Health Facility TW: Training Workshop SHP: Sub-Health Post FP/MNC: Family Planning/Maternal and Newborn Care ROC: Rights of Child GTOT: General Training of Trainers CDP: Community Drug Program IMCI: Integrated Management of Childhood Illness COFP: Comprehensive Family Planning LQAS: Lot Quality Assurance Sampling TBA: Traditional Birth Attendant PRA: Participatory Rural Appraisal TOT: Training of Trainers FCHV: Female Community Health Volunteer CSSA: Child Survival Sustainable Assessment 41 E. Pregnant Women’s Group Impact Study Summary Report OF IMPACT ASSESSMENT OF PREGNANT WOMEN GROUP STRATEGY PLAN NEPAL, CHILD SURVIVAL PROJECT, BARA DISTRICT, NEPAL (2001-2006) 42 ACKNOWLEDGEMENT Impact assessment of Pregnant Group Strategy in terms of Maternal and Infant mortality indicators was a daunting task in view of obtaining sufficient sample size of the study subjects to represent the whole district. Therefore 49 VDCs representing half of the population of the district has to be covered to get precise estimates. This task would not have been possible without the enthusiasm and determination of Impact Assessment Team lead and supervised by the National Health Coordinator of PLAN himself. The hard works of 32 enumerators and supervisors working in the field have my highest commendation. It were they who toiled in dangerous and uncertain field setting during the period of civil war between the state and Maoist, and the high tide’s of people’s movement of 2062/2063. I also would like to express my thanks to the staff of Plan Nepal, Bara District who made the field survey every way enjoyable, and the community level health workers and volunteers who helped to improvise information that were not available in the register. Last but not the least; I am grateful to the cooperation of respondent mothers without whom this study would not have been possible. 43 IMPACT ASSESSMENT TEAM: Dr. Mahesh Maskey MBBS, MPH, DSc, Epidemiologist - Consultant Contribution: Selection of method, literature review, sample size calculation, tools development, training design of the study team, training of Supervisors and enumerators, pre-testing of tools, field supervision at early stage of the study and report preparation. Dr. Kedar Baral - National Health Coordinator, Plan Nepal Country Office Contribution: Inception of study, development of terms of reference including objectives, literature review, overall management of study and team, management of implementation of study, field supervision, data management and collaborated with consultant for selection of method, sample size calculation, tools development, training design of the study team, training of supervisors and enumerators, pre-testing of tools and report preparation. Bhagawan Das Shrestha - Child Survival Project Coordinator, Plan Nepal Contribution: Participated on the discussion of the study from early stage, day to day management of team including logistics at field level and data management, field supervision and custodian of data at field. Shusil Joshi - Research Officer, Plan Nepal Country Office Contribution: Tools development, develop strategy for data analysis and software development for data management. Dipak Dahal - Health Information System Coordinator, Plan Nepal, Child Survival Project Contribution: Tools development, field supervision, ensuring correctness of information while submitting completed forms by encoders, supervised data encoding and quality assurance, data analysis including use of software and collaborated with Shusil for development of strategy for data analysis and software development and data management. Subarna Raj Gurung - Training Coordinator Contribution: Field supervision, training and logistics management In addition, following members contributed for field level work of the study: Biswo Nath Khatri, Krishna Kumar Shrestha, Sudarshan Paudel, Nitu Acharya, Yam Bahadur Thapa, Krishna Bahadur Achhami, Yogendra Giri, Sajit Kumar Adhikari, Hari Dev Sah, Saraswoti Kharel, Rajendra Pd. Sah, Sharmila Budhathoki, Raj Kishor Chaudhary, Md. Firaj, Bijaya Sah, Shiv Dayal Yadav, Pawan Kumar Rai, Indira Mainali, Sharmila Khadka, Ramesh Sah, Devendra Kumar Sen, Jay Mangal Thakur, Hari Shankar Raut, Ezaj Ansari, Renu Chaudhary, Prem Chandra Jaiswal, Diwakar Mishra 44 ACRONYMS: CI - Confidence Interval DIP - Detailed Implementation Plan ENMR - Early Neonatal Mortality Rate ICD-10 - International Classification of Disease (10th revision) IMR - Infant Mortality Rate LB - Live Birth MMR - Maternal Mortality Rate MOH - Ministry of Health NGO - Non-Governmental Organization NMR - Neonatal Mortality Rate PMR Perinatal Mortality Rate PNMR - Post-neonatal Mortality Rate PWG - Pregnant Women’s Group SB - Still Birth SBR - Still Birth Rate TB - Total Birth VDC - Village Development Committee 45 IMPACT ASSESSMENT OF PREGNANT MOTHER’S GROUP STRATEGY PLAN NEPAL, CHILD SURVIVAL PROJECT, BARA DISTRICT, NEPAL INTRODUCTION: Plan Nepal is completing a five-year child survival project in partnership with the Ministry of Health (MOH) and Non-Governmental Organizations (NGOs) in Bara District, Nepal. The goal of the project is to assist Ministry of Health in its effort to improve the health status of children under five years of age and of women of reproductive age, in accordance with the approach underlined in the Detailed Implementation Plan (DIP, 2002). The project has endeavored to address major health problems of mothers, neonates and infants by adopting target group reaching strategy of Pregnant Women’s Group (PWG). The consenting pregnant women were enrolled in groups that received the integrated package of health education and care during pregnancy, delivery and child rearing phase. A substantial number of women were also outside the PWG program, thus making the situation appropriate for a comparative study for impact assessment of PWG program. While the midterm evaluation of the project carried in April, 2004, mainly focused on the process indicators such as practicing health behavior and seeking medical care from trained providers, increased access to health education, quality care and essential medicines, community mobilizations, communication and capacity building for improved community health care, the evaluation at the end of the project aimed at measuring the outcome indicators of maternal and child health. The PWG group strategy has drawn interest from stakeholders, especially MOH departments. And it was crucial for Plan Nepal to evaluate and measure the impact of PWG strategy in terms of childhood and maternal mortality indicators. The expertise of a consultant epidemiologist was sought to carry out the evaluation study with following objectives and expected outcome. OBJECTIVES: ƒ To examine relationship between PWG members and non-members (exposure) on mortality of mother and children especially on perinatal, neonatal, infant and maternal mortality (outcome). ƒ To draw epidemiological conclusion whether PWG approach is an appropriate strategy for the reduction of maternal and infant mortality in settings similar to Bara district of Nepal. EXPECTED OUTCOME: ƒ An epidemiological study report that would provide the comparison of perinatal, neonatal, infant and maternal mortality between PWG members and non-members METHOD: Several important documents of Plan Nepal health policies and projects were reviewed to understand the context of PWG approach. These included Detail Implementation Plan 46 describing the Child Survival XIII Cost Extension Project that endorsed PWG formation at Bara District. Other documents reviewed were Child Survival Project (FA)-A-00-97- 00042-00) and Mid-term Evaluation (April 2004). The study design was cross sectional-comparative study and the data were collected by applying “Motherhood Method”, - a variant of Participatory community Survey (Maskey and DesChene, 2005, Maskey, 2005). A total of 32 field supervisors and enumerators were given orientation and training, workshop in the field setting for two days. The pre￾testing of tools was done in nearby community. The data was analyzed using EPI info 2002 software. RESULT: The survey of 49 VDCs for birth and death of children and mothers in two years period (Shrawan 2060-Ashad 2062) yielded 14,919 live births and 246 stillbirths. The number of total birth was 15,165. Number of maternal death as defined in ICD-10 in that period was 67. Total number of Infant deaths was 718 of which 496 (69%) died in neonatal period and 437 (61%) died in early neonatal period (see table-1). Table-1: Distribution of birth and deaths PWG and non-PWG members Total PWG Non-PWG Live Birth 14,919 4,334 10,585 Still Birth 246 41 205 Total Birth 15,165 4,375 10,780 Maternal Death 67 12 55 Infant Death 718 108 610 Neonatal Death 496 81 415 Early Neonatal Death 437 72 365 Perinatal Death 687 113 574 Post Neonatal Death 59 9 50 There were 4,334 live births and 41 stillbirths of mothers who were the members of PWG and 10,585 live births and 205 stillbirths of non-members of PWG. Likewise, 12 mothers died among PWG members while 55 died in non-PWG members. Accordingly, there were 108 infant, 81 neonatal and 72 early neonatal deaths in PWG, while in non-PWG members infant death was 610, neonatal 415 and early neonatal death was 365. It was found that 69% of infant mortality comprised neonatal deaths and 61% of infant deaths occurred in early neonatal period. Moreover, 88% neonatal mortality occurred in first seven days of birth. 47 Table-2: Mortality indices of Bara district compared with national estimates Mortality Rates PWG Non PWG Total National IMR /1000 LB 24.9 57.6 48.1 61 NMR/ 1000 LB 18.7 39.2 33.2 36 ENMR/ 1000 LB 16.6 34.5 29.3 PMR/1000 TB 25.8 53.2 45.3 SB/1000 TB 9.4 19.0 16.2 PNMR/1000 LB 2.1 4.7 3.9 MMR/100000 276.9 519.6 449.1 539 Mortality rates for Bara district were computed and found that maternal mortality rate (MMR) was 449.1/100000 live birth (LB), Infant mortality rate (IMR) was 48.1/1000 LB, Neonatal mortality rate (NMR) 33.2/1000 LB, Post-neonatal mortality rate (PNMR) 3.9/1000 LB, Early Neonatal mortality rate was 29.3/1000 LB. The perinatal mortality rate (PMR) and stillbirth rate (SBR) was calculated with total births in the denominator. It was 45.3/1000 TB and 16.2/1000 TB respectively. These mortality rates were also calculated for PWG and non-PWG members (Table-2). Mortality rates of PWG and non-PWG members were compared using odds ratio as measure of association and confidence interval and p-value as measure of statistical precision. It was found that the odds of maternal deaths in non-PWG members were 1.88 (95% CI-0.98-3.70) times higher than the mothers in PWG. Similarly, the odds of infant deaths were 2.39 (95% CI-1.93-2.96) times higher and the odds of stillbirth 1.74 (95% CI-1.32-2.31) times higher in the non-PWG member as compared to the PWG members. The odds of perinatal mortality rate in non-PWG members was 2.12 (95% CI-1.72-2.62) times higher, neonatal mortality rate 2.14 (95% CI-1.67-2.75) times higher and early neonatal mortality was 2.11 (95% CI-1.63-2.75) times higher than the PWG members. The differences in terms of odds ratios of mortality rates between the two groups were found statistically highly significant. Except for maternal mortality (p <.04) and post neonatal mortality (p <.0193) the differences in all other mortalities had a p-value less than .0001 (Table-3). Table–3: Comparison of Mortality rates in PWG and non-PWG members Mortality Rates PWG Non-PWG Odds ratio (95% CI) p-value IMR /1000 LB 24.9 57.6 2.39 (1.93 – 2.96) <.0001 NMR/ 1000LB 18.7 39.2 2.14 (1.67 – 2.75) <.0001 ENMR/ 1000LB 16.6 34.5 2.11 (1.63 – 2.75) <.0001 PMR/1000 TB 25.8 53.2 2.12 (1.72 – 2.62) <.0001 SB/1000 TB 9.4 19.0 1.74 (1.32 – 2.31) <.0001 PNMR/1000 LB 2.1 4.7 2.28 (1.08 – 4.97) <.0193 MMR/100000LB 276.9 519.6 1.88 (0.98 – 3.70) <.0441 48 DISCUSSION AND CONCLUSION: The study has demonstrated with high precision that PWG members have reduced risk (about 50% less) of dying during pregnancy, childbirth and puerperium as compared to the non-PWG members. Their children also have similar lower risk of dying during perinatal and infancy periods. The evidence generated by the study is supportive of positive impact of PWG strategy in the reduction maternal and child mortality in Bara district. This strategy may be appropriate for other rural Terai districts of Nepal as well. The large proportion of infant deaths being early neonatal (61%) and only about 8% deaths being post neonatal is a pointer to the difficult challenge of further reduction of infant mortality. Vaccination, drastic reduction in neonatal tetanus deaths and other childcare services like pneumonia case management, use of ORS during diarrhea have most likely reduced late and post neonatal mortality. Such efforts have less impact on early neonatal and perinatal mortality. To save lives and reduce morbidity in these periods, the intervention has to be made during pregnancy, even earlier, and management of neonatal problem. Both neonatal and early neonatal deaths were reduced by more than 50% in the PWG members that signifies for further requirement of prevention and management of problem in early months of life. Child Survival project in Bara district has made an impressive impact on health of the mother and child in terms of reduction in mortality rates. The PWG members have significantly lower mortality risks of their own lives and the lives of their babies as compared to the mothers who were not members of PWG. PWG strategy works very closely with government health care services and deserves serious consideration as an appropriate strategy for reduction in maternal and infant deaths in other lowland rural districts of Nepal. 49 F. Interview questionnaires Community-based Volunteers 1. How long have you been working as a volunteer? 2. What training was most helpful on your day to day work? (CB-IMCI) 3. How has the training helped you in your work? 4. Who comes to support you in completing your activities? How often do you receive support? 5. How have their visits been helpful to you? 6. When did people from community last visit you to get any health services or commodities? Tell me about the visit. 7. How does the community support you in your work? 8. What problems do people face in using your services? 9. What problems do you face in your day-to-day work? How are these resolved? 10. How has your ability to deliver services improved in the last 2 years? 11. What materials do you have to educate people in the community? How do you use them? 12. What commodities do you sell? How much do you charge for them? 13. Who decided that these commodities could be sold in the community? 14. How do you get supplies (such as oral rehydration solution packets, iron tablets, vitamin A, family planning supplies)? 15. How do community members feel about the selling of supplies? 16. How often do you meet as a group and how do you plan your activities? 17. How does the health facility support you in your work? 18. How does the Village Development Committee support you in your work? 19. How can the support you receive be improved? 20. What changes have occurred in the health status of the community in your area in the last two years? 21. How do you plan to continue your work after the Child Survival project finishes? 22. How can Child Survival project activities be improved in the future? VHW/MCHW 1. How long have you been working as a health worker? 2. What training was most helpful on your day to day work? How?(CB-IMCI) 3. How has the training helped you in your work? 4. What are your activities? 5. How often do you provide support to community-based volunteers? What are the areas of support (mothers' groups, supplies etc)? 6. How many mothers' groups and pregnant women's groups are there in your village development committee? How often do you attend their meetings? 7. How has your ability to deliver services improved in last 2 years? 8. What educational materials have you received? How do you use them to educate community members? 9. How would you like to improve your services? 50 Chief of Health Facility 1. In what heath related activities have you involved the community? 2. Describe the participation of community members in these activities. 3. What barriers have you faced in mobilizing communities? Give an example of how you solved them. 4. How effective is the project's behavior change communication approach? 5. How has the health facility improved over the last two years? Give an example. 6. How will Child Survival project activities continue after the project finishes? 7. What tools do you use for training needs assessment? 8. What types of method do you use for training? 9. How do you follow-up with participants after training? 10. How frequently do you supervise staff members who report to you? 11. What methods do you use during supervision? 12. Please give an example of a work-related problem faced by your staff during the last month. Tell me how it was solved. 13. What health-related information do you and your staff collect? How frequently are these collected? How are they analyzed? 14. How do you use the information to track the effectiveness of the project? 15. How is the information helpful in decision making? 16. What training was most useful? 17. Have you received training in community-based integrated management of childhood illness? 18. How are you using the skills you learned during the training? 19. What are the benefits of community-based integrated management of childhood illness? 20. How can community-based integrated management of childhood illness be improved? 21. How is community-based integrated management of childhood illness helping other national health programs? Health Facility Committee member 1. When was the committee formed? 2. How many members in the committee? 3. How frequently are you meeting? 4. When was the last meeting? 5. What was discussed and decided? 6. Do you have segregation duty (work division) or not? 7. How you contributing for the health program 8. Do you have CDP program? 9. When was the CDP program started? 10. Do you have banking operation or not? 11. How much money do you have now? 12. Was there a stock out in the last 3 months? 51 13. How do you decide when to purchase new drug? 14. Do you have the problem to implementing the CDP program? 15. How would you improve the health program including CDP? Mother's group 1. When and how was the mother's group formed? 2. How often are the meetings? 3. Who facilitate the meeting? 4. When was the last meeting? 5. What was discussed in the meeting? 6. How do you help the FCHV? 7. How is the FCHV helping the group? 8. Are their other interest groups in the ward like pregnant women and child club etc.? 9. If yes, how are they collaborating to other groups? 10. Are you happy with the health service provider? 11. What are the suggestions for better health service? Pregnant Women's Group/Child Club and other groups 1. When and how was the pregnant mother's group formed? 2. How often are the meetings? 3. Who facilitate the meeting? 4. When was the last meeting? 5. What was discussed in the meeting? 6. How do you help the FCHV? 7. How does the FCHV help the group? 8. What is the advantage of being in-group? 9. Are you happy with the health service provider? 10. What are the suggestions for better health service? Community key informant 1. Do you know the child survival project? 2. If yes, what are the major actives in your village? 3. Who provides the pneumonia cases treatment in your ward? 4. Where the EPI out reach side? 5. Who and when was EPI service provided? 6. Have you been involved in any kind of health activities? 7. If yes, what are they? 8. Do you know who is the FCHV in your ward and what is her role? 9. Do you have any suggestion for the further improvement in the health services? 52 G. LQAS Results – Comparison of coverage for 8 time periods Table 1: Comparison of coverage proportion for key indicators collected between seven-time period (first, second, third, fourth, fifth, sixth, seventh, eighth LQAS - From October 2001 to Jun 2006) SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI BREAST FEEDING AND CHILD NUTRITION INDICATORS 1 M1 Breastfeeding Initiation Percent of children aged 0-11 months who are breastfed with in the first hour after birth 9 4.87 18 6.58 29 7.74 32 7.90 36 8.16 40 8.32 48 8.49 66 8.0 4 2 M1 Exclusive Breast feeding Rate Percent of infants aged 0-5 months who were fed breastfed milk only in the last 24 hours 62 8.26 95 5.15 83 8.94 90 6.73 75 11.58 93 6.03 80 10.41 100 3 M1 Complementory Feeding Rate Percent of infant aged 6-9 months who received breast milk and solid foods in the last 24 hours 73 15.82 64 15.69 82 10.84 81 13.91 81 9.70 87 9.54 88 9.18 96 4.81 4 M2 Continued breast feding Percent of children aged 20-23 months who are still breast feeding 77 14.72 87 11.80 74 14.48 78 15.68 89 9.18 95 9.55 88 11.14 85 12.2 5 M2 Vitamin "A" Coverage Percent of Children aged 6-23 months who received a vitamin A does in the last six months 91 4.17 88 4.68 98 2.07 98 2.52 97 2.90 99 1.47 95 3.80 99 1.47 CHILDHOOD IMMUNIZATION INDICATORS 6 M2 Possession of vaccination Card Percent of Children aged 12-23 months who have a Vaccination Card 19 6.64 25 7.34 36 8.16 44 8.43 51 8.50 59 8.35 59 8.35 74 7.41 7 M2 EPI Access Percent of children aged 12-23 months who received DPT 1 16 6.20 18 6.54 35 8.12 43 8.41 50 8.50 59 8.35 56 8.44 71 7.68 8 M2 RAPID Catch Indicator: Measles Vaccination Coverage Percent of children aged 12-23 months who received measles vaccine 11 5.22 14 5.81 29 7.68 38 8.26 41 8.35 52 8.49 53 8.49 72 7.62 53 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI 9 M2 Droup Out Rate Percent of drop out- rates between DPT1 and DPT 3 14 14.97 17 14.91 2 4.04 10 7.84 6 5.59 5 4.83 5 5.15 3 3.38 10 M2 Rapid Catch Indicator EPI Coverage Percent of children aged 12-23 months who received BCG, DPT3, OPV3 and measles vaccines before the first birthday 10 5.05 11 5.38 29 7.68 34 8.04 40 8.32 52 8.49 50 8.50 67 8.00 11 M2 EPI Coverage II (Liberal Criteriaon ) Percent of children aged 12-23 months who received OPV 3 14 5.95 16 6.20 35 8.08 37 8.20 48 8.49 56 8.43 55 8.46 72 7.62 SICK CHILD 12 M1 & M2 Maternal Knowledge of child danger sign Percent of mothers of children aged 0-23 months who know at least THREE signs of childhood illness that indicate the need for treatment 73 5.34 62 5.83 80 4.77 83 4.51 90 3.57 95 2.68 97 1.92 98 1.78 DIARRHEA INDICATORS 13 M1 & M2 Diarrhea prevalence Percent of children aged 0-23 months with diarrhea in the last two weeks 20 4.83 18 4.58 17 4.51 20 4.77 17 4.54 17 4.51 22 4.96 21 4.87 14 M1 & M2 ORT use during a Diarrhea Episode Percent of children aged 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/ or recommended home fluids (RHF) 16 9.78 13 9.54 36 13.99 35 12.93 46 14.39 31 13.53 57 12.75 60 12.9 15 M1 & M2 Increased breastfeed During a Diarrhea Episode Percent of children aged 0-23 who received breastfed same amount or more during diarrhea in last two weeks. 62 12.84 62 13.90 82 11.17 87 9.28 87 9.73 89 9.18 90 7.84 95 6.00 54 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI 16 M1 & M2 Increased drink during a diarrhea Episode Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more drink / fluid during the illness 24 11.23 34 13.55 71 13.24 63 13.09 83 10.95 76 12.56 91 7.22 93 6.86 17 M1 & M2 Increased food during a diarrhea Episode Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount or more food during the illness 27 11.77 28 12.79 62 14.17 54 13.55 70 13.30 67 13.77 81 10.09 95 6.00 18 M1 & M2 Care–seeking for Diarrhea Percent of Children aged 0-23 months with diarrhea in the last two weeks whose mothers Sought outside advice or treatment for the illness 76 11.23 72 12.79 80 11.69 81 10.71 80 11.46 80 11.69 71 11.71 91 7.60 19 M2 Maternal Competency in ORS Preparation Percent of mothers who can correctly prepare ORS 34 8.60 39 8.29 64 8.16 71 7.74 74 7.41 67 8.00 75 7.34 84 6.20 20 M2 Maternal Hand Washing before Food Preparation Percent of mothers who usually wash their hands with soap or ash before food preparation. 23 7.10 22 7.02 40 8.32 64 8.16 61 8.29 77 7.19 81 6.64 92 4.68 21 M2 Maternal Hand Washing before Food Preparation before feeding /after attending to a child who has defecated Percent of mothers who usually wash their hands with soap or ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated. 5 3.80 4 3.23 18 6.54 26 7.41 33 8.00 50 8.50 47 8.49 78 7.02 ARI IDICATOR 22 M1 & M2 ARI Care -seeking Percent of Children aged 0-23 months with cough and fast / difficult breathing in the last two weeks who were taken to a health facility or received treatment. 79 9.99 73 10.82 75 9.89 61 9.71 45 9.39 51 10.44 93 8.16 80 14.3 55 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI PRENATAL CARE INDICATORS 23 M1 Maternal Health Card Presentation Percent of mothers with a maternal card (Card-confirmed) for the youngest child less than 12 months of age 17 6.43 22 7.02 26 7.48 24 7.26 30 7.79 21 6.93 31 7.85 65 8.12 24 M1 Tetanus Toxoid Coverage Percent of mothers who received at least TWO tetanus toxoid injections (Card confirmed) before the birth of the youngest child less than 12 months of age. 13 5.67 21 6.93 18 6.80 20 7.16 24 7.44 22 7.26 32 8.18 63 8.20 25 M1 Prenatal Care Coverage Percent of mothers who had at least ONE prenatal visit prior to the birth of her youngest child less than 12 months of age 45 8.46 53 8.48 56 8.43 66 8.04 68 7.95 65 8.08 74 7.48 78 7.02 26 M1 Iron Supplimentation Coverage Percent of mothers who received /brought iron supplements while pregnant with the youngest child less than 12 months of age. 36 13.30 53 8.48 74 7.41 77 7.19 83 6.31 79 6.93 87 5.67 93 4.27 PLACE OF DELIVERY AND DELIVERY ATTENDED 27 M1 Delivery by skilled Health Personnel Percent of children aged 0-11 months whose delivery was attended by a skilled health personal upto TBA level 32 7.95 47 8.49 53 8.48 68 7.90 65 8.08 63 8.20 68 7.95 77 7.19 28 M1 Delivery by skilled Health Personnel Percent of children aged 0-11 months whose delivery was attended by a skilled health personal upto MCHW level 33 8.00 30 7.79 40 8.32 42 8.39 56 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI 29 M1 Clean Cord Care Percent of children aged 0-11 months whose delivery involved use of a clean birth kit or whose cord was cut with a new razor 96 3.23 95 3.53 30 M1 Clean Cord Care Percent of children aged 0-11 months whose delivery involved use of a clean birth kit 30 7.79 51 8.50 53 8.49 78 7.02 73 7.55 75 7.34 31 M1 Immediate Breast Feeding Percent of children aged 0-11 months who were immediately breastfed with the mother immediately after birth. 2 2.52 14 5.95 32 M1 Placement at Birth Percent of children aged 0-11 months who were placed with the mother immediately after birth 25 7.34 28 7.62 51 8.50 66 8.04 74 7.48 70 7.79 74 7.48 82 6.54 POSTPARTUM CARE 33 M1 Postpartum Contact Percent of mother who had at least ONE postpartum check-up 11 5.38 14 5.81 15 6.07 31 7.85 41 8.35 40 8.32 48 8.49 63 8.20 34 M1 Knowledge of maternal Danger Signs Percent of mothers able to report at least TWO known maternal danger signs during the postpartum period 41 8.37 52 8.49 35 M1 Knowledge of Neonatal Danger Signs Percent of mothers able to report at least THREE known neonatal danger signs 91 4.87 90 5.05 95 3.80 94 4.04 94 4.04 98 2.07 36 M1 Knowledge of Neonatal Danger Signs Percent of mothers able to report at least TWO known neonatal danger signs 71 7.68 87 5.67 96 3.23 98 2.52 98 2.07 98 2.52 97 2.90 98 2.07 57 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI 37 M1 Maternal Vitamin A supplementation Percent of mothers who received a Vitamin A dose during the first six weeks after delivery 16 6.20 17 6.43 34 8.04 42 8.39 50 8.50 59 8.35 67 8.00 80 6.74 38 M1 Maternal iron supplementation Percent of mothers who received at least 1 month iron tablets during the first two months after delivery 10 5.05 28 7.62 42 8.39 52 8.49 45 8.46 48 8.49 62 8.26 75 7.34 CHILD SPACING 39 M3 Contraceptive Use Among Mothers Who Want to limit or space births Percent of non pregnant mothers who desire no more children in the next two years or are not sure, who are using a modern method of child spacing 24 7.54 32 7.90 44 8.44 43 8.41 46 8.47 55 8.46 40 8.32 66 8.04 40 M3 Knowledge of source of Child spacing methods Percent of mothers who report at least one place where she can obtain a method of child spacing 54 8.47 71 7.74 81 6.64 93 4.27 97 2.90 98 2.52 98 2.07 100 41 M3 Adequate birth interval between surviving children Percent of children aged 0-23 months who were born at least 24 months after the previous surviving child 58 11.10 58 10.96 56 10.74 68 11.37 59 12.34 67 10.78 76 9.42 69 12.9 42 M3 Adequate Birth interval Between youngest Surviving Children (Less Stringent Criteria) Percent of children aged 0-23 months who were born at least 36 months after the previous surviving child 12 7.26 13 7.42 21 8.77 23 10.24 41 12.34 38 11.15 32 10.26 14 9.80 KNOWLEDGE OF DANGER SIGNS DURING PREGNANCY, POSTNATAL AND NEW BORN CHILD 43 M1 Danger signs/ symptoms during pregnancy Percent of mothers (15-49 years) who know at least TWO danger signs/symptoms during pregnancy 29 7.68 41 8.35 87 5.67 90 5.05 95 3.80 95 3.53 95 3.53 100 58 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan’05 Confidence Interval ± CI LQAS Jul’05 Confidence Interval ± CI LQAS Jun'06 Confide Interva CI 44 M1 Danger signs/ symptoms during pregnancy Percent of mothers (15-49 years) who know at least THREE danger signs/symptoms during pregnancy 26 7.48 41 8.35 69 7.85 74 7.48 85 6.07 84 6.20 92 4.48 99 1.47 45 M1 Danger signs after delivery Percent of mothers who knows at least TWO danger signs/ symptoms of after delivery 41 8.37 52 8.49 80 6.74 86 5.81 92 4.48 93 4.27 98 2.07 46 M1 Danger signs after delivery Percent of mothers who knows at least THREE danger signs/ symptoms of after delivery 11 5.22 62 8.23 59 8.37 75 7.34 76 7.26 89 5.22 95 3.80 47 M1 Danger signs of new born Percent of mothers who know at least TWO danger sign of new born 71 7.68 87 5.67 96 3.23 98 2.52 98 2.07 98 2.52 97 2.90 98 2.07 48 M1 Danger signs of new born Percent of mothers who know at least THREE danger sign of new born 37 8.20 62 8.23 91 4.87 90 5.05 95 3.80 94 4.04 94 4.04 98 2.07 DANGER SIGNS OF PNEUMONIA AND DIARRHEA 49 M2 Danger signs/ symptoms of pneumonia Percent of mothers who know at least THREE danger signs/ symptoms of pneumonia 15 6.07 39 8.29 71 7.74 85 6.07 90 5.05 90 5.05 92 4.68 98 2.52 50 M2 Danger signs / symptoms of diarrhea/ dysentery Percent of mothers who know at least THREE danger sign of diarrhea / dysentery 14 5.95 17 6.31 50 8.50 62 8.26 69 7.85 77 7.10 85 6.07 92 4.48 KNOWLEDGE ON HIV/AIDS/STD 51 M3 Knowledge about HIV/AIDS and STD transmission Percent of mothers who knows at least ONE HIV/AIDS and STD transmission (MOT) 14 5.95 18 6.54 28 7.62 37 8.20 42 8.39 47 8.49 53 8.49 68 7.95 52 M3 Knowledge about HIV/AIDS and STD Prevention Percent of mothers who knows at least ONE HIV/AIDS and STD prevention (MOT) 14 5.95 17 6.31 26 7.48 36 8.16 42 8.39 46 8.47 53 8.49 69 7.85 59 Table 2: Rapid Core Assessment Tool for Child Health (CATCH) from October 2001 to June 2006 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan'05 Confidence Interval ± CI LQAS Jul'05 Confidence Interval ± CI LQAS Jul'06 Confidenc Interval ± CI SENTINEL MEASURE OF CHILD HEALTH AND WELL-BEING 1 M1 & M2 Underweight ChildrenPercentage of Children age 0-23 months that is underweight (-2 SD from the median weight-for-age, according to the World Health Organization (WHO)/National Center for Health Statistics (NCHS) 29 5.45 29 5.45 2 M3 Birth Spacing Percent of children age 0-23 months that was born at least 24 months after the previous surviving child 58 11.10 58 10.96 56 10.74 68 11.37 59 12.34 67 10.78 76 9.42 69.4 12.90 3 M1 Delivery Assistance Percent of children age 0-23 months whose birth were attended by skilled health personal upto MCHW 17 6.31 23 7.19 31 7.19 33 8.00 33 8.00 30 7.79 40 8.32 42.1 8.39 4 M1 Maternal Tetanus Toxoid (TT) Percent of mothers with children age 0-23 months that received at least TWO tetanus toxoid injections before the birth of their youngest child. 13 5.67 21 6.93 18 6.80 20 7.16 24 7.44 22 7.26 32 8.18 63.2 8.20 5 M1 Exclusive Breastfeeding Percent of children age 0-5 months that was exclusively breastfed during the last 24 hours 62 8.26 95 5.15 83 8.94 90 6.73 75 11.58 93 6.03 80 10.41 100.0 6 M1 Complementory Feeding Percent of children age 6-9 months that received breast milk and complementary foods during the last 24 hours 73 15.82 64 15.69 82 10.84 81 13.91 81 9.70 87 9.54 88 9.18 95.7 4.81 7 M2 Full Vaccination Percent of children age 12-23 months that is fully vaccinated (against the five vaccine preventable diseases) before the first birthday 10 5.05 11 5.38 29 7.68 34 8.04 40 8.32 52 8.49 50 8.50 66.9 8.00 60 SN Mod# Indicator Indicator/ Definition Baseline (LQAS) Oct'01 Confidence Interval ± CI LQAS Jan'03 Confidence Interval ± CI LQAS Jul'03 Confidence Interval ± CI LQAS Jan'04 Confidence Interval ± CI LQAS Jul'04 Confidence Interval ± CI LQAS Jan'05 Confidence Interval ± CI LQAS Jul'05 Confidence Interval ± CI LQAS Jul'06 Confidenc Interval ± CI 8 M2 Measles Percent of children age 12-23 months that received a measles vaccine 11 5.22 14 5.81 29 7.68 38 8.26 41 8.35 52 8.49 53 8.49 72.2 7.62 9 M1 & M2 Bednets Percentage of children age 0-23 months that slept under an insecticide-treated net (in malaria risk areas) the previous night 1 1.04 1.5 1.46 10 M3 HIV/AIDS Percent of mothers with children age 0-23 months that cited at least TWO known ways of reducing the risk of HIV infection 9 4.87 12 5.53 20 6.74 25 7.34 26 7.41 38 8.26 37 8.20 51.1 8.50 11 M2 Hand Washing Percent of mothers with children age 0-23 months that reported they wash their hands with soap or ash before food preparation and feeding children and after defection and attending to a child who has defecated 5 3.80 4 3.23 18 6.54 26 7.41 33 8.00 50 8.50 47 8.49 63.2 8.20 MANAGEMENT/TREATMENT OF ILLNESS 12 M1 & M2 Danger Signs Percent of mothers of children aged 0-23 months that knew at least TWO signs of childhood illness that indicate the need for treatment 84 4.42 88 3.91 94 2.94 98 1.63 100 0 98 1.46 99 1.04 99.6 0.74 13 M1 & M2 Sick Child Percent of sick children age 0-23 months that received increased continued feeding during an illness in the past two weeks 62 12.84 62 13.90 82 11.17 87 9.28 87 9.73 89 9.18 90 7.84 94.5 6.00 14 M1 & M2 Sick Child Percent of sick children age 0-23 months that received increased fluids during an illness in the past two weeks 24 11.23 34 13.55 71 13.24 63 13.09 83 10.95 76 12.56 91 7.22 92.7 6.86 Note: Indicators indicated (indicator # 13 and 14) are merged in generic RAPID CATCH but the Plan Nepal; CS Project in the same indicators has been collecting information separately. 61 H Integrated Health Facility Assessment: Comparison of Key Indicators between 2002 and 2006 Year 2002 Year 2006 SN Indicator Clinical Examination Numerator Denominator % Numerator Denominator % 1 Proportion of assessment tasks completed for sick children with a history of diarrhea 82 155 53% 164 200 82% 2 Proportion of assessment tasks completed for sick children with a history of ARI 273 356 77% 196 208 94% 3 Proportion of assessment tasks completed for sick children with a history of fever 70 172 41% 173 204 85% Immunization 4 Proportion of children who had vaccination card checked at sick child visit 3 122 2% 54 85 64% 5 Proportion of children who needed an immunization who received it in on the day of the visit or were referred for vaccination 15 53 28% 66 85 78% Treatment 6 Proportion of children who received an appropriate medication for the diagnosis made by the health worker 60 122 49% 81 85 95% 7 Proportion of children with simple diarrhea who received ORS/ RHF 7 24 29% 30 40 75% 8 Proportion of pneumonia cases who received an appropriate antibiotic 24 35 69% 21 21 100% Supporting Information Proportion of children with diarrhea who received an antibiotic or an anti-diarrheal 18 24 75% 6 6 100% Proportion of children with URTI who received an antibiotic 3 46 7% 29 29 100% Interpersonal Communication 9 Proportion of treatment counseling tasks completed for sick children 130 267 49% 212 252 84% 10 Proportion of children whose caretakers were counseled on the importance of giving fluids at home 69 122 57% 79 85 93% 62 Year 2002 Year 2006 SN Indicator Clinical Examination Numerator Denominator % Numerator Denominator % 11 Proportion of children whose caretakers were counseled on the importance of giving food or breastfeeding at home 56 122 46% 74 85 87% 12 Proportion of children whose caretakers were given advice on when to return 26 122 21% 83 85 98% Supporting Information Proportion of children whose caretakers were told how to administer oral medications 77 101 76% 80 84 95% Health worker Knowledge Training 13 Proportion of health workers who saw sick children and who had received training in the management of child illness in the last 12 months 23 25 92% 25 25 100% 14 Proportion of health workers with correct knowledge of when to refer a sick child 19 25 76% 13 25 52% Supervision 15 Proportion of health workers who had received at least one supervisory visit in the last 6 or 12 months 23 25 92% 20 25 80% Supporting Information Average number of supervisory visits per year per health facility 23 25 92% 20 25 80% Proportion of health workers who had received feed back from supervisor 13 23 57% 20 25 80% Caretaker Knowledge and Practice Management of the sick Child at Home 16 Proportion of children receiving oral medications whose caretakers knew correctly how to administer the treatment at home 39 98 40% 80 85 94% 17 Proportion of caretakers who knew how to correctly manage the child at home 42 122 34% 85 85 100% 63 Year 2002 Year 2006 SN Indicator Clinical Examination Numerator Denominator % Numerator Denominator % 18 Proportion of caretakers knew at least two signs of when to return if the child became worse at home 76 122 62% 84 85 99% Facility Equipment Organization and drugs and supplies 19 Proportion of health facilities that had experienced at least one stock￾out of ORS in the previous month 3 25 12% 0 25 0% Record Keeping 20 Proportion of health facilities with up to date immunization and patient registers 11 25 44% 25 25 100%