Plan Nepal Child Survival Project, Bara District, Nepal FAO-A-00-97-00042-00 30 September 2001 – 29 September 2006 Mid-term Evaluation April 2004 Report Prepared By Kedar Baral, Plan - Nepal Dennis Cherian, Plan - USA Babu Ram Devkota, Plan - Nepal Pierre-Marie Metangmo, Plan - USA Karunesh Tuli, Consultant TABLE OF CONTENTS Page A. Summary .........................................................................................................1 B. Assessment of Progress..................................................................................5 1. Technical Approach ...................................................................................5 2. Cross-cutting Approaches..........................................................................8 C. Program Management...................................................................................13 1. Planning .................................................................................................13 2. Staff Training............................................................................................13 3. Supervision ..............................................................................................13 4. Human Resources and Staff Management ..............................................14 5. Financial Management.............................................................................14 6. Logistics……………. ................................................................................14 7. Information Management .........................................................................14 8. Technical and Administrative Support......................................................15 D. Other Issues Identified by Evaluation Team ..................................................16 E. Conclusions and Recommendations .............................................................17 F. Results Highlight............................................................................................20 ATTACHMENTS A. Baseline Information from Detailed Implementation Plan B. Evaluation Team Members and their titles C. Evaluation Assessment Methodology D. List of Persons Interviewed and Contacted E. Diskette with Electronic Copy of Report F. Special Report G. Updated Project Data Sheet H. Findings of Four Lot Quality Assessments I. Work Plan J. Mid-term Evaluation Questionnaires 1 A. Summary 1. Program Description Plan International has been implementing a five-year Child Survival project in partnership with the Ministry of Health and non-governmental organizations (NGOs) in Bara district, Nepal since 30 September 2001. The district is in the Terai, a lowland area near the Indian border. The project area includes all the 98 Village Development Committees (VDCs) in the district; approximately 80,000 children under five years of age and 110,000 women of reproductive age are the target beneficiaries. The goal of the project is to assist the Ministry of Health to improve the health status of children under five and of women of reproductive age in Bara district. Project interventions include control of diarrhea, pneumonia case management, maternal and newborn care, and child spacing. 2. Accomplishments and Progress in Achievement of Objectives Project team members have forged excellent working relationships with each other and with staff from partner organizations. Together they carry out valuable work in the communities in Bara. Pregnant women's groups have been meeting for a year now and serve as a forum for discussion of health problems and for education of members and other women in maternal and child health issues. Members visit health facilities for preventive health services. Child club members participate in activities aimed at raising awareness in their communities about health problems and have helped with polio vaccination campaigns. A strong monitoring system is in place. Pregnant women's groups prepare village maps that provide a snapshot of preventive services received by their members. Female community health workers compile data on these services and share them with project staff. The project also conducts two surveys a year to monitor progress in achieving targets. These surveys employ the lot quality technique and generate information about the seven field areas into which the district is divided. Project partners discuss findings, identify problems, and recommend remedial action. Many organizations in Nepal -- including Save the Children -- have obtained assistance from project staff to initiate similar assessments in their projects. The project has already achieved many of the targets that it aimed to attain by 2006. Mothers' knowledge and practices related to diarrhea, pneumonia, child spacing, and pregnancy have improved substantially in the district. 3. Constraints, Problems, and Areas in Need of Attention • The evaluation team found that health facilities and community-based health workers face shortages of some medicines and other commodities. There have 2 been stock-outs of BCG vaccine, oral rehydration solution packets, and vitamin A (for administration to women after delivery). • Ministry of health staff members at the district office and health posts are frequently transferred. This disrupts programmatic decision making and service relationships with community members. • Plan has identified two NGO partners in Bara -- Child Welfare Society and Community Welfare Centre. The two partners have so far related to the project as a single entity (with staff from the two organizations working as one team operating from a single office). The rationale for this arrangement was to allow them to learn from each other. However, in practice the arrangement has not worked well. • Project staff compile and track a very large number of indicators every six months (through lot quality assessments). It is difficult for project partners to meaningfully discuss the long list of 50 indicators that is generated by the assessments. • Project staff have used Ministry of Health documents to develop training and education materials. The resulting materials, although consistent with Ministry recommendations, are primarily directed at a literate audience. Literacy levels are low in the district (both among community members and community-based volunteers). It is important to enhance the quality of the pictorial content of the materials. 4. Capacity-building Efforts The project has built capacity to deliver and track health care services at the community level and among health facility, partner NGO, and Plan staff. The strongest efforts are at the community level -- formation of pregnant women's groups and child clubs and training of traditional birth attendants and female community health volunteers. Although Ministry of Health functionaries participate in training activities, they depend on Plan staff and NGO partners for community mobilization, monitoring, and other project-related activities. 5. Sustainability Prospects Project staff have carefully avoided setting up a service delivery infrastructure that duplicates Ministry of Health mechanisms. However, while the latter will continue to function when the project ends, their ability to monitor and support community-based project activities is limited. Community groups -- pregnant women's groups and child clubs -- offer the best chance that project activities will continue after funding ends. The evaluation team 3 interacted with groups that continue to meet and conduct activities even when project staff are unable to attend. 6. Priority Recommendations • Form a national advisory group to review project progress and provide support. Include representatives from the Ministry of Health (the divisions of Child Health and Family Health and the unit that is responsible for improving neonatal survival), Plan (Kathmandu and Bara), USAID Nepal, Nepal Family Health Program, Child Welfare Society, and Community Welfare Centre. Organize meetings of the advisory group every six months (shortly after each lot quality assessment). • Child Welfare Society and Community Welfare Centre should work separately with the Child Survival project. Plan should assess their organizational strengths and identify activities in which each can contribute effectively. Staff from the two organizations can continue to learn from each other and from other project partners by participating in advisory group meetings at the district and national levels. • Use a short list of indicators for discussion with project partners. Plan staff should identify one or two indicators within each technical area in consultation with the partners. The advisory groups should review these priority items during their meetings. They should give special attention to indicators on which there has been slow progress during the first half of the project (such as tetanus toxoid administration to pregnant women and supervisory visits by Ministry of Health staff). • Incorporate local art forms in project training and education materials. The project district is close to Janakpur -- a region that is well known for Mithila art. Women decorate the walls of their houses with traditional designs and colorful pictures. Staff should explore the possibility of having local artists prepare pictures based on Mithila art for project training and educational materials. 4 7. Plan’s Response To Priority Recommendations (ACTION PLAN) Activity By When Formation of a national advisor group to review project progress and provide support Explore the possibility formation of Advisory Group by Nov 2004 November 2004 Split of CWC and CWS partnership and allow to work separately July 04 (Already completed in July) Use a short list of indicators for discussion with project partners July 04 (Already completed in July, 2004 Incorporate local art forms in project and educational materials Explore the potential art Preparation Implementation September 04 November 04 December 04 – February 05 5 B. Assessment of Progress The project is on-track in implementing much of the workplan presented in the Detailed Implementation Plan despite the uncertain political situation in the country. Work in the district is disrupted frequently because of strikes declared by Maoists and the political parties. However, project staff have managed to stay on schedule except for a few activities that are precluded by the current security environment (such as puppet shows) and some that are delayed because of the absence of elected representatives in village development committees (such as the community drug program). 1. Technical approach a) Project Overview The project is located in the Bara District of the Narayani Zone of the Central Development Region. The district is in the Terai, a lowland area near the Indian border. The project area includes all 98 Village Development Committees -- VDCs, administrative divisions -- of the district. This project followed a previous Child Survival project, which was located in 33 VDCs in Rautahat district and 17 VDCs in Bara District. Approximately 80,000 children under five years of age and 110,000 women of reproductive age are the target beneficiaries. The goal of the project is to assist the Ministry of Health to improve the health status of children under five and of women of reproductive age in Bara district. The end-of-project objectives are as follows. • Behavioral: Women of reproductive age and mothers of children under five years of age will be practicing healthy behaviors and seeking medical care from trained providers. • Increased access to services: Communities and families will have increased access to health education, quality care and essential medicines. • Quality of care: Ministry of Health personnel, community volunteers and other service providers will be practicing appropriate integrated management of sick children (particularly pneumonia and diarrhea case management). Practitioners and volunteers will also deliver quality family planning and maternal and newborn preventive care. 6 • Institutional strengthening: Community-based organizations, local non￾governmental organizations, and Ministry of Health facilities in the district will be developed and strengthened to support and implement activities that enhance child survival. Project interventions include control of diarrhea, pneumonia case management, maternal and newborn care, and child spacing. Project partners are implementing two complementary strategies to reach project objectives. First, the project is setting up a community based health system that is linked to and supports the Ministry of Health services in the district. Second, partners are implementing the Integrated Management of Childhood Illness (IMCI) approach at the facility level and expanding the existing integrated community level approach (for management of diarrhea and pneumonia) throughout the district. b) Progress Report by Intervention Area • Control of Diarrheal Disease and Pneumonia Case Management Activities: The two interventions are being implemented together through the IMCI approach. Training is being given to Ministry of Health workers at the health facilities and to female community health volunteers and traditional birth attendants in the community. Staff from the project and the Ministry of Health are jointly supervising and monitoring the volunteers and birth attendants. They also procure and supply oral rehydration solution packets and cotrimoxazole tablets to health facilities and to the volunteers and birth attendants. Results: The baseline lot quality assessment -- conducted in October 2001 -- found that 16% of children with diarrhea in the past two weeks received oral rehydration therapy. The most recent assessment (January 2004) showed that 35% had received it; the end-of-project target is 50%. The proportion of mothers who know at least three signs of pneumonia has increased from 15% to 85% (the project target is 50%). Data for additional indicators are presented in Attachment H. The health facility assessment conducted in 2001 found that 69% of children with pneumonia who visited a health facility received an appropriate antibiotic. Female community health volunteers also administer cotrimoxazole tablets to children with pneumonia. The mid-term evaluation team found that health facilities and community￾based health workers face shortages of oral rehydration solution packets. As with some other commodities (vitamin A for administration to women after delivery and BCG vaccine), shortages are attributed to increased demand resulting from promotional activities carried out by the project in the communities. Demand is reportedly outstripping the capacity of Ministry of Health to keep facilities and workers supplied. 7 • Maternal and newborn care and child spacing Activities: The project is training female community health volunteers and traditional birth attendants in maternal and newborn care and child spacing. Birth attendants find training on safe and clean delivery practices valuable. The volunteers educate pregnant women about the need for prenatal and postnatal care, tetanus toxoid vaccination, and iron and folic acid supplementation through pregnant women's groups. Volunteers and birth attendants distribute contraceptives, iron and folic acid tablets, and clean home delivery kits in the communities where they work. The project has also developed a flipbook on newborn care and distributed it to volunteers for use in educating mothers. Results: During the baseline lot quality assessment 32% of mothers reported that a trained provider attended their last delivery; the proportion increased to 68% in the January 2004 assessment. The end-of-project target is 80%. The proportion of mothers who reported having received two doses of tetanus toxoid increased from 13% at baseline to 20% (the end-of-project target is 50%). At baseline 24% of mothers who do not desire children in the next two years or are not sure reported using a modern method of contraception; in January 2004, 43% did so. Attachment G contains data for other indicators. c) Special Features The project conducts two surveys a year to monitor progress in achieving targets. These surveys employ the lot quality technique and generate information about the seven field areas into which the district is divided. Project partners discuss findings, identify problems, and recommend remedial action. Many organizations Maternity Home During the previous Child Survival project, community members in Nijgadh (in the northern part of Bara district) established a maternity home with support from Plan. The facility was initially located in rented premises. It is now housed in its own building and does not receive funding from Plan. Auxiliary nurse midwives at the home are able to admit women with high-risk pregnancies and provide skilled care to them during the final weeks of pregnancy. In addition, they offer routine prenatal and postnatal checkups, family planning services, and child health care. The home operates two ambulances to transport patients to referral hospitals. The ambulance service generates enough revenue to meet all of its costs and a part of the other expenses of the maternity home. 8 in Nepal -- including Save the Children -- have obtained assistance from project staff to initiate similar assessments in their projects. Project staff have documented the use of the lot quality technique in several publications. An issue of a Child Survival newsletter highlighted the project's monitoring strategy (Espeut D. Effective Monitoring with Efficient Methods: Plan/Nepal's Experience with LQAS in Project Monitoring, Child Survival Connections, Volume 1, Issue 2, Calverton, MD: Child Survival Technical Support Project and the CORE Group). A chapter in a recent book on community-based health care also describes the methodology (Valadez JJ and Devkota BR. 2002. Decentralized Supervision of Community Health Programs: Using LQAS in Two Districts of Southern Nepal, In Community-Based Health Care: Lessons from Bangladesh to Boston, Edited by Rohde J and Wyon J, Boston, MA: Management Sciences for Health). 2. Cross-cutting Approaches a) Community Mobilization Project staff, in consultation with District Health Office staff, initiated the formation of pregnant women's groups in 2003. Approximately 300 groups are currently active. Around 12 pregnant women form one group. Female community health volunteers facilitate the groups; meetings are held once a month. The groups serve as a forum for discussion of health problems and for education of members and other women in maternal and child health issues. Members visit health facilities for preventive health services. Since the creation of the groups, health facility staff report increased uptake of such services as antenatal check-ups, iron and folic acid supplementation among pregnant women and lactating mothers, postnatal vitamin A supplementation, tetanus toxoid vaccination among pregnant women, and immunization among children aged 12- 23 months. The project also supports child clubs in the district. Project partners have trained club members in staging dramas in their villages to deliver health messages. Club members participate in activities aimed at raising awareness in their communities about health problems and have helped with polio vaccination campaigns. The project helps the Ministry of Health with national campaigns to deliver vitamin A, deworming medication (albendazole), and polio vaccine to children. Female community health volunteers inform community members about campaign dates and locations, administer the drug or vaccine, and keep track of the number of children who participate. The project assists the District Health Office with procurement and distribution of supplies. The Maoist insurgency has contributed to inadequate program support particularly in the district interior due to reduced security. Calling meetings with key district authorities has also been hampered by shifting district priorities occasioned by the 9 present conflict. This notwithstanding, as mentioned at the beginning of Section B "Assessment of Progress", most activities in the communities have been conducted as scheduled. Project staff attribute this success to "neutrality, transparency, and good relations". They welcome participation by all in discussions about project activities in the villages -- without regard to political affiliation; they describe project progress and the management of project resources in an open manner; and involve community-based volunteers in efforts to build positive relationships with communities. b) Communication for Behavior Change The project employs a variety of methods including the use of puppet shows to change behaviors among community members and health facility staff. Pregnant women's groups and child clubs provide project partners the opportunity to deliver messages on maternal and child health in a group setting. Meetings of women's groups often attract non-members (such as mothers and mothers-in-law). As mothers-in-law have important roles in household decision making, their participation is encouraged. Members of child clubs take what they learn in their groups into the community. The evaluation team visited a club that has been very active in community affairs over the last three years. The children persuaded community elders to stop several child marriages -- of girl club members 12 or 13 years of age. Project staff provide one-on-one counseling and coaching on interpersonal skills to Ministry of Health functionaries in an attempt to improve the way they interact with patients who visit health facilities. In addition, health functionaries participate in the training of community volunteers and in outreach activities in the communities. Having served as trainers and because of interactions with community members outside health facilities, these functionaries are expected to provide better care to patients who are referred by volunteers and to others who visit their facilities. Project staff have used Ministry of Health documents to develop training and education materials. Many of the resulting printed materials, although consistent with Ministry recommendations, are primarily directed at a literate audience. To supplement these materials, project staff have recently prepared a flip chart and pictorial cards on maternal and newborn care. They have also placed billboards with health messages in the project area as well as planned the use of radio broadcasts to reach the population. Literacy levels are low in the district (both among community members and community-based volunteers) and pictorial materials assume great importance in educational activities. By incorporating local art forms in project training and education materials, the project can enhance their quality and make them more attractive to community members. The project district is close to Janakpur -- a region that is well known for Mithila art. Women decorate the walls of their houses with traditional designs and colorful pictures. 10 Staff should explore the possibility of having local artists prepare pictures based on Mithila art for project training and educational materials. c) Capacity Building Approach Overview: The project has built capacity to deliver and track health care services at the community level and among health facility, partner NGO, and Plan staff. The strongest capacity building efforts are at the community level -- formation of pregnant women's groups and child clubs and training of traditional birth attendants and female community health volunteers. Although Ministry of Health functionaries participate in training activities, they depend on Plan staff and NGO partners for community mobilization, monitoring, and other project-related activities. Results: By project-end, 80% of village development committees are expected to have at least three pregnant women's groups (or other mothers' groups) with demonstrated health promotion activities. In January 2004, 60% of the village development committees were reported to have such groups. Compared to an end-of-project target of 25%, 18% of female community health volunteers had established community health funds and mechanisms for cost recovery by January 2004. In October 2002, 10% of the volunteers reported conducting community mobilization and education activities eight times in the last 12 months. In January 2004, the proportion had increased to 29% (the final target is 70%). Progress has also been slow in improving the frequency of supervisory visits by Ministry of Health staff. At baseline 7% of health workers reported at least one occasion in the last three months when a Ministry of Health supervisor visited them. District Health Office records show that in January 2004, the proportion had increased to 23%; the final target is 75%. During an interview with the mid-term evaluation team, the chief of the District Health Office explained that supervisors find it difficult to visit facilities and community-based workers because they feel insecure (due to Maoist threats) and do not have access to vehicles. However, they are interested in improving the level of supervision and will carry out joint visits with Plan staff. Training: Project partners have conducted several training events during the first thirty months (for staff members from Plan, partner non-governmental organizations, and Ministry of Health and community-based volunteers). The project training summary is presented in Attachment L. Training has included: • Transformation for health (training based on Paulo Freire's philosophy and designed to help trainees gain skills in solving problems in communities in a participatory way) • General training of trainers (a course on training methods and facilitation techniques) • Integrated management of childhood illness • Control of diarrheal disease and pneumonia case management 11 • Maternal and newborn care and family planning • Clean home delivery (training of traditional birth attendants) • Infection control (training of peons -- support staff -- from health facilities) • Drama (training of child club members in techniques for staging plays on health issues) • Community drug program • Lot quality technique for monitoring project progress • Health facility assessment methodology Participation in Meetings: In addition to attending formal training events and workshops in the project area, Plan staff (from Bara, Kathmandu, and Washington, DC) participate in national and international meetings and workgroups organized by other agencies (such as the Ministry of Health, Global Health Council, the Child Survival Technical Support Project, and the Child Survival Collaborations and Resources Group). In May 2002, Babu Ram Devkota, the Project Coordinator, participated in Global Health Council's annual meeting in Washington D.C. In September 2002 he presented a paper at the Data for Action Workshop organized by the Child Survival Technical Support Project and the Child Survival Collaborations and Resources Group ("Using Biannual LQAS Data to Improve Child Survival and Safe Motherhood in Nepal: Program Results from 1999 to 2001"). Supportive Supervision: In order to strengthen capacity for delivery of health services in Bara, the project supplements training events with review meetings and supportive supervision. Trainees are encouraged to report problems they face in their daily work during supervisory visits and they receive support in solving problems from their supervisors. Plan has designed several checklists with input from the Ministry of Health. These checklists help supervisors remember key issues when they observe project activities in health facilities and communities. Effect of Capacity Building Efforts on Facilities and Community-based Volunteers: The team that conducted the baseline health facility assessment in January 2002 was able to visit only a third of the facilities (the rest were closed). All the facilities visited by the mid-term evaluation team were open and project staff report that most health posts are now operational year-round. Facility staff members stated, during interviews by the team, that their ability to manage diarrhea, pneumonia, and other childhood illnesses has improved after attending project-sponsored training events. They were also pleased with the training provided to peons (on infection control). Sanitation at health posts has improved. At the district hospital, the chief of the District Health Office personally leads an effort to clean the premises every month. During interviews with the evaluation team, traditional birth attendants reported that training in clean home deliveries has been very useful. 12 d) Sustainability Strategy The project aims to promote long-lasting improvements in child health in Bara through positive changes in health practices of beneficiaries and continued access to information, services, and supplies. Strengthening the managerial and technical capacity of government and NGO partners and community-based volunteers is critical. It is also important to find ways to draw on community resources to offset project costs. Project staff have carefully avoided setting up a service delivery infrastructure that duplicates Ministry of Health (MOH) mechanisms. The monitoring and evaluation system of the latter has received project support to strengthen quality of HMIS reporting. However, while the MOH will continue to function when the project ends, their ability to monitor and support community-based project activities is limited. Community groups -- pregnant women's groups and child clubs -- offer the best chance that project activities will continue after funding ends. The evaluation team interacted with groups that continue to meet and conduct activities even when project staff are unable to attend. Project partners have demonstrated an ability to create financially viable health care systems. As described in Section B.1.b (Progress Report by Intervention Area: Maternal and newborn care and child spacing), the maternity home in Nijgadh is now operating without support from Plan. The project has also transferred its LQAS skills to all NGO partners working with it and to other leading PVOs in the country including Save the Children (US). However, implementation of the community drug program, an important initiative to create a self-sustaining system for the supply of medicines, has been delayed due to the absence of elected representatives in village development committees. Elected members were expected to have a vital role in planning, implementing, and monitoring the program. In 2002 the government decided against extending the tenure of elected bodies. This decision has hampered many community-based activities but most critically the drug program. Earlier this year, staff from the Ministry of Health, NGO partners, and Plan received training in the program and the initiative will move forward when elected bodies become functional again. 13 C. Program Management Project personnel in Bara are highly skilled managerially and technically and are led by a very knowledgeable and experienced Project Coordinator, Babu Ram Devkota. They receive good support from Dr. Kedar Baral, the Kathmandu-based National Health Advisor and from Dr. Pierre Marie Metangmo and Dr. Dennis Cherian (from Plan's Washington, DC office), and Dr. Sun Lal Thapa (Child Health Division, Ministry of Health, Kathmandu). Project team members have forged excellent working relationships with each other and with staff from partner organizations. Many of them have worked together for years (during the previous Child Survival project and the current project). During the mid-term evaluation, representatives of local non-governmental organizations, District Health Office, and Plan's Kathmandu and US offices commended project staff for their good teamwork. 1. Planning Bara district is divided into seven field areas. Field teams consisting of a Community Health Officer and 1-3 Assistant Community Health Officers conduct much of the planning and decision-making for project management in each area. The teams work closely with female community health volunteers, traditional birth attendants, pregnant women's groups, and health facility staff. Two advisory groups meet at the district level every two months (the Reproductive Health Committee and the Quality Control Committee). Project management issues are discussed in these committees. However, some supply and staffing issues can only be addressed at the national level. Project management is likely to improve if a national advisory group is constituted to review project progress and provide support. 2. Staff Training As described in Section B.2.c (Cross-cutting Approaches: Capacity Building Approach), project personnel have participated in a number of training events during the first half of the project. Trainees are tested before and after the events to measure changes in knowledge levels. New knowledge and skills are also monitored and reinforced through review meetings and supportive supervision. 3. Supervision Project staff visit health facilities and community health volunteers to monitor their work, reinforce knowledge and skills gained during training workshops, and to help solve problems. They encourage District Health Office staff to accompany them during their visits. As described in Section B.2.c (Cross-cutting Approaches: Capacity Building Approach), supervision by district office staff has increased over the course of the project. However, even now supervisors from the district office are able to visit only 23% of health workers over a three-month period. They lack adequate transportation and feel unsafe traveling to villages because of the Maoist insurgency. 14 4. Human Resources and Staff Management Ministry of health staff members at the district office and health posts are frequently transferred. This disrupts service relationships with community members and programmatic decision making. In one health post, the auxiliary nurse midwife has not been available for many months. Central authorities often overturn staffing decisions made by the chief of the District Health Office. This is a matter that needs attention from a national advisory group. The Plan team in Bara has been a remarkably stable group with little turnover. Plan managers in Bara and Kathmandu (Project Coordinator, National Health Advisor, Human Resource Manager, and Country Director) are aware of the need to plan ahead in order to find jobs for project staff after funding ends in 2006. They plan to compile staff resumes in a document that will be circulated among development agencies in Nepal. They will also explore the possibility of moving some personnel into positions with other Plan projects in Bara and other districts of Nepal. 5. Financial Management The Finance Officer provides monthly financial reports to the Project Coordinator, who consolidates the information and submits quarterly reports to the Operations Support Manager via the Program Unit Manager. The reports are then passed on to Plan's U.S. office where they are reviewed for compliance with USAID requirements and then submitted to USAID. Funding transfers are made on a monthly basis from the US office to the project to reimburse project expenditures. 6. Logistics As mentioned in Section B.1.b (Progress Report by Intervention Area: Control of Diarrheal Disease and Pneumonia Case Management), health facilities and community-based health workers face shortages of some medicines and other commodities (such as BCG vaccine, oral rehydration solution packets, and vitamin A for administration to women after delivery). District officials point to insufficient supplies from the national level and say that they quickly move commodities to health facilities and workers, with help from Plan and Nepal Family Health Program, after they are received from central supply units. They would like the Ministry of Health to step up supplies of these commodities to keep pace with the increased demand resulting from promotional activities carried out by the project in the communities. 7. Information Management A strong monitoring system is in place. Pregnant women's groups prepare village maps that provide a snapshot of preventive services received by their members. Female community health workers compile data on these services and share them with project staff. The project also conducts a survey every six months to monitor progress in changing health knowledge and practices of caregivers in Bara. The lot quality sampling technique is used to identify 19 households in each field area. Project staff conduct interviews with women aged 15-49 years and mothers with children 0-23 months of 15 age. They then summarize survey results, discuss problems, and suggest solutions for each field area. They also combine findings for the seven field areas to generate results for the entire district. While this is an excellent system for identifying and solving problems, it is also too detailed and produces information on a very large number of variables (50 indicators). It is difficult for project partners to meaningfully discuss such a long list. Project monitoring can be improved by focusing attention on a short list comprised of one or two indicators for each technical area. 8. Technical and Administrative Support Several organizations and individual consultants have given technical assistance to the project. Basic Support for Institutionalizing Child Survival II Project (BASICS II) helped train health facility staff in improving the quality and coverage of routine immunization. Trainers from the National Pediatric Society of Nepal conducted basic and follow-up training in integrated management of childhood illness. Individual consultants were hired to help develop the detailed implementation plan, to train staff in "transformation for health", and to evaluate the information, education, and communication strategy. Dr. Pierre-Marie Metangmo (Plan - USA) helped with the project start-off workshop and he, along with Dr. Dennis Cherian and other colleagues from his office, edited annual reports, answered technical and administrative questions, and shared technical materials with staff in Bara. Project staff are planning an operational research study to assess the effectiveness of pregnant women's groups in improving care-seeking behaviors. They will need help from a sociologist or anthropologist with expertise in operational research to design and conduct the study. Project staff will also benefit from training in the Windows version of EpiInfo (an epidemiological and statistical software package designed by a team from the Centers for Disease Control and Prevention, Atlanta, Georgia). 16 D. Other Issues Identified by Evaluation Team Plan works with two NGO partners in Bara -- Child Welfare Society and Community Welfare Centre. The two partners have so far related to the project as a single entity (with staff from the two organizations working as one team operating from a single office). The rationale for this arrangement was to allow them to learn from each other. However, in practice the arrangement has not worked well. The two NGOs will be able to contribute more effectively to child health activities in the area by working separately with the Child Survival project. Child Welfare Society has been working in Bara for many years. The organization operated family planning clinics in the district when the previous Child Survival project was being implemented and can help Plan in strengthening health facilities (especially in the delivery of reproductive health services). The Society also rehabilitates disabled children. Community Welfare Centre is new to the district but offers special expertise. The organization has been active in the prevention of HIV/AIDS in Kathmandu and has access to staff with strong skills in behavior change communication. Separate working arrangements with the Child Survival project will enable staff from the two NGOs to draw upon their unique experience and skills and free them of the constraint they currently face (having to work jointly on a set of activities that only partially match their respective organizational profiles). 17 E. Conclusions and Recommendations The Child Survival project is well planned, draws on the resources of carefully selected partners, and is being competently managed by the Bara-based team with help from Plan's Kathmandu and U.S. offices and the Ministry of Health. Summarized below are project strengths, areas that need improvement, and recommendations of the evaluation team. Project strengths • Excellent working relationships Many project team members have worked together for years (during the previous Child Survival project and the current project). They work well with each other and with staff from partner organizations. Representatives of local non-governmental organizations, district health office, and Plan's Kathmandu and US offices commended project staff for their excellent teamwork. • Strong community level work Pregnant women's groups have been meeting for a year now and serve as a forum for discussion of health problems and for education of members and other mothers in maternal and child health issues. Members are encouraged to visit health facilities for preventive health services. Child club members are participating in activities aimed at raising awareness in their communities about health problems and have helped with polio vaccination campaigns. • Strong monitoring system A strong monitoring system is in place. Pregnant women's groups prepare village maps that provide a snapshot of preventive services received by their members. Female community health workers compile data on these services and share them with project staff. The project also conducts two surveys a year to monitor progress in achieving targets. These surveys employ the lot quality technique and generate information about the seven field areas into which the district is divided. Project partners discuss findings, identify problems, and recommend remedial action. Many organizations in Nepal -- including Save the Children -- have obtained assistance from project staff to initiate similar assessments in their projects. • Good achievement of targets The project has already achieved many of the targets that it aimed to attain by 2006. Mothers' knowledge and practices related to diarrhea, pneumonia, child spacing, and pregnancy have improved substantially in the district. 18 Areas for improvement and recommendations • Provision of supplies and supervision by Ministry of Health staff Ministry of Health staff members at the district office and health posts are frequently transferred. This disrupts service relationships with community members, hampers programmatic decision-making, and makes regular supervision difficult. Health facilities and community-based health workers face shortages of some medicines and other commodities (such as BCG vaccine, oral rehydration solution packets, and vitamin A for administration to women after delivery). District-level committees meet regularly and discuss project management issues. However, some supply and staffing issues can only be addressed at the national level. Recommendation Form a national advisory group to review project progress and provide support. Include representatives from the Ministry of Health (the divisions of Child Health and Family Health and the unit that is responsible for improving neonatal survival), Plan (Kathmandu and Bara), USAID Nepal, Nepal Family Health Program, Child Welfare Society, and Community Welfare Centre. Organize meetings of the advisory group every six months (shortly after each lot quality assessment). • Participation of non-governmental organizations Two non-governmental organizations, Child Welfare Society and Community Welfare Centre, have so far related to the project as a single entity (with staff from the two organizations working as one team operating from a single office). The rationale for this arrangement was to allow them to learn from each other. However, in practice the arrangement has not worked well. Recommendation Child Welfare Society and Community Welfare Centre should work separately with the Child Survival project. Plan should assess their organizational strengths and identify activities in which each can contribute effectively. Staff from the two organizations can continue to learn from each other and from other project partners by participating in advisory group meetings at the district and national levels. • Monitoring system Project staff compile and track a very large number of indicators every six months (through lot quality assessments). It is difficult for project partners to meaningfully discuss the long list of 50 indicators that is generated by the assessments. 19 Recommendation Use a short list of indicators for discussion with project partners. Plan staff should identify one or two indicators within each technical area in consultation with the partners. The advisory groups should review these priority items during their meetings. They should give special attention to indicators on which there has been slow progress during the first half of the project (such as tetanus toxoid administration to pregnant women and supervisory visits by Ministry of Health staff). • Training and education materials Project staff have used Ministry of Health materials to develop training and education materials. They have ensured that the text of manuals and guides used to train health workers and flipcharts and other aids used to educate community members is consistent with nationally recommended documents. Literacy levels are low in the district (both among community members and community-based volunteers). It is important to enhance the quality of the pictorial content of the materials. Recommendation Incorporate local art forms in project training and education materials. The project district is close to Janakpur -- a region that is well known for Mithila art. Women decorate the walls of their houses with traditional designs and colorful pictures. Staff should explore the possibility of having local artists prepare pictures based on Mithila art for project training and educational materials. 20 F. Results Highlight In February 2003, during a staff meeting held to review the findings of a lot quality assessment, an important issue was raised. The mothers’ groups conducted by female community health volunteers were not serving a useful purpose. Pregnant women and mothers of under-five children were not participating as expected and key project messages were not being delivered. During the discussion, the project team came up with a strategy to rejuvenate mother’s groups -- the creation of pregnant women's groups. Three hundred groups have been formed so far. Each group is comprised of 7-15 pregnant women living in the same village who meet once a month to discuss issues related to mother and child health. Postnatal mothers, mothers of under-five children, and mother in laws are also encouraged to participate. Female community health volunteers facilitate the meetings. Outreach workers from the local health facility are encouraged to participate and support the sessions technically and managerially. In some areas group meetings are linked with outreach clinics operated by the workers. The following steps are followed to create and manage the groups and to share health￾related information during group sessions. • Project staff members discuss the concept with local health facility staff. Together, they orient female community health volunteers and traditional birth attendants to the strategy. • Pregnant women are invited to get together for a social mapping session. They draw a map of their community on the ground and identify major trails, roads, intersections, temples, schools, and the volunteer's house. Then they locate their own houses by placing a stone or piece of brick on the map. The map is copied onto a large piece of white cloth. Marker pens are used to draw lines and boundaries. Volunteers stick printed icons on the cloth to show the location of public, private, and religious institutions and houses, schools, and health posts. Each pregnant woman is asked to update the status of health indicators (such as the number of tetanus toxoid injections received and the number of iron tablets consumed) by sticking colored dots next to her house on the map. • The volunteer distributes iron tablets. She also administers vitamin A capsules to postnatal women and family planning commodities to married women who join the session. She delivers an educational talk of about 10-15 minutes at the end of the session. Since the creation of the groups, health facility staff report increased uptake of such services as antenatal check-ups, iron and folic acid supplementation among pregnant women and lactating mothers, postnatal vitamin A supplementation, tetanus toxoid vaccination among pregnant women, and immunization among children aged 12-23 months. In some areas uptake is reported to have increased from below 5% to 70%.This pregnant women group strategy/approach contributes directly to Intermediate Result 3 and more indirectly to Intermediate Result 1 and 2. 21 ATTACHMENT A. Information from Detailed Implementation Plan Important information about the project is summarized below. There have been no substantial changes after the detailed implementation plan was approved. Project duration 30 September 2001 – 29 September 2006 Project area and population 98 Village Development Committees in Bara district of the central Terai plain; total population 525,799 Target beneficiaries 266,313 (78,870 children under five years of age, 110,418 women of reproductive age, and 77,025 new births during project period) Ministry of Health facilities 1 hospital, 3 Primary Health Centers, 11 Health Posts, and 84 Sub-health Posts Social and economic profile of project population The project population is composed of various ethnic groups and is strongly influenced by the caste system. The target group works mostly in the capacity of tenant farmers and lives below the subsistence level. Many migrants live in the area since it is near the border with India. The literacy rate in the district, for people over five years of age, is 14% among females and 42% among males. In most of the project area people do not have access to proper sanitation and safe drinking water. Overall goal Assist the Ministry of Health in improving the health status of children under five years and women of reproductive age (15-49 years) in Bara district. Project interventions (1) Diarrhea Case Management (2) Pneumonia Case Management (3) Maternal and New Born Care (4) Child Spacing Objectives (1) Behavioral: Women of reproductive age and mothers of children under-five years will be practicing healthy behaviors and seeking medical care by trained providers; (2) Increased access to services: Communities and families will have increased access to health education and quality care and essential medicines; (3) Quality of care: MOH personnel, community health volunteers and other service providers will be practicing appropriate integrated management of sick children particularly pneumonia and diarrhea case management. Practitioners and volunteers will also deliver quality family planning, and maternal and preventive newborn care; and (4) Institutional strengthening: Community based organizations, local NGOs, and district MOH facilities will be developed and strengthened to support and implement activities that enhance child survival. Strategies (1) Improved training; (2) Improved supervision and follow-up training for MOH health workers and volunteers; (3) Support and non-financial incentives for community health volunteers; (4) Strengthened community partnerships and establishment of cost recovery mechanisms; (5) Development and support of community drug program; (6) Use of innovative communication strategies to promote behavioral change; (7) Integration of Child Survival activities with Plan's other development work; and (8) Collaboration with other local partners 22 ATTACHMENT B. Evaluation Team Members The core team consisted of the following. • Kedar Baral, National Health Advisor, Plan - Nepal • Babu Ram Devkota, Project Coordinator, Plan - Nepal • Pierre-Marie Metangmo, Plan - USA • Sun Lal Thapa, Child Health Division, Ministry of Health • Karunesh Tuli, Consultant The following guided or assisted the core team in conducting the evaluation. • K. B. Achhami, Plan - Nepal • Sharmila Budhathoki, Plan - Nepal • Dennis Cherian, Health Associate, Plan - USA • Satish Chaudhary, Chief, District Health Office • Dipak Dahal, Health Information System Coordinator, Plan - Nepal • Sanjay Das, Child Welfare Society • Subarna Raj Gurung, Plan - Nepal • Biswonath Khatri, Plan - Nepal • Padam Malla, Plan - Nepal • Arjun Nepal, Nepal Family Health Program • Minty Pande, Country Director, Plan - Nepal • John Quinley, Health & Child Survival Advisor, USAID - Nepal • Dharmpal Raman, Program Specialist, USAID - Nepal • Hari Dev Sah, Plan - Nepal • Chandra Kumar Sen, District Project Coordinator, Plan - Nepal • Krishna Shrestha, Plan - Nepal • Banamali Subedi, Chairman, Community Welfare Center • Yam Thapa, Plan - Nepal • Badri Prasad Upadhyay, District Health Office 23 ATTACHMENT C. Assessment Methodology During a two-day workshop in Birganj core team members worked with staff from Plan, Ministry of Health, and other partners to plan the evaluation, develop data collection instruments, and conduct a mock focus group discussion. Babu Ram Devkota presented findings of a quantitative assessment of project progress through January 2004 (based on lot quality surveys conducted twice every year). Workshop participants then regrouped into three field teams (led by Kedar Baral, Babu Ram Devkota, and Sun Lal Thapa) to visit health facilities, interview health functionaries, and carry out focus group discussions with women in project communities over a two-day period. They spent a day compiling findings of the field visits and got together again for a daylong workshop to share findings and recommendations with the other teams. John Quinley and Dharmpal Raman (USAID-Nepal) joined the group at the end of the workshop and went out to the project area for additional observational visits. The consultant conducted a group discussion with Plan Community Health Officers and Assistant Community Health Officers to understand the challenges they face in their work. In addition, he interviewed staff from Plan, Ministry of Health, and other partners in the district and in Kathmandu. Kedar Baral, Babu Ram Devkota, and the consultant conducted two debriefing sessions, one for Plan and another one for USAID Nepal. 24 ATTACHMENT D. List of Persons Interviewed and Contacted Community Female community health volunteers, traditional birth attendants, and members of pregnant women's groups, child clubs, and Community-based Maternity Home Management Committee. Ministry of Health • Satish Chaudhary, Chief, District Health Office • Sun Lal Thapa, Child Health Division • Badri Prasad Upadhyay, District Health Office • Staff at ten facilities (doctors, health assistants, auxiliary nurse midwives, auxiliary health workers, village health workers, and maternal and child health workers). Other Partner Organizations • Sanjay Das, Child Welfare Society • Arjun Nepal, Nepal Family Health Program • Banamali Subedi, Chairman, Community Welfare Centre Plan - Nepal • Kedar Baral, National Health Advisor • Dipak Dahal, Health Information System Coordinator • Babu Ram Devkota, Project Coordinator • Vicky Ebona, Operation Support Manager • Krishna Ghimire, Sponsorship & Grants Support Manager • Subarna Raj Gurung • Ramrajya Joshi, Program Support Manager • Kamalesh Kumar Lal, Human Resource Manager • Minty Pande, Country Director • Prem Pant, Operations Support Unit Manager • Hari Dev Sah • Chandra Kumar Sen, District Project Coordinator Plan - USA Pierre-Marie Metangmo Save the Children US - Nepal Neena Khadka, Program Manager, Saving Newborn Lives United States Agency for International Development (USAID) • John Quinley, Health & Child Survival Advisor, USAID - Nepal • Dharmpal Raman, Program Specialist, USAID - Nepal • Sharon Arscott-Mills, USAID - Washington, DC 25 ATTACHMENT E. Diskette with Electronic Copy of Report 26 ATTACHMENT F. Special Report Pregnant Women’s Group - An Approach for Improving Mother and Child Health in Rural Nepal Babu Ram Devkota, Child Survival Project Coordinator 1. Background In February 2003, during a staff meeting held to review the findings of a lot quality assessment, an important issue was raised. The mothers’ groups conducted by female community health volunteers were not serving a useful purpose. Pregnant women and mothers of under-five children were not participating as expected and key project messages were not being delivered. During the discussion, the project team came up with a strategy to rejuvenate mother’s groups -- the creation of pregnant women's groups. 2. What is Pregnant Women’s Group? This is a group comprising of 7-15 pregnant women living in the same village who meet once a month to discuss issues related to mother and child health. Postnatal mothers, mothers of under-five children, and mother in laws are also encouraged to participate. Female community health volunteers facilitate the meetings. Outreach workers from the local health facility, especially Village Health Workers and Maternal and Child Health Workers, are encouraged to participate and support the sessions technically and managerially. In some areas group meetings are linked with outreach clinics operated by the outreach workers. The administrative boundary of a particular community is not the chief criterion in constituting a group. Easy access for members is more important. In general, walking distance for participating women should not be more than ten minutes. Staff members from the project and the local health facility help the volunteer in managing the group. The aim is to empower the group in such a way that members are able to demand basic health services of a high quality from government and non-governmental health care providers. The volunteers and participating women make all the decisions required to create and operate the group. 3. Goal and Objectives The goal is to empower female community health volunteers, traditional birth attendants, and pregnant women by enhancing their ability to organize themselves in groups to discuss health issues, gain access to health services, and improve the health status of under-five children and women of reproductive age. 27 The objectives are to increase, by the end of the project (September 2006), the proportion of participants who • Consume iron and folic acid supplements for at least four months during pregnancy from 5% to 80% • Consume iron and folic acid supplements for at least four months during the lactation period from 3% to 80% • Receive at least two tetanus toxoid injections during pregnancy from 13% to 90% • Have at least four antenatal checkups by local health facility staff from 5% to 70% • Are prepared for delivery (have collected clean delivery materials or a clean home delivery kit) from 10% to 85% • Are delivered by trained attendant to 50% • Receive a vitamin A supplement during the postpartum period from 16% to 70% In addition, 90% of infants born to participants should be completely immunized during the first year of life. 4. Indicators The following indicators are updated for each participant during monthly meetings by placing a colored dot on the social map (see Figures 1 and 2 at the end of this report for examples of social maps; Figure 3 is a form used to summarize indicator information). • Number of tetanus toxoid injections received • Number of months she has consumed an iron supplement • Number of antenatal checkups • Whether she has procured a clean home delivery kit (or other safe delivery materials) in preparation for delivery • Whether she consulted facility staff for any danger signs 5. Creation and Management of Group The following steps are followed to create and manage the groups and to share health￾related information during group sessions. • Project staff members discuss the concept with local health facility staff (chief of health post or sub-health post, auxiliary nurse midwife, auxiliary health worker, maternal and child health workers, and village health workers). • Project and health facility staff members orient female community health volunteers and traditional birth attendants to the strategy. The volunteers decide the number of participants and the geographical area from which participants will be drawn. 28 • Pregnant women are invited to get together for a social mapping session. They draw a map of their community on the ground. Volunteers use participatory rural appraisal techniques to facilitate the session. Participants identify major trails, roads, intersections, temples, schools, and the volunteer's house on the map. Then they locate their own houses by placing a stone or piece of brick on the map. • The map is copied onto a large piece of white cloth. Marker pens are used to draw lines and boundaries. Volunteers stick printed icons on the map to show the location of public, private, and religious institutions and houses, schools, and health posts. • Each pregnant woman is asked to update the status of health indicators (listed in Section 4) by sticking colored dots next to her house on the map. • The volunteer distributes iron tablets to participants. She also administers vitamin A capsules to postnatal women and family planning commodities to married women who join the session. • Participants discuss and schedule a date and time for the next session. • The volunteer delivers an educational talk of 10-15 minutes at the end of the session. • The volunteer makes home visits to deliver supplements (such as vitamin A) to those women who are unable to attend the session. 6. Role of Female Community Health Volunteers, Health Workers, and Project Staff The following are the major responsibilities of female community health volunteers. • Explain the purpose of the group to pregnant women and schedule a date and venue for social mapping. • Procure materials required for the social mapping session from project staff. • Invite Maternal and Child Health Worker, Village Health Worker, and project field staff to join the group during the first few sessions (especially the first two meetings). • Contact pregnant women prior to the first session to ensure that they understand the purpose of the session and to remind them about the date, time and venue. • Bring necessary educational materials and other supplies to the sessions. • Facilitate the sessions. Health workers from the local health facility and project field staff are expected to provide technical and managerial support to the female community health volunteer. Health workers are responsible for supplying iron supplements, high dose vitamin A capsules, family planning commodities, oral rehydration solution packets, and cotrimoxazole tablets. Project field staff are expected to provide commodities that are not available at the local health facility. 29 7. Status As of March 2004, 300 pregnant women's groups have been constituted and have met for at least one session other than the initial preparatory meeting. Project staff expect that 350 groups will be active in Bara district by June 2004. 8. Sustainability Pregnant women's groups are likely to continue meeting after project funding finishes because of the following reasons. • The groups do not require a lot of resources. Start-up supplies (materials for preparing the social map) cost less than one US dollar per group. Most materials are locally produced. • Constitution of mothers' groups is an important component of the national strategy for community mobilization for health. Therefore, Ministry of Health workers from local facilities will continue to support female community health volunteers in forming and facilitating pregnant women's groups in future. • Pregnant women who participate in group meetings benefit from them. They get easy access to health advice and nutritional supplements; this motivates them to participate in future sessions. 30 Figure 1. Social map FCHV's home 31 Figure 2. Social map with information on health indicators Legend (1) Red dots: Antenatal care checkup (2) Blue dots: Tetanus toxoid (3) Black dots: Iron and folic acid (4) Green dots: Preparation for delivery and procurement of clean home delivery kit FCHV: Female Community Health Volunteer FCHV's home 32 Figure 3. Pregnant women's group reporting form Plan Nepal Rautahat/Bara PU Year: _______ FA#: ____ VDC Name: ___________________ VDC Code:____ Ward#: __ Child Survival Project Pregnant Women's Group Recording sheet Indicators Month Total Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun 1) ANC Check-up 1a) First (I) 1b) Second (II) 1c) Third (III) 1d) Fourth (IV) 2) Iron consumption 2a) First (I) 2b) Second (II) 2c) Third (III) 2d) Forth (IV) 3) TT 3a) TT 1 (I) 3b) TT 2 (II) 4) Delivery preparation 4a) # of CHDK purchase Total # of pregnant women 5) # of mothers delivered 6) Vitamin A given 33 ATTACHMENT G. Updated Project data Sheet A. Child Survival Grants Program Project Summary 1. Mid Term Submission: Oct-29-2004 Plan Nepal 2. Field Contact Information: First Name: Babu Ram Last Name: Devkota Address: Plan Kalaiya c/o Plan Nepal Country Office P.O. Box 8980 City: Lalitpur State/Province: Zip/Postal Code: Country Nepal Telephone: 977-53-551017 Fax: 977-53-551016 E-mail: baburam.devkota@plan-international.org Project Web Site: - 34 3. Project Information: Project Description: To assist the Ministry of Health of Nepal to (1) improve the health of children and women of reproductive age, (2) increase health education, (3) strengthen the health service, and (4) increase community participation in health care programs. Partners: Ministry of Health of Nepal, Child Welfare Society, Community Welfare Center Project Location: Bara District 4. Grant Funding Information: USAID Funding:(US $) $1,132,342 PVO match:(US $) $377,622 5. Target Beneficiaries: Type Number 0-59 month old children: 78,870 Women 15-49: 110,418 Estimated Number of Births: 77,025 6. Beneficiary Residence: Urban/Peri-Urban % Rural % (No Data) 100% 35 7. General Strategies Planned: Strengthen Decentralized Health System 8. M&E Assessment Strategies: KPC Survey Health Facility Assessment Lot Quality Assurance Sampling Community-based Monitoring Techniques Participatory Evaluation Techniques (for mid-term or final evaluation) 9. Behavior Change & Communication (BCC) Strategies: Mass Media Interpersonal Communication Peer Communication Support Groups 10. Capacity Building Targets Planned: PVO Non-Govt Partners Other Private Sector Govt Community US HQ (CS unit) Field Office HQ CS Project Team Local NGO Traditional Healers National MOH Dist. Health System Health Facility Staff Health CBOs Other CBOs CHWs 11. Interventions: Acute Respiratory Infection 15 % 36 ** IMCI Integration ** CHW Training ** HF Training *** Pneum. Case Mngmnt. *** Case Mngmnt. Counseling *** Access to Providers Antibiotics *** Recognition of ARI Danger Signs Control of Diarrheal Diseases 30 % ** IMCI Integration ** CHW Training ** HF Training *** Hand Washing *** ORS/Home Fluids *** Feeding/Breastfeeding *** Care Seeking *** Case Mngmnt./Counseling Maternal & Newborn Care 30 % ** IMCI Integration ** CHW Training ** HF Training *** Emerg. Obstet. Care *** Recog. of Danger signs *** Newborn Care 37 *** Post partum Care *** Delay 1st preg Child Spacing *** Integr. with Iron & Folate *** Normal Delivery Care *** Birth Plans Child Spacing 25 % ** CHW Training ** HF Training *** Child Spacing Promotion *** Pre/Post Natal Serv. Integration *** Knowledge/Interest *** FP Logistics *** Community-Based Distribution *** Social Marketing *** Male Reproductive Health *** Youth FP Promotion *** Quality Care *** Human Capacity Development *** FP/HIV integration *** Maternal/Neonatal Integration *** Cost Recovery Schemes *** Community Involvement *** Access to Methods *** Policy 38 Indicator Numerator Denominator Estimated Percentage Confidence line Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) NA NA 0.0 0.0 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 44 65 68.0 11.4 Percentage of children age 0-23 months whose births were attended by skilled health personnel 91 133 68.0 7.9 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 25 122 20.0 7.6 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 70 78 90.0 6.7 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 25 31 81.0 13.9 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 45 133 34.0 8.0 Percentage of children age 12-23 months who received a measles vaccine 51 133 38.0 8.3 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) NA NA 0.0 0.0 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 221 266 83.0 4.5 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 28 52 54.0 13.6 39 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 48 133 36.0 8.2 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 34 133 26.0 7.4 Comments The project performs LQAS every 6 months and these Rapid Catch Indicators present the snapshot in January 2004. The following Indicator descriptions were revised/reworded as follows: 3. Percentage of children age 0-11 months whose delivery was attended by a skilled health personnel up to TBA level 4. Percentage of mothers who received at least two tetanus toxoid injections (card confirmed) before the birth of the youngest child less than 12 months of age 7. Percentage of children age 12-23 months who received BCG, DPT3, OPV3 and measles vaccines before the first birthday 10. Percentage of mothers of children age 0-23 months who know at least three signs of childhood illness that indicate need for treatment 12. Percentage of mothers who know at least one method of HIV/AIDS and STD transmission AND percentage of mothers who know at least one method of HIV/AIDS and STD prevention 13. Percentage of mothers of children age 12-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 40 ATTACHMENT H. Findings of Four Lot Quality Assessments # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) BREAST FEEDING AND CHILD NUTRITION INDICATORS 1 Percent of children aged 0-11 months who were breastfed with in the first hour after birth 9 4.87 18 6.58 29 7.74 32 7.90 2 Percent of infants aged 0-5 months who were fed breastfed milk only in the last 24 hours 62 8.26 95 5.15 83 8.94 90 6.73 3 Percent of infant aged 6-9 months who received breast milk and solid foods in the last 24 hours 73 15.82 64 15.69 82 10.84 81 13.91 4 Percent of children aged 20-23 months who are still breast feeding 77 14.72 87 11.80 74 14.48 78 15.68 5 Percent of children aged 6-23 months who received a vitamin A does in the last six months 91 4.17 88 4.68 98 2.07 98 2.52 CHILDHOOD IMMUNIZATION INDICATORS 41 # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) 6 Percent of children aged 12-23 months who have a vaccination card 19 6.64 25 7.34 36 8.16 44 8.43 7 Percent of children aged 12-23 months who received DPT 1 16 6.20 18 6.54 35 8.12 43 8.41 8 Percent of children aged 12-23 months who received measles vaccine 11 5.22 14 5.81 29 7.68 38 8.26 9 Percent of drop out- rates between DPT1 and DPT 3 14 14.97 17 14.91 2 4.04 10 7.84 10 Percent of children aged 12-23 months who received BCG, DPT3, OPV3 and measles vaccines before the first birthday 10 5.05 11 5.38 29 7.68 34 8.04 11 Percent of children aged 12-23 months who received OPV 3 14 5.95 16 6.20 35 8.08 37 8.20 SICK CHILD 12 Percent of mothers of children aged 0-23 months who know at least THREE signs of childhood illness that indicate the need for treatment 73 5.34 62 5.83 80 4.77 83 4.51 DIARRHEA INDICATORS 13 a Percent of children aged 0-23 months with diarrhea in the last two weeks 20 4.83 18 4.58 17 4.51 20 4.77 13 b Percent of children aged 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/ or recommended home fluids (RHF) 16 9.78 13 9.54 36 13.99 35 12.93 42 # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) 14 Percent of children aged 0-23 who breastfed same amount or more during diarrhea in last two weeks. 62 12.84 62 13.90 82 11.17 87 9.28 15 Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount (or more) fluids during the illness 24 11.23 34 13.55 71 13.24 63 13.09 16 Percent of children aged 0-23 months with diarrhea in the last two weeks who were offered the same amount (or more) food during the illness 27 11.77 28 12.79 62 14.17 54 13.55 17 Percent of children aged 0-23 months with diarrhea in the last two weeks whose mothers sought outside advice or treatment for the illness 76 11.23 72 12.79 80 11.69 81 10.71 18 Percent of mothers who can correctly prepare ORS 34 8.60 39 8.29 64 8.16 71 7.74 19 Percent of mothers who usually wash their hands with soap or ash before food preparation. 23 7.10 22 7.02 40 8.32 64 8.16 20 Percent of mothers who usually wash their hands with soap or ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated. 5 3.80 4 3.23 18 6.54 26 7.41 ARI IDICATOR 43 # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) 21 Percent of children aged 0-23 months with cough and fast / difficult breathing in the last two weeks who were taken to a health facility or received treatment. 79 9.99 73 10.82 75 9.89 61 9.71 PRENATAL CARE INDICATORS 22 Percent of mothers with a maternal card (card￾confirmed) for the youngest child less than 12 months of age 17 6.43 22 7.02 26 7.48 24 7.26 23 Percent of mothers who received at least TWO tetanus toxoid injections (card confirmed) before the birth of the youngest child less than 12 months of age. 13 5.67 21 6.93 18 6.80 20 7.16 24 Percent of mothers who had at least ONE prenatal visit prior to the birth of youngest child less than 12 months of age 45 8.46 53 8.48 56 8.43 66 8.04 25 Percent of mothers who received /brought iron supplements while pregnant with the youngest child less than 12 months of age. 36 13.30 53 8.48 74 7.41 77 7.19 PLACE OF DELIVERY AND DELIVERY ATTENDED 26 Percent of children aged 0-11 months whose delivery was attended by a skilled health personal up to TBA level 32 7.95 47 8.49 53 8.48 68 7.90 44 # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) 27 Percent of children aged 0-11 months whose delivery involved use of a clean birth kit or whose cord was cut with a new razor 96 3.23 95 3.53 28 Percent of children aged 0-11 months whose delivery involved use of a clean birth kit 30 7.79 51 8.50 29 Percent of children aged 0-11 months who were immediately breastfed by the mother immediately after birth. 2 2.52 14 5.95 30 Percent of children aged 0-11 months who were placed with the mother immediately after birth 25 7.34 28 7.62 51 8.50 66 8.04 POSTPARTUM CARE 31 Percent of mother who had at least ONE postpartum check-up 11 5.38 14 5.81 15 6.07 31 7.85 32 Percent of mothers able to report at least TWO known maternal danger signs during the postpartum period 41 8.37 52 8.49 33 Percent of mothers able to report at least THREE known neonatal danger signs 91 4.87 90 5.05 34 Percent of mothers able to report at least TWO known neonatal danger signs 71 7.68 87 5.67 96 3.23 98 2.52 35 Percent of mothers who received a vitamin A dose during the first six weeks after delivery 16 6.20 17 6.43 34 8.04 42 8.39 36 Percent of mothers who received at least 30 iron tablets during the first two months after delivery 10 5.05 28 7.62 42 8.39 52 8.49 45 # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) CHILD SPACING 37 Percent of non pregnant mothers who desire no more children in the next two years or are not sure, who are using a modern method of child spacing 24 7.54 32 7.90 44 8.44 43 8.41 38 Percent of mothers who report at least one place where she can obtain a method of child spacing 54 8.47 71 7.74 81 6.64 93 4.27 39 Percent of children aged 0-23 months who were born at least 24 months after the previous surviving child 58 11.10 58 10.96 56 10.74 68 11.37 40 Percent of children aged 0-23 months who were born at least 36 months after the previous surviving child 12 7.26 13 7.42 21 8.77 23 10.24 KNOWLEDGE OF DANGER SIGNS DURING PREGNANCY, POSTNATAL AND NEW BORN CHILD 41 Percent of mothers (15-49 years) who know at least TWO danger signs/symptoms during pregnancy 26 7.48 41 8.35 87 5.67 90 5.05 42 Percent of mothers (15-49 years) who know at least THREE danger signs/symptoms during pregnancy 29 7.68 41 8.35 69 7.85 74 7.48 43 Percent of mothers who knows at least TWO postpartum danger signs/ symptoms 41 8.37 52 8.49 80 6.74 46 # Indicator Oct'01 Confidence Interval (±) Jan'03 Confidence Interval (±) Jul'03 Confidence Interval (±) Jan'04 Confidence Interval (±) 44 Percent of mothers who knows at least THREE postpartum danger signs/ symptoms 11 5.22 62 8.23 59 8.37 45 Percent of mothers who know at least TWO danger signs of new born 71 7.68 87 5.67 96 3.23 98 2.52 46 Percent of mothers who know at least THREE danger signs of new born 37 8.20 62 8.23 91 4.87 90 5.05 DANGER SIGNS OF PNEUMONIA AND DIARRHEA 47 Percent of mothers who know at least THREE danger signs/symptoms of pneumonia 15 6.07 39 8.29 71 7.74 85 6.07 48 Percent of mothers who know at least THREE danger signs of diarrhea /dysentery 14 5.95 17 6.31 50 8.50 62 8.26 KNOWLEDGE OF HIV/AIDS/STD 49 Percent of mothers who knows at least ONE method of HIV/AIDS and STD transmission 14 5.95 18 6.54 28 7.62 37 8.20 50 Percent of mothers who knows at least ONE method of HIV/AIDS and STD prevention 14 5.95 17 6.31 26 7.48 36 8.16 ATTACHMENT I. Work Plan (July 2004 - September 2005) Plan Nepal Child Survival project Action Plan for the period of July 2004 to September 2005. SN Activities Responsible Persons Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep 1 Biannual LQAS survey CS Project Coordinator and HMIS Coordinator * * * 2 Refresher training on CB-IMCI to Female Community Health Volunteers TC & CHOs * * * 3 Refresher training on Maternal and Newborn Care to Female Community Health Volunteers TC & CHOs * * 4 CB-IMCI Training for the remaining staffs of Local Health Facilities Training Coordinator * * 5 Community Drug Program (CDP) Training to Health Facility (HF) Staffs Sr.CHO, TC * * 6 CDP Orientation and Community level Training CHO * * 7 Facilitate to implement CDP at Local HF of Bara District Sr.CHO/CHO * * * * * * * * * * * * * * 8 Formation and CHO/ACHO * * * * * * * * * * * * * * * 48 SN Activities Responsible Persons Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep strengthening of PWG and disseminating CS health massage through it. 9 Support to reactivate PHC outreach services and establish of linkages to pregnant women’s groups CHO/ACHO * * * * * * * * * * * * * * * 10 Iron folic acid distribution to pregnant, lactating and adolescence girl. CHO/ACHO * * * * * * * * * * * * * * * 11 Operational research study to assess effeteness of pregnant women’s groups in increasing coverage of services targeted to pregnant and lactating women under￾five children. Project Coordinator, Sr.CHO * * 12 On site coaching on CS related skill to the community health volunteers and Community Health Worker (CHW) by using supervisory checklist. CHO/ACHO * * * * * * * * * * * * * * * 13 Monitoring and supervision of local HF CHO/ACHO * * * * * * * * * * * * * * * 49 SN Activities Responsible Persons Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep by using integrated checklist. 14 IEC material production Flip chart, Bill board, training on Mithila art BCC Coor. * * 15 Dissemination of CS key health massage through various BCC strategy (street drama, Bill board installation, Mithila art painting) BCC Coor * * * 16 CS massage broadcast from Local FM radio BCC Coor * * * 17 Support on Measles campaign and Polio upping up program All CS staffs * 18 Technical and drug Support on National Vitamin A Distribution campaign and De￾worming campaign to the districts. All CS staffs * * 19 LQAS Training to the NGO and DHO staffs Project Coordinator, HMIS Coordinator * 20 Support to strengthen and formation of child clubs and mobilizing them in the health activities. CHO/ACHO * * * * * * * * * * * * * * * 21 Health information data collection, analysis and Sr.CHO/HMIS Coor. * * * * * 50 SN Activities Responsible Persons Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep presentation 22 Support to organize RHCC meeting to strengthen RH Program in District Project Coordinator, Sr. CHO * * * * * * * 23 Support to organize Quality Assurance Management Committee Meeting in District Project Coordinator, Sr. CHO * * * * * * * 24 Drug supply to the District Health Office for epidemic management Admin and Finance Officer * * 25 Organizing joint supervision with the District Health Office CS Project to the local Health Facilities and community level Project Coordinator and Sr. Community Health Officer * * * * * * * * * * * * * * * DI. Staffs Training DII. 26 Computer Basic for ACHOs Training Coordinator * 27 Computer Training on EP Info. 2000 & SPSS for Supportive Staffs Training Coordinator * 28 Report Writing Training Basic Training Coordinator * 29 Report Writing Training Advance Training Coordinator * 30 PRA Refresher training Training Coordinator * 51 SN Activities Responsible Persons Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep 31 CCCDA Training Training Coordinator * 32 Proposal / Report Writing Training Basic Training Coordinator * 33 Advance HMIS Training Training Coordinator * 34 Office Management Training Coordinator * 35 Exposure Visit Project Coordinator, TC * 36 ToT on Rights of Child Training Coordinator * 37 ToT on Child to Child Training Coordinator * Acronyms: LQAS – Lot Quality Assurance Sampling, HMIS – Health Management Information System, CS – Child Survival, TC – Training Coordinator, CHO – Community Health Officer, ACHO – Assistance Community Health Officer, HF- Health Facility, RHCC- Reproductive Health Coordination Committee , BCC Coor.- Behavior Change Communication , CCCDA – Child Center Community Development Approach, PRA – Participatory Rural Appraisal, CDP – Community Drug Program, PHC – Primary Health Care, IEC – Information Education and Communication, ToT – Training of Trainers, CB-IMCI – Community Based Integrated Management of Childhood Illness. NGO – Non-governmental Organization, DHO – District Health Office 52 ATTACHMENT J. Mid-term Evaluation Questionnaires Community-based Volunteers 1. How long have you been working as a volunteer? 2. What training have you received from the Child Survival project? 3. How has the training helped you in your work? 4. Who comes to support you in completing your activities? How often do you receive support? 5. How have their visits been helpful to you? 6. When did people from community last visit you to get any health services or commodities? Tell me about the visit. 7. How does the community support you in your work? 8. What problems do people face in using your services? 9. What problems do you face in your day-to-day work? How are these resolved? 10. How has your ability to deliver services improved in the last 2 years? 11. What materials do you have to educate people in the community? How do you use them? 12. What commodities do you sell? How much do you charge for them? 13. Who decided that these commodities could be sold in the community? 14. How do you get supplies (such as oral rehydration solution packets, iron tablets, vitamin A, family planning supplies)? 15. How do community members feel about the selling of supplies? 16. How often do you meet as a group and how do you plan your activities? 17. How does the health facility support you in your work? 18. How does the Village Development Committee support you in your work? 19. How can the support you receive be improved? 20. What changes have occurred in the health status of the community in your area in the last two years? 21. How do you plan to continue your work after the Child Survival project finishes? 22. How can Child Survival project activities be improved in the future? 53 Health Workers 1. How long have you been working as a health worker? 2. What training have you received from the Child Survival Project? 3. How has the training helped you in your work? 4. What are your activities? 5. How often do you provide support to community-based volunteers? What are the areas of support (mothers' groups, supplies etc)? 6. How many mothers' groups and pregnant women's groups are there in your village development committee? How often do you attend their meetings? 7. How has your ability to deliver services improved in last 2 years? 8. What educational materials have you received? How do you use them to educate community members? 9. How would you like to improve your services? Chief of Health Facility 1. In what heath related activities have you involved the community? 2. Describe the participation of community members in these activities. 3. What barriers have you faced in mobilizing communities? Give an example of how you solved them. 4. How effective is the project's behavior change communication approach? 5. How has the health facility improved over the last two years? Give an example. 6. How will Child Survival project activities continue after the project finishes? 7. What tools do you use for training needs assessment? 8. What types of method do you use for training? 9. How do you follow-up with participants after training? 10. How frequently do you supervise staff members who report to you? 11. What methods do you use during supervision? 12. Please give an example of a work-related problem faced by your staff during the last month. Tell me how it was solved. 13. What health-related information do you and your staff collect? How frequently are these collected? How are they analyzed? 14. How do you use the information to track the effectiveness of the project? 15. How is the information helpful in decision making? 16. Have you received training in community-based integrated management of childhood illness? 17. How are you using the skills you learned during the training? 18. What are the benefits of community-based integrated management of childhood illness? 54 19. How can community-based integrated management of childhood illness be improved? 20. How is community-based integrated management of childhood illness helping other national health programs?