Midterm Evaluation of the Landak Child Survival Project July 29 - August 8, 2002 Report prepared by Marc Debay On behalf of the MTE Team FAO-A-00-99-00027-00 World Vision i LCSP Midterm Evaluation Table of contents Executive summary ...................................................................................................................... iii I. Assessment of progress towards objectives ........................................................................ 1 A. Technical approaches .......................................................................................................... 3 (1) Malaria control ............................................................................................................ 3 (2) Immunization .............................................................................................................. 7 (3) Diarrhea diseases control ............................................................................................ 9 (4) Acute respiratory infections control .......................................................................... 11 (5) Vitamin A deficiency control .................................................................................... 12 B. Cross-cutting approaches .................................................................................................. 14 (1) Community mobilization and advocacy ................................................................... 14 (2) Behavior change communication .............................................................................. 15 (3) Capacity building ...................................................................................................... 17 (4) Health workers performance improvement ............................................................... 20 (5) Sustainability ............................................................................................................. 22 C. Program management........................................................................................................ 24 (1) Planning, monitoring and evaluation ........................................................................ 24 (2) Human resources management ................................................................................. 25 (3) Financial management and logistics ......................................................................... 27 (4) Information management .......................................................................................... 28 (5) Technical and administrative support ....................................................................... 30 D. Conclusions and recommendations ................................................................................... 32 E. Results highlights .............................................................................................................. 36 II. Action Plan ....................................................................................................................... 37 ATTACHMENTS ....................................................................................................................... 38 A. Baseline Information from the DIP ................................................................................... 39 B. 2002 Demographic and Health Services Profile of the Project Area ................................ 62 C. MTE Terms of reference ................................................................................................... 63 D. MTE Team Members ........................................................................................................ 66 E. MTE Methodology ............................................................................................................ 68 F. MTE Field Visits Guidelines ............................................................................................ 71 G. MTE Individual Interview Guideline for LCSP staff ....................................................... 74 H. MTE Schedule ................................................................................................................... 75 I. List of key findings from MTE KPC Survey .................................................................... 77 J. Project implementation timeline: July 2000 – August 2002 ............................................. 80 K. Milestones of the Landak Child Survival Project ............................................................. 81 L. Follow up on recommendations from First Annual Review ............................................. 85 M. Training events attended by community members ....................................................... 90 N. Training events attended by LCSP staff............................................................................ 92 O. ADP Pontianack – Landack Child Survival Project Organizational Chart ....................... 94 World Vision ii LCSP Midterm Evaluation Acronyms ADP Area Development Program ARI Acute Respiratory Infections BCC Behavior Change Communication CBDDS Community-based Disease and Death Surveillance CU Credit Union DHO District Health Office DIP Detailed Implementation Plan EPI Expanded Program of Immunization HC Health Center HIS Health Information System HIV Human Immuno-Deficiency Virus IMCI Integrated Management of Childhood Illness ITN Insecticide Treated Nets KPC Knowledge, Practices and Coverage survey LCSP Landak Child Survival Project KSM Community Self Help Group MCH Maternal Child Health MOH Ministry of Health MTE Midterm Evaluation NGO Non-Governmental Organization ORT Oral Rehydration Therapy PVO Private Voluntary Organization TBA Traditional Birth Attendant TOT Training of Trainers USAID United States Agency for International Development VHC Village Health Committee VHV Village Health Volunteers WV World Vision WVAPRO World Vision Asia and Pacific Regional Office WVIDN World Vision Indonesia Indonesian terms commonly used: Desa Village Posyandu Village Health Post, or Integrated Service Post Posyandu Cadre Village Health Volunteer Pos Obat Desa (POD) Village Drug Post Polindes - Bidan Village Birthing Post with a midwife Polindes + Bidan Village Birthing Post without midwife Dusun Sub village Pustu Sub village Health Center Kecamatan Sub district Puskesmas Sub district Health Center World Vision iii LCSP Midterm Evaluation Executive summary World Vision was initially awarded a four-year grant to implement a child survival project in Poso, Central Suwalesi Province (under the name of Rural Entrepreneurs and Advocates for Child Health –REACH, October 1999 – September 2003). As social unrest and insecurity took place in Poso and the entire project area in April 2000, the project site was changed and the Landak Child Survival Project (LCSP) implementation began in July 2000. The LCSP covers two sub districts of the rural area of Landak District in West Kalimantan Province, and the total population is 78,384, with 1845 infants and 21,388 children under five. The goal of this four-year project (2000-2003) is to “To contribute to the reduction of mortality and morbidity among under five children and mothers in two sub-districts of Mandor and Sengah Temila.” The local partners are the provincial and the district-level health management teams and the health centers of the MOH, one local Credit Union (Pancur Kasih), and various Community Self Help (KSM) groups. The main strategies defined in the LCSP Detailed Implementation Plan (DIP) are to (1) Strengthen the quality and coverage of existing child survival programs in the area; (2) Empowerment and equip communities for the prevention, early & complete management of common diseases; (3) Initiate and establish linkages between micro enterprise activities and health; (4) Integrate child survival activities to Area Development Programs’ activities and programs; and (5) Become a demonstration site and multiplier to scale up. The LCSP initially focused on four technical interventions: immunization, vitamin A supplementation, control of diarrheal diseases, and control of malaria. The baseline assessment was conducted in September 2000 and the DIP workshop was held in October 2000. The DIP was approved with a few recommendations including the addition of Acute Respiratory Infection control as an intervention, the need to adopt process, quality of care and capacity building monitoring indicators and to revise the malaria indicators, and the need to develop a plan for scaling up at the level of the district. A First Annual Review of the LCSP was conducted in September 2001 by a team led by one external consultant assisted by the LCSP staff and representatives from the MOH, USAID Jakarta, and World Vision Landak, Jakarta, and the Asia and Pacific Regional Office. The main recommendations of the review were to accelerate the integration of the LCSP into the Landak Area Development Program, to increase the number and quality of the Posyandus, to increase the quality of health center services, to develop capacity building indicators, and to implement the marketing and use of long-lasting insecticide treated nets. The MTE Team comprised the LCSP staff members, other ADP staff members, and representatives of the MOH RI, USAID/Indonesia, World Vision, and the external evaluator and team leader. Prior to the MTE, the LCSP project conducted a series of surveys and also prepared summary information on project operations and accomplishments. The fieldwork ran from July 30th through August 8th, 2002. The MTE Team of about twenty members discussed the problems and developed the field data collection tool before spending 3 days of site visits conducting group discussions, interviews, and observations. The entire MTE Team met again to discuss the World Vision iv LCSP Midterm Evaluation findings and develop recommendations, and prepare the various presentations made to the sub District, District, and Provincial Health Offices, and to the Ministry of Health in Jakarta. The main findings of the MTE are the following:  Most field activities began when the entire project team was on board in April 2001. During the first year, the LCSP began reactivating and creating Posyandus and training their cadres, and provided logistical support to the health centers. The level of activity increased in the second year, with the training program including TBAs and shopkeepers in addition to Posyandu cadres. Technical approaches  The baseline and midterm KPC surveys show an increase in the coverage of immunization of infant and pregnant women, postpartum vitamin A supplementation, use of ORS, early treatment of children with fever, and an increase in the knowledge of mothers of young children about the signs of pneumonia. However, postpartum vitamin A supplementation, immunization of infant and pregnant women, use of ORS, dietary management of diarrhea, health care seeking for children with fever and malaria or with signs of pneumonia, and the use of ITN by children are still well below the project objectives. Crosscutting approaches  There are gaps in the quality of services at the Posyandus level such as occasional shortages of vitamin A capsules; weaknesses in the cold chain and occasional vaccine shortages; missed opportunities for tetanus toxoid immunization; and insufficient supply of ORS.  The LCSP made significant progress in mobilizing and building the capacity of its partners through their training and involvement in various project activities, but much remains to be done to ensure their full participation in and ownership of project activities.  The number of functional Posyandus increased from 76 in July 2000 to 100 in July 2002, and 80% of the population in the project area now has access to a functional Posyandu.  One thousand community members involved in health activities have been trained in specific CS interventions and related skills. Selected DHO and most of the HC staff has been trained in training skills, malaria control, and IMCI. The LCSP provided logistical support to the health centers for their facility-based and for their outreach activities, including supervision. These activities have improved and will continue to improve the quality of care in health facilities and Posyandus.  The LCSP has achieved positive changes in specific behaviors primarily through training, equipping with BCC materials, and encouraging Posyandu cadres to educate their communities. However, the type and use of BCC materials and methods may not always have been the most effective for the communities in the project area. Program management  The LCSP staff and activities are progressively integrated into the ADP Pontianack management structure, and the LCSP maintains good management and financial practices.  The LCSP has motivated and competent technical and managerial staff, but has experienced difficulties in recruiting and maintaining this staff. All the current LCSP staff shows high World Vision v LCSP Midterm Evaluation potential for professional development, and most of them had opportunities to participate in relevant training events through the LCSP.  The LCSP conducted a series of assessment including two KPC surveys, two health facility assessments, and PLA assessments in 15 communities. These studies were well-conducted and documented but may not include full analyses and interpretation of the data.  The LCSP began introducing several revised and new community-based health information systems and tools. Although they are all relevant and promising, pilot testing and achieving sustainable implementation of these systems will require time and careful follow-up.  WVIDN, WVAPRO and WVUS provided continued and constructive technical and administrative support to the LCSP, but there remains a need for assistance in setting this new project as a demonstration site for later scale up at the Landack District level.  The LCSP project has been implemented for two of its three years of duration with one third of its initial budget. The MTE made the following main recommendations: To the LCSP and its partners  Conduct joint visioning exercises at the village level to increase the support from community leaders to project activities and integrate health in community development activities.  Continue increasing the number of functional Posyandus and improving the quality of their services using supervision guidelines to assess the performance of the cadres.  Assist the DHO and HC in the management, analysis, and use of the data collected by the Posyandu cadres and health centers staff.  Assist the DHO in the development, adoption, and implementation of policies regarding ORS, vitamin A and vaccine supply.  Support the introduction and implementation of IMCI in the Landack District, and particularly at the Posyandu level.  Develop a phase-out plan for the last year of the project implementation. To WVIDN, WVARO, and WVUS  Continue investing in the professional development and management of the LCSP to ensure job satisfaction and performance.  Provide focused technical backstopping and assistance in the definition of explicit capacity building activities for the LCSP partners; development of a behavior change strategy and action plan; development of a performance improvement plan for Posyandu cadres; development of protocols for specific studies; introduction of credit with education within Pancur Kasih Foundation.  Assist the LCSP team in the development and adoption of a definite table of project monitoring indicators.  Begin planning for scaling up project activities at the Landak District level. One step in this process should be a lessons learned workshop with the LCSP partners.  Request a 2-year no cost extension of the LCSP, from October 2003 to September 2005 to scale up the activities at the level of the Landak DHO and achieve the capacity building and sustainability objectives of the project. World Vision 1 LCSP Midterm Evaluation I. Assessment of progress towards objectives World Vision was initially awarded a four-year grant to implement a child survival project in Poso, Central Suwalesi Province: the Rural Entrepreneurs and Advocates for Child Health (REACH) project, October 1999 – September 2003. As social unrest and insecurity took place in Poso and the entire project area in April 2000, the project site was changed to the Landak District, West Kalimantan Province. The Landak Child Survival Project (LCSP) implementation began in July 2000. The objectives, background information, main design features, partnerships, and health information system of the Landak Child Survival Project (LCSP) as described in the Detailed Implementation Plan (DIP), and the related changes since its approval, are reproduced in Appendix A. The goal of this three-year project (2000-2003) is to “To contribute to the reduction of mortality and morbidity among under five children and mothers in two sub-districts of Mandor and Sengah Temila.” The local partners are the provincial and the district-level health management teams and the health centers of the MOH, one local Credit Union (Pancur Kasih), and various Community Self Help (KSM) groups. The Landak Child Survival Project (LCSP) covers two sub districts of the rural area of Landak District in West Kalimantan Province. This project area is further divided into three areas each served by a health center: the Mandor sub District, and the Senakin and Pahauman areas in the Sengah Temila sub District. The estimate of the total population that the LCSP uses is 78,384, with 1845 infants and 21,388 children under five. Selected demographic and health services data for the three project areas are provided in Appendix B. The main strategies defined in the LCSP Detailed Implementation Plan are to:  “Strengthen the quality and coverage of existing child survival programs in the area  Empowerment and equip communities for the prevention, early & complete management of common diseases  Initiate and establish Micro Enterprise and Health Linkage  Integrate to Area Development Programs’ (ADP) activities and programs  Become a demonstration site & multiplier to scale up” The implementation timeline of the LCSP from July 2000, when the decision was made to change the project site, to August 2002, when the midterm evaluation was conducted, is presented in Appendix J. A more detailed listing of the LCSP milestones is provided in Appendix K. The baseline assessment (Knowledge, Practice, Coverage (KPC) and other surveys) was conducted in September 2000, and the DIP and sustainability workshop was held in October 2000. The DIP was approved in December 2000 with a few recommendation including the addition of Acute Respiratory Infection control as an intervention for the project, the adoption of World Vision 2 LCSP Midterm Evaluation process, quality of care and capacity building monitoring indicators, a revision of the malaria indicators, and the development of a plan for scaling up at the level of the district. The First Annual Review (FAR) of the LCSP was conducted in September 2001 by a team led by one external consultant, assisted by several team members from the MOH national, provincial, and district levels, from USAID Jakarta Office, and World Vision Landak, Jakarta, and the Asia and Pacific Regional Office. This team conducted a series of field visits including staff interviews and observations, exit interviews in various communities and health centers. The main recommendations of the review, which include responses to recommendation from the DIP review, are to accelerate the integration of the LCSP into the Landak Area Development Program, to increase the number and quality of the Posyandus, to increase the quality of health center services, to develop capacity building indicators, and to implement the marketing and use of long-lasting insecticide treated nets. The status of their implementation as reported by the LCSP is presented in Appendix L and discussed in the present report when appropriate. The LCSP had to recruit an entire new team (except for the Project Officer and the Financial Officer) when it relocated to its new site. Only in April 2001 was the entire team of Officers and health motivators hired and ready to begin field activities. WVIDN then had to continue recruiting for the LCSP as several staff members resigned since then and until recently. Between March 2001 and June 2002, a total of 923 community members (524 Posyandus cadres, 22 POD cadres, 185 TBAs, 192 shopkeepers, and 106 KSM members), and most of the staff in the GOI health centers of the project area (about 40) have been trained at least once in a variety of LCSP related topics. Most of the LCSP staff participated in various relevant training programs, in Indonesia and overseas, between October 2000 and February 2002. The LCSP managed to provide a series of equipment and technical health-related materials in the project area. In September and October 2001, the LCSP handed over MCH booklets and portable generators. On May 28, 2002, the District Head of Landak launched the Long-Lasting Insecticide Treated Nets program in Ngabang Town House. This event was an opportunity to make available 2000 ITNs and engage 32 self-help groups in their distribution in their respective communities. The LCSP also purchased 100 TBA kits, 15 vaccine flasks, and various health education materials. The methodology of the MTE is described in Appendix E. The following sections present the results of the analyses conducted by the MTE team according to the general outline proposed in the USAID/DCHA/PVC-CSGP guidelines: technical interventions, cross-cutting approaches, and program management. For each topic within these three main categories, the findings are presented as accomplishments, challenges, and recommendations. Section III of the report simply summarizes these findings. World Vision 3 LCSP Midterm Evaluation A. Technical approaches The LCSP DIP focused on four technical interventions: immunization, vitamin A supplementation, control of diarrheal diseases, and control of malaria. Upon recommendations from the DIP review, the LCSP staff added the control of acute respiratory infections (ARI) as a fifth technical intervention. The micro enterprise development (MED) activities presented in the DIP as an intervention are discussed in this MTE report under section I.B (3) on capacity building. The five LCSP interventions are discussed below by descending order of importance as estimated by the author of this report. This may differ from the percentage of effort in the DIP and reproduced in Appendix A since the ARI intervention was added and efforts on malaria and IMCI may surpass those initially expected. The five sections below begin with a summary table of the objectives and the related baseline and assessments indicators from the baseline and midterm surveys, followed by a summary of the key strategies. (1) Malaria control Objectives and related baseline and midterm assessments for the malaria intervention End-of-project Objective Sept ’03 Baseline Midterm Sept '00 KPC July '02 Target Sept '02 1. 75% of mothers with child less than 2 years of age who was ill with fever during the past two weeks seek treatment for their child 26.0% 57.8% 60.0% 2. 45% of children less than 2 years of age with a febrile episode that ended during the last two weeks were brought to a health facility within 48 hours after the fever began Not available 29.9% Not available 3. 30% of mothers with a child less than 2 years of age report that they have an ITN in their house Not available 4.5% Not available 4. 20% of children less than 2 years of age slept under an ITN the previous night Not available 1.6% Not available Note on the indicators: The DIP only reports indicator 1 as baseline; indicators 2, 3, and 4 were adopted during the FAR and data collected during the midterm KPC survey. The two indicators of case management of malaria (1 and 2) are based on the children who had fever within last 2 weeks prior to the survey (99 and 187 in the baseline and midterm surveys, respectively), and therefore on relatively small samples. The end-of-project objectives of the three new indicators are recommendations from the MTE team. The LCSP malaria control strategies are:  Health education of mothers and other community members in early recognition and treatment of malaria and in the use of ITNs by children under five and pregnant mothers.  Early diagnosis and effective treatment of cases of malaria by health providers  Social marketing of ITNs World Vision 4 LCSP Midterm Evaluation Achievements The increase in appropriate health care seeking behavior for fever evidenced by the KPC surveys is encouraging. Health staff in the health centers, Polindes and Pustus have been trained in the diagnosis and treatment of malaria in May 2001, and it is likely that the cases of malaria are better diagnosed and treated in these facilities. LCSP also plans to have trained 30 midwives and health center staff by the end of September 2002, and this will also improve the diagnosis and case management of malaria in the health centers. One laboratory technician from the District Health Office has been trained in the microscopic diagnostic of malaria. The LCSP managed to introduce Long Lasting Impregnated Bed Nets (PermaNet®), thereafter referred to ITNs, in the project area and create demand for this new product through training and education of community members and leaders. Two thousand units were available at the ADP office when the LCSP organized an ITN “Launching” event that took place on May 28, 2002, in Ngabang Town House and with the District Head of Landak. WVIDN and 32 KSMs from Mandor, Sengah Temila and Toho sub-districts (the ADP Pontianak’s areas) signed a MOU stating that the KSMs would assist distributing the ITNs in their areas with the agreed price, and that the profits would be used to buy more ITNs. The DHO also purchased an additional 1000 ITNs. The use of KSM groups for the distribution of ITN to shopkeepers and other selling points promises to be a sustainable distribution mechanism. Several KSM groups have started selling ITNs. In addition to the KSM groups, the LCSP considers the Credit Union from the Pancur Kasih foundation as one possible partner to distribute ITNs. The Pancur Kasih coordinator in Sidas expressed interest in such activity, and in particular as a way to decrease the number of medical claims by subscribers to their new health insurance system. He recognized, however, that the population would have to be well informed of the benefits of ITNs before they start buying them. Challenges Even though health care seeking behavior for fever and malaria improved, the MTE team found that mothers of children under two years of age do not have adequate knowledge of the correct dosage and duration of the treatment of malaria with chloroquine. Shopkeepers regularly report that their clients come and ask for one or two chloroquine tablets, irrespective of their advice to buy a complete treatment. There is no capability in microscopic diagnostic of malaria in the health centers, even though there are microscopes and laboratory technicians in each center. The MOH does not recommend intermittent protective treatment of pregnant women by sulfadoxine-pirymethamine (SP) because Plasmodium vivax is responsible for as much infection as Plasmodium falciparum in Indonesia, and the use of SP while CQ is still effective against World Vision 5 LCSP Midterm Evaluation Plasmodium vivax constitutes an unnecessary risk to the pregnant mother and increases the risk to create resistance against SP. The MOH National Malaria Control program is currently collecting evidence on the level of Plasmodium falciparum resistance to chloroquine in different areas of the country. There are reports of significant resistance in East Kalimantan, but no data on the situation in West Kalimantan. Health center staff members in the project area have reported clinical suspicion of chloroquine resistant malaria to the MTE team. The distribution of ITNs started late (ITNs launching event on May, 28 2002), and only 197 ITNs had been purchased at the time of the MTE. The MTE team found that community members are already aware of the benefits and interested in the use of ITNs, but that they do not know how and where to obtain them. Also, KSMs groups have limited skills and knowledge about the possible distribution routes. Finally, there are only 32 KSM groups in the four ADP sub districts, and therefore a large proportion of the target population may have very limited access to ITNs through this distribution mechanism. Illegal gold mining and deforestation practices remain major factors that contribute to the high malaria transmission in the Landak District, and in the Mandor sub district in particular. Gold miners clean (meaning, cut trees in) the area where they think there is plenty of gold, usually near a river where it is easy to find water. As they usually move fast from one location to the other, this practice has already created many big and deep holes. When the rain comes, these holes are filled with rainwater and become suitable breeding places for mosquitoes, including Anopheles. The local government estimated that there are more than one thousand groups of gold miners in Mandor Sub-district only.1 Recommendations The LCSP should conduct participatory strategic planning exercise at the community and district levels focusing on possible local malaria control strategies, including environmental action, and build consensus and commitment for action. The LCSP should continue to research the best strategies for sustainable distribution of ITNs and related coverage objectives. In particular, it should explore the possibilities for Pancur Kasih to become the main distributor for the area in replacement of the Landak ADP; the potential roles of ADP, KSM, Credit Unions (CU), and other community groups; and the role of the village midwives and other health center staff and of the Posyandu cadres in the promotion of the purchase and use of ITNs, maybe as part of a neonatal package of services. The LCSP should design and implement a vigorous malaria behavior change program including campaigns just before the peak malaria season at the end of the rainy season (April –May). 1 Gold miners also use mercury for their gold mining practices and pour large quantities of mercury residues in the nearby rivers. One neighboring province in which there is also a lot of illegal gold mining reported that the mercury concentration in the river was so high that the fish was already contaminated. World Vision 6 LCSP Midterm Evaluation The LCSP should promote effective malaria case management at the community level and:  Train or retrain Posyandu cadres, POD cadres, shopkeepers, TBAs, KSM groups, Credit Union staff, and farmer’s groups in effective malaria treatment and prevention  Develop and distribute job aids for cadres with simple contact description and related treatment regimen. The LCSP should give opportunities for one microscopist in each health center in the project area to receive training or retraining in malaria diagnosis for two weeks in Pontianak, and provide new microscopes to their health centers if needed. The LCSP should assist Landak District Health Office in conducting a malaria epidemiology assessment in the project area and:  Conduct an in-vivo chloroquine resistance study according to the WHO standard protocol and in coordination with the National Malaria Control Program. This study would also be an opportunity to assess proportion of vivax and falciparum malaria cases  Initiate vector surveillance to determine areas where transmission is the highest  Document the role of illegal mining and deforestation on malaria transmission in the project area  Identify socioeconomic (geography, habitat, occupation, culture) differences in malaria epidemiology, including knowledge and behavior in the project area The LCSP should initiate advocacy activities at the District, Provincial and National level to raise awareness of the role of illegal mining and deforestation on malaria transmission in the project area, and induce political and community action. At the national level, the CDC and Environmental Health and the Community Health Departments should be involved. WVI and LCSP should join the National Anti Malaria Coalition (Koalisi Anti-Malaria Indonesia (KAMI), the USAID-funded secretariat hosted by the US Navy Medical Research Unit No 2 and the Ministry of Health. World Vision 7 LCSP Midterm Evaluation (2) Immunization Objectives and related baseline and midterm assessments for the immunization intervention End-of-project Objective Sept ’03 Baseline Midterm Sept '00 KPC July '02 Target Sept '02 1. 80 % of children 12 to 23 months completely immunized (verified by card) 23.0% 46.8% 70.0% 2. 50 % of mothers with child less than 2 years of age received TT2 before the birth of their youngest child (verified by card) 2.0% 14.9% 35.0% Note on the indicators: The two indicators used to assess the progress in immunization coverage are based on evidence from immunization cards at the time of the survey. More reliable than indicators based on recall of the history of immunization by the mothers or care givers, these indicators underestimate the true current coverage. When evidence from the card and history is used to estimate coverage, the proportion of mothers with child less than 2 years of age who received TT2 before the birth of their youngest child is 47.3%, and the proportion of completely immunized children is also higher. The coverage of children 12-23 months is based on 186 children in that age group and the coverage of mothers is based on 449 mothers included in the sample. The LCSP immunization intervention strategies consist in:  Reactivation of the Posyandus and training of the cadres  Behavioral change communication for mothers and other community members  Reduction of missed immunization opportunities for infant and pregnant women  Introduction of the MCH booklet and a new community level register and tracking system  Provision of management and logistic support to the DHO and the HCs Achievements The baseline and midterm KPC surveys demonstrates an increased immunization coverage of children under five and pregnant women. The quality of the cold chain in the health facilities has improved as a result of the LCSP activities such as baseline cold chain assessment; training of health staff; provision of one portable generator and one motorcycles for each Health Center (the maintenance and recurrent cost is to be charged to the health center/district health office) and 15 carriers to bring vaccines to the Posyandus; and enforcement of a temperature tracking system. At the time of the MTE, 100% of health facilities have twice a day cold chain temperature recording, as compared to 20% only at the beginning of the project. In August 2001, the project carried out two immunization sweepings in Agak Hilir and Tampi in Mandor sub-district, because there were no Posyandu in these areas located far from any health facility. The LCSP achievements with respect to the introduction of the MCH booklet and the community EPI and pregnancy tracking forms are discussed in section C. (5). World Vision 8 LCSP Midterm Evaluation Challenges Overall, the coverage (card documented) has increased but remains low. In most areas, immunization services are only available in Posyandus, but antenatal services are typically not offered in Posyandus. This situation results in missed TT immunization opportunities in health centers. Also, it appears that the Posyandus cadres and Polindes bidans do not systematically refer newly pregnant mothers to Posyandus. There are known weaknesses in the cold chain, in safety of injections and disposal of wastes, and in vaccine availability at the Posyandu level. These problems were identified by the LCSP staff in October 2001 through systematic observations of health workers in several Posyandus (see Quarterly Report October – December 2001) and appeared to persist at the time of the MTE. Although the monitoring of the cold chain is satisfactory at the health center level, it remains questionable at the Posyandu level. Vaccine shortages were still reported in several villages in May and June 2002 (see bimonthly report). The estimates of immunization coverage from the DOH services statistics are consistently higher than those from the KPC surveys. Among factors that may explain the discrepancies are the use of underestimated population figures for the service statistics estimates, the different periods of reference of these two estimates,2 the inclusion of children beyond 12 months in the services statistics, and the only recent introduction of the MCH booklets. As the booklets and immunization cards are more widely used, the discrepancies will gradually decrease. Recommendations: Given the success of the reactivated and the new Posyandus in increasing attendance to immunization services, the LCSP should create new Posyandus in uncovered areas. The LCSP should initiate monitoring of the quality of EPI services at the Posyandu level, using observation checklists, exit interviews, and other explicit methods. The LCSP should train Posyandu cadres and TBAs to refer newly identified pregnant women to the Posyandu, Pustus, Polindes, or Health Centers for TT vaccination. The LCSP should promote BCC messages on immunization in KSM, CU, farmers, and mothers groups. The LCSP should continue to facilitate the distribution of MCH booklets for newly identified pregnant women 2 In KPC survey the period of reference period is the two years before the survey, while the service statistics typically cover one calendar year. World Vision 9 LCSP Midterm Evaluation The LCSP should introduce incinerators for safe disposal of sharp and used syringes in Health Centers, Pustus, and Polindes. Another possibility is to use small incinerators, if available in the local market, which could be carried during Posyandu activities. The LCSP should support the National Immunization Days in Landack, while avoiding losing focus on strengthening routine immunization services. The LCSP should reconciliate the immunization coverage estimates from the KPC surveys and from the services statistics to better understand the true coverage of the immunization program, improve the monitoring of future activities, and eventually increase the immunization coverage. (3) Diarrhea diseases control Objectives and related baseline and midterm assessments for the diarrheal diseases control intervention End-of-project Objective Sept ‘03 Baseline Midterm Sept '00 KPC July '02 Target Sept '02 1. 75% of children less than 2 years of age who had diarrhea in the last two weeks received ORS 31.5% 42.9% 60.0% 2. 75% of children less than 2 years of age who had diarrhea in the last two weeks received same amount or more fluids 63.0% 58.6% 75.0% 3. 70% of children less than 2 years of age who had diarrhea in the last two weeks received same amount or more foods 34.0% 29.0% 60.0% 4. 10% of children less than 2 years of age who had diarrhea in the last two weeks received antidiarrheal medicine (antibiotics, traditional medicines, or injection) 40.0% 20.2% 20.0% Note on the indicators: These four indicators of case management of diarrhea are based on the 203 cases of diarrhea among the 449 children included in the survey, and it is likely that the small differences observed between the baseline and midterm KPC surveys for indicators 2 and 3 are not statistically significant. The LCSP diarrheal diseases control strategies consist in:  Education of mothers and other community members in effective home management of diarrhea  Training of health workers and volunteers in standard case management of diarrhea  Support construction and use of latrines and protection of water sources Achievements The increase in the use of ORS and the decrease in the use of antidiarrheal medicines since the beginning of the project suggest that the health education activities combined with the availability of ORS have successfully changed health care behavior of mothers and health providers. World Vision 10 LCSP Midterm Evaluation Challenges The supply of ORS is inadequate in Posyandus. The health center staff typically brings10 packs of Oralit, which obviously cannot meet the need of the cadres and TBA between two Posyandus sessions. Despite an increase in ORS use, the dietary management of diarrhea is still low, suggesting a relative weakness in the health education activities with this respect, or specific barriers to adoption of this behavior. The prevalence of diarrhea during the last 2 weeks among children below 24 months of age is of concern in the project area: 59.5% of mothers of children under 24 months of age at baseline and 45.2% at midterm reported that their child had diarrhea during the last two weeks. However, the KPC indicator of prevalence of diarrhea, primarily used to measure the use of ORS among children with diarrhea, is difficult to compare to any reference prevalence of diarrhea since there is no case definition (typically, 3 or more stools per day), the diarrhea prevalence is highly dependent on age and season, and the sample size is relatively small. Recommendations: The LCSP should assist the District Health Office in the development and adoption of a policy, and in the management and distribution of ORS in health centers and Posyandus. The LCSP should improve dietary management of diarrhea among mothers by strengthening the related interpersonal and counseling skills of Posyandu cadres and health staff. The ongoing training in IMCI of health centers staff, to be followed by that of Posyandus cadres, may already address dietary management of diarrhea. The LCSP should continue the promotion of measures of diarrhea prevention such as breastfeeding, domestic hygiene (hand washing, use of latrines, proper waste disposal) and access to clean water. The LCSP should introduce case definition of diarrhea and more detailed questions on diarrhea prevalence and related behaviors in next KPC surveys, while maintaining the current indicator for comparison purposes. World Vision 11 LCSP Midterm Evaluation (4) Acute respiratory infections control Objectives and related baseline and midterm assessments for the acute respiratory infections intervention End-of-project Objective Sept ‘03 Baseline Midterm Sept '00 KPC July '02 Target Sept '02 1. 80% of mother with children less than 2 years of age who mention at least 1 symptom of pneumonia 24.6% 39.9% Not available 2. 75% of mother with children less than 2 years of age who had symptoms of pneumonia (cough and rapid or difficult breathing) seek treatment 58.8% 58.7% Not available Note on the indicators: The indicator of case management of diarrhea is based on the 108 children with symptoms of pneumonia among the 449 children included in the survey (47 cases among 252 children at baseline). The end-of-project objectives are recommendations of the MTE team. The ARI intervention was added to the LCSP original design in response to a recommendation from the DIP review. There is no explicit strategy for this intervention. Achievements The increase in knowledge of symptoms of pneumonia by mothers of children under 24 months of age suggests that the health education activities have been effective. The two KPC surveys show relatively high levels of health care seeking behavior with respect to symptoms of pneumonia. Challenges The two KPC surveys do not show any change in health care seeking behavior with respect to signs of pneumonia among children under two. The MTE team found that Posyandu cadres have low confidence in their ability to educate mothers on acute respiratory infections and pneumonia, and that midwives do not provide education with this respect during maternal health services (they have not been trained yet). Also, the training curriculum for TBAs do not include case management of ARI, and the LCSP does not have any promotion material on ARI to provide to health workers and volunteers. Recommendations The LCSP should ensure that the ongoing training in IMCI of health centers staff, to be followed by that of Posyandus cadres, effectively address case management of pneumonia. The LCSP should train midwives and Posyandu cadres to target pregnant women with education in early recognition of pneumonia sign and symptom and in timely health care seeking by appropriate providers. World Vision 12 LCSP Midterm Evaluation The LCSP should develop an appropriate curriculum for and train TBAs in early pneumonia recognition and referral as part of newborn care package. The LCSP should find or develop BCC materials on pneumonia early recognition and referral. (5) Vitamin A deficiency control Objectives and related baseline and midterm assessments for the Vitamin A intervention End-of-project Objective Sept ‘03 Baseline Midterm Sept '00 KPC July '02 Target Sept '02 1. 90% of children 12 to 23 months received Vitamin A in the past 6 months 54.0% 60.2% 70.0% 2. 50 % of mother received Vitamin A within one month of their last delivery 12.0% 25.6% 30.0% Note on the indicators: The indicator of coverage among children 12-23 months is based on the 186 children in that age group and the coverage of mothers is based on the 449 mothers included in the sample. The indicator of coverage of mothers refers to the past 24 months and may be less sensitive to recent change than the indicator of coverage for children 12 to 23 months, which refers to the past 6 months. The LCSP vitamin A deficiency control intervention consist in:  Education of mothers and other community members on the benefits and timing of vitamin A supplementation  Training of health workers and volunteers on the benefits and mode of administration of vitamin A supplementation  Support to District Health Office and health centers staff for the supply and distribution of vitamin A capsules  Promotion of cultivation of vitamin A rich food Achievements The KPC survey shows an increase in the vitamin A supplementation coverage of postpartum mothers. Since this intervention has only recently been introduced, the recent coverage may be higher that the one estimated for the 24 months prior to the KPC survey. The LCSP staff supports the vitamin A supplementation intervention during regular Posyandu activities, but also during vitamin A supplementation days. Challenges The KPC survey does not show significant increase in coverage of vitamin A supplementation for children aged 12 to 23 months. During the February 2002 round of mass distribution of vitamin A capsules, the Puskesmas Service Statistics estimated that 75.8% of children 6 to 59 months had received Vitamin A. The discrepancy may be due to the use of low estimate of the population as denominator in the service statistics coverage, and to the fact that supplementation is often not recorded on the immunization cards or the MCH booklet. World Vision 13 LCSP Midterm Evaluation Shortages of vitamin A (blue) capsules for children 6-11 months are reported in many Posyandus, and in the Mandor sub district in particular. Shortages of this and the other vitamin A capsule occasionally occur when the attendance to Posyandus increase more than expected by the health center staff. There may also be problems with the supply at the health center or DHO level. The MOH RI recommends to track vitamin A supplementation coverage among children 6-11 months of age because this age group is often not covered as expected. The current KPC question about vitamin A supplementation underestimate the true coverage among children 6-11 months of age since many children in that age group may not have had the chance to get their supplement at the time of the survey but may still get it before their first birthday. A remedy to this issue is to ask the age at which each capsule was received to all children 12-23 months of age like it is done to estimate immunization coverage of infants. The vitamin A supplementation coverage for postpartum mothers is still low, and there is no concrete plan to improve procurement and distribution of vitamin A capsules to this target group. Recommendations The LCSP should assist the District Health Office in the development of a plan of action for postpartum vitamin A supplementation that includes BCC and supply. Although it is not the MOH policy to have TBA distribute vitamin A capsules, the LCSP should explore this and other option for involving TBA in this intervention. The LCSP and the DHO should improve the recording of vitamin A supplementation on immunization cards and MCH booklets (growth cards and MCH booklets have a column for this), and consider introducing a sticker system to track vitamin A supplementation on the EPI tracking form. The LCSP and the DHO should promote vitamin A messages during vitamin A months in February and August. World Vision 14 LCSP Midterm Evaluation B. Cross-cutting approaches (1) Community mobilization and advocacy Achievements The LCSP staff has conducted various valuable strategic community mobilization activities such as involving community members in the baseline assessments and in joint planning exercises to set end-of-project targets and identify respective roles in project implementation. The LCSP also increased the number of cadres and reactivated and created new Posyandus in remote areas (see capacity building section). The LCSP staff also purposely builds on the achievements of and collaborates with the ADP motivators in training KSM members and holding meetings with community leaders and farmer groups. Challenges The LCSP staff and Posyandu cadres often mention insufficient support for their activities from the community leaders. There is often poor attendance in LCSP-supported activities such as the training offered to shopkeepers and TBAs. Each community has many different stakeholders, but usually no unifying perception that health is linked to development. While some community members would like to link health and community development, they usually do not know how to do it. While key stakeholders such as KSM group and CU members are working closely with the ADP, they do not see themselves as working closely with the LCSP. The community mobilization strategy is not fully articulated, and there is no indicator to monitor progress and achievements with this respect. Recommendations The ADP staff, including that from the LCSP, should conduct visioning exercises at the villages/Dusun level to galvanize all stakeholders and articulate a unified vision for health and development. These exercises should involve village leaders, religious leaders, teachers, cadres and KSM groups. ADP Pontianak already has one staff member trained as facilitator of such visioning exercises. The ADP staff, including that from the LCSP, should develop a common community mobilization strategy and related indicators. WV should strengthen LCSP staff’s communication and advocacy skills to influence community organizations and government health staff. World Vision 15 LCSP Midterm Evaluation (2) Behavior change communication The DIP characterizes the LCSP behavior change approach as follows:  Emphasis on household and community behaviors  Selection of behaviors to change and development of specific messages with the health center staff and community members  Provision of information and education to all community members  Creation of an enabling environment that supports sustained behavior change  Use of non-visual and participatory methods to deliver messages Achievements The LCSP has already achieved positive behavioral change in the project area. The two baseline and midterm KPC surveys showed improved health care seeking behavior for fever and malaria, increased use of ORS for diarrheal diseases, and increased use of preventive health services such as immunization and vitamin A supplementation. The LCSP staff trained Posyandu cadres in the provision of health education talks about key behaviors during Posyandus sessions or other community gathering, and in counseling during individual encounters during Posyandu sessions, home visits, or other opportunities. The LCSP staff themselves have also provided numerous health education sessions during Posyandu days.3 The LCSP provided the following BCC materials: o 5,000 multicolored posters promoting bi-annual Vitamin A supplementation. o 5,000 multicolored posters promoting diarrhea prevention and re-hydration therapy o 2,000 multicolored posters promoting the use of ITNs o 50 T-Shirt promoting malaria treatment o 250 T-Shirt promoting ITN use o 250 T-Shirt promoting use of Oralit (ORS) o 500 T-Shirt promoting Posyandus o 1000 T-Shirt promoting Exclusive Breastfeeding (for cadres) o 200 T-Shirt promoting Vitamin A Capsule (for TBAs) o 1,000 stickers promoting immunization o 10,000 new MCH Handbooks for West Kalimantan Province, of which 3,000 were allocated to the Landak District4 o 5,000 Growth Monitoring Cards for Landak District5 3 These sessions are reported in the LCSP quarterly report with the dates, topics, number of attendees and health center and LCSP staff who delivered the talks. 4 The MCH handbook of the MOH includes various educational, in addition to data recording, components (see further discussion in Section I.B (5). 5 The Growth Charts are provided when the MCH Handbook is not available. World Vision 16 LCSP Midterm Evaluation Some BCC materials are written in local language. Following the FAR recommendation, the LCSP has adopted the following 11 among the 16 key family practices of Community IMCI identified in the WHO/UNICEF meeting in Durban in June 2000. 1. Breastfeed infants exclusively for at least 4 months. 2. Starting about six months of age, feed children freshly prepared energy and nutrient￾rich complementary foods, while continuing to breastfeed up to two years or longer. 3. Ensure that children receive adequate amounts of micronutrients (Vitamin A and Iron, in particular), either in their diet or trough supplementation. 4. Dispose of feces, including children’s feces, safely; and wash hands after defecation, before preparing meals, and before feeding children. 5. Take children as scheduled to complete a full course of immunization (BCG, DPT, Polio, and Measles) before their first birthday. 6. Protect children in malaria-endemic areas, by ensuring that they sleep under insecticide-treated bed nets. 7. Continue to feed and offer more fluids, including breast milk, to children when they are sick. 8. Give sick children appropriate home treatment for infections. 9. Recognize when sick children need treatment outside the home and seek care from appropriate providers. 10. Follow the health worker’s advice about treatment, follow-up and referral. 11. Ensure that every pregnant woman has adequate antenatal care. This includes having at least four antenatal visits with an appropriate health care provider and receiving the recommended doses of the tetanus toxoid vaccination. The mother also needs support from her family and community in seeking care at the time of delivery and during the postpartum and lactation period. The family practices above adequately correspond to the interventions of the DIP and FAR, leaving out the 5 other practices related to mental and social health, HIV/AIDS, injuries, child abuse and the involvement of men in child health. Challenges The LCSP has not carried out any formative assessment and developed a detailed BCC strategy and related action plan. Some of the BCC materials developed and produced by LCSP may not be easily understood by the target audiences and therefore may have limited impact on knowledge and behavior. The materials available for ARI in particular was found inadequate. Many BCC activities proposed in the DIP such as education, puppet shows, songs, and skits/drama, have not been implemented. World Vision 17 LCSP Midterm Evaluation The MCH booklet is not fully utilized as a health education tool in addition to a record of health data for mothers and children. LCSP does not fully use the resources and communication channels provided by the other ADP activities. Recommendations WVIDN and WVUS should assist the ADP staff, including that of the LCSP, in the development a detailed health behavior change communication strategy and action plan with local partners and communities. This strategy should be based on formative assessment to identify the specific behaviors to be changed among the various community members, the messages most likely to be understood and accepted, the most effective media and agent of change for each audience (including non-visual methods such as puppet shows, songs, skits, local radio, etc). This assessment should encompass all the behaviors included in IMCI, and conducted to benefit the entire Landak District. The LCSP may choose to focus first on the set of community and individual behaviors related to malaria, including the use of ITNs, individual malaria prevention measures and treatment, and collective and environmental measures to decrease transmission. The LCSP and the DOH should train the cadres on the use of the MCH booklet for delivering health messages during home visit. The LCSP should pilot test the introduction micro-credit with health education within the Pancur Kasih Foundation. The LCSP should consider the use of a mobile videocassette players/monitors/generators for screening health education messages at community and Posyandu levels. (3) Capacity building The LCSP aims at enabling communities, the local health care delivery system and NGO partners to provide CS interventions. In direct impact areas, the project seeks to:  Revitalize Posyandus and their cadres  Train self-help groups (KSM) to deliver CS health education6  Build capacity of informal health partners to deliver CS interventions (traditional healers, shopkeepers, traditional birth attendants)  Assist health center in supervision and use of data for targeting and decision-making. 6 This corresponds to the “Micro Enterprise Development” intervention listed in the Estimated Program Effort And USAID Funding By Intervention table of the DIP. World Vision 18 LCSP Midterm Evaluation  Train all staff in health center, sub center, and village post staff for CS interventions  Build capacity of local NGO, including preparation and use of activity plans and operations in accordance with established MOU.  Meet regularly with District Health Office and ADP for planning and priority setting.  Meet regularly with Provincial Health Office for project updates and for provision of facilitators for training events.  Build capacity of ADP staff in CS interventions and related skills such as behavior change communication, quality improvement, data for decision-making, credit with education, etc. Achievements The LCSP has made progress in building the capacity of its various partner organizations: community-based organizations such as the Posyandus and KSM groups; the Pancur Kasih Foundation; the DHO and HC in the project area; and World Vision. The number of functional Posyandus in the project area increased from 76 in July 2000 to 96 in September 2001 and to 100 in July 2002. As result, 80 % of the population in the 2 project sub districts now has access to a functional Posyandu. The creation and support of micro credit groups (KSM) are one of the ADP community development strategies on which the LCSP builds upon. 7 There are 32 KSM groups in the 4 sub districts of the ADP Pontianak. The LCSP trained a total of 106 members. In November and October 2001, the ADP Pontianak and LCSP introduced health-related activities in KSM groups, and several of them (in Saham, for example) now have a health subdivision in their organizational structure. In May 2002, the LCSP project signed MOUs with 32 KSM groups for the distribution of the ITNs to shopkeepers. In FY 2003, the LCSP plans to create links between Posyandu and KSM groups to increase KSM groups’ involvement in health activities. The LCSP involved 24 and 30 community members, including cadres and teachers, as data collectors and supervisors to conduct the baseline and midterm KPC surveys. This constitutes a valuable pool of experienced surveyors that can be used in future health surveys. Regular meetings and contacts between the project’s Technical Team Leader and the Provincial and District Health Officers have been useful in sharing information and experience and in strengthening the involvement on the MOH in the project activities. In FY 2002, WVIDN conducted a Self Review of its 1998-2001 programs during which the ongoing humanitarian and economic crises led to drastic increases in budget, scope and human resources. A Task Force of 17 staff from various departments collected data through document review, and focused group discussion and interviews with staff from WVIDN and various partner organizations at the national and project levels. The LCSP staff participated in this data collection effort. The Task Force recommended that WVIDN organize its programs around 4 7 By end of FY 2001, WVIDN had established 28 ADPs in six provinces and had more than 61,000 children in its sponsorhip programs. Priority was given to the development of self-help groups to ensure community participatory process, ownership and sustainability. World Vision 19 LCSP Midterm Evaluation regional teams combining integrated development and technical staff. The LCSP adapted its organizational structure to better correspond to this integration of child health and survival activities in general development programs. The LCSP team leader and staff now report to the ADP Pontianak Manager. The LCSP provided multiple professional development opportunities to its staff. These are briefly discussed in section I.B (2). Challenges The capacity building indicators developed during the FAR in response to the DIP review recommendation have not been operationalized and they still have no benchmarks (See Appendix L – E.). The plan in the DIP and the FAR that each partner organization would conduct a baseline Organization Self-Assessment and other baseline assessments has not been implemented. This is may result from a lack of technical input in this particular area. The tools for these assessments are now available at the level of WVUS, and will shortly be used in the field. The Pancur Kasih Foundation is the only NGO in the project area with which the LCSP can partner to achieve project objectives. This organization does not have health-related expertise. Philosophical and operational differences between the LCSP and some of the community-based partner organizations have resulted in a lack of interest in the training offered by LCSP because it does not appear related to their work. The LCSP staff report difficulties to work with health centers staff. In October 2001, the project assessed of service in the Posyandu by using: (1) Checklist for observation for the health worker performing health check up and immunization in the Posyandu; and (2) Exit interview to mothers with under-five children and pregnant women who attended the Posyandu day (see Quarterly report Oct.-Dec. 2001). This assessment pointed out to various gaps in the quality of care and resulted in useful actions at the level of the LCSP and the DHO. Although definite progress has been observed since then, Posyandus still face problems such lack of health staff to provide immunization and other services, lack of cadres to organize and record the services, and occasional lack of specific vaccines or drugs. Recommendations WVUS and WVIDN should assist the LCSP in clarifying its capacity building objectives and indicators, in conducting the baseline assessments, and in implementing the relevant activities. This plan should address the DHO and HC in the project areas, Pancur Kasih, the KSM groups, and the Posyandus. World Vision 20 LCSP Midterm Evaluation The LCSP should establish new Posyandus to cover the remaining 46 uncovered dusuns, and develop indicators and instruments to track the quality of services provided in the Posyandus. The LCSP should strengthen to capacity of KSM groups to engage in other community health activities than ITN distribution. Such activities are for instance building fish ponds, gardening of vitamin A rich vegetables, establishment of Village Drug Posts, and provision of health education to their members. The LCSP should introduce the “Credit with Education” approach within the Pancur Kasih Foundation The LCSP should formalize, document, and include various project staff in the otherwise already established regular meetings and contacts that the TTL has with the PHO/DHO staff. This should be begin in FY 2003 and would be essential during the scale up phase of the remaining of the project. WVUS, WVIDN and the LCSP should begin planning for the a lessons learned/promising practices workshops to take place at the end of FY 2003 and provide the basis for scaling up at the District level. (4) Health workers performance improvement Achievements In April 2001, the LCSP in collaboration with the PHO and the DHO organized two simultaneous Training of Trainers seminars in Mandor and Sengah Temila. Most of the HC staff (37) in the project area, and several Pontianak ADP motivators (12) and LCSP staff (14) participated in this training that focused on the 5 project components: Malaria, Immunization, ARI, Diarrhea diseases, and Vitamin A. At the time of the MTE, the LCSP and HC staff had already trained about a thousand of community members involved in health related activities. The dates, areas, and number of participants of the various training events jointly facilitated by the LCSP and HC staff for community members are listed in Appendix M. The total numbers of trainees by project area and type of participants are summarized below. Target Mandor Senakin Pahauman Total Posyandu Cadres 148 153 223 524 POD Cadres 22 0 0 22 TBA 85 26 74 185 Shopkeepers 91 93 8 192 Total 346 272 305 923 From June 2001 to June 2002, the LCSP and the HC staff were able to train 524 cadres in 100 Posyandus in the three project areas (229 in 2001) on management of child survival interventions, nutrition counseling, and data recording. At the time of the MTE, all 100 World Vision 21 LCSP Midterm Evaluation Posyandus had several trained cadres to conduct community health activities. The MTE team found that in general, Posyandu cadres have adequate knowledge in diarrhea management and in vitamin A supplementation (except in postpartum), but very low level of knowledge about EPI and ARI. The LCSP has conducted refresher training for 45 Posyandu cadres in May 2002. In January 2002, the LCSP enlisted a total of 90 TBAs from the three project areas in the perspective of offering them training and support. From February to July 2002, the LCSP and HC staff trained a total of 185 TBAs, most of these in Mandor and Pahauman. At the time of the MTE, 133 of these TBAs had already participated in a second training session, and a smaller number had participated in a third or fourth session. The LCSP staff considers that there are still more TBAs who will join this training program. The LCSP has contacted the USAID mission and the Maternal and Neonatal Health project to ensure coordination of its TBA program with other similar efforts in the country. The LCSP and HC staff trained a total of 192 shopkeepers in the project areas. Most of these (173) were trained between January to April 2002 and in Mandor and Senukin. The LCSP and HC staff also trained 22 POD cadres in Mandor. According to the DOH policy, POD cadres are chosen among trained village shopkeepers so that they do not have to set up a special place for the POD. In addition to the Training of Trainers, the LCSP offered several training opportunities to the staff from the various health centers in the project area. In May 2001, the LCSP organized one training seminar on Malaria that was attended by 32 DHO health center and 10 LCSP staff. The training was provided by the National Malaria Coordinator8 of the MOH RI, assisted by the PHO, DHO, and Pontianak ADP/LCSP staff. In May and June 2002, the LCSP supported 6 midwifes and 1 District Health Office to participate in an IMCI training seminar organized by the PHO. The LCSP is planning to support IMCI training of a total of 30 health center staff by the end of September. In January 2001, the LCSP also support the Head of the CDC Sub section of Landak District Health Office to join training in microscopy in Jakarta. Challenges The results of the interviews of Posyandu cadres during the MTE, and the low attainment of some LCSP targets, suggest that their performance is not entirely satisfactory. The MTE Team also observed that TBAs appear torn between their old traditional practices and the new skills taught by the LCSP. During the interviews of TBAs, only one TBA reported excitement about applying new skills. Recommendations The LCSP should conduct regular assessments of the performance of the community members (Posyandu cadres, TBAs, POD cadres and shopkeepers) trained by the project during supervision visits, using explicit standards of performance. They should then develop and implement a 8 Dr. Ferdinand Laihad, who was also member of the MTE Team. World Vision 22 LCSP Midterm Evaluation performance improvement plan based on these findings and that gives more emphasis on on-the￾job training, coaching and mentoring rather than only initial or refresher training. The LCSP and HC staff should evaluate the current system of supervision of Posyandus to assess its effectiveness and modify it accordingly. The LCSP should empower the Posyandu cadres by providing them skills in analyzing their own data and use them for action. The LCSP should include the LCSP health motivators with clinical background in the MOH IMCI training so that they can fully support the introduction of IMCI in health center and Posyandus. (5) Sustainability The vision of sustainability of the LCSP is that at the end of the project:  Health center and post staff regularly supervises and encourages Posyandu cadres and other volunteers.  Health center staff and village-based volunteers utilize participatory education methods and visual health education materials to promote CS interventions.  Shopkeepers and traditional healers are able to state treatment protocols and danger signs indicating need for referral.  Increased coverage for all CS interventions demonstrates high community participation.  At least 70% of the villages use community based EPI and Pregnancy Registers and a Community Based Death and Disease Surveillance.  At least 60% of the self-help village groups spend profits on items or activities that contribute to improved household health and child survival.  ADP staff is trained in key CS interventions and ADP plans include at least 20% resources to be spent on health. Achievements The LCSP made substantial progress in promoting sustainability of its activities and expected results: the number of functional Posyandus has increased, and HC staff and Posyandu cadres promote CS; immunization coverage has increased; and 12 ADP motivators have been trained as trainers in the 5 project CS interventions. The LCSP is progressively integrating its activities into the ADP development programs. The organogram of the project was modified to ensure coordination and synergy between the LCSP and the other ADP activities, and to build the capacity of the ADP staff who will continue activities when the LCSP ends. Also, the ADP staff participated in all training events organized for LCSP staff, and in all significant LCSP planning and evaluation activities. World Vision 23 LCSP Midterm Evaluation The LCSP had begun discussions with the KSM about their involvement in some financial aspects of LCSP. Substantial progress has also been made in promoting sustainability within the MOH services. For instance, the PHO/DHO has already contributed to the replication and dissemination of health education materials such as the 10,000 MCH booklets. The DHO also purchased 1,000 ITNS and will distribute them through the KSM groups just like ITN provided directly by the LCSP. Challenges During the MTE, most key stakeholders expressed their desire for LCSP to continue activities until the communities are self-reliant. This indicates a concern for sustainability of the project activities. The availability of District government funds is limited and therefore the capacity of the DOH to provide direct support to LSCP activities such as the supervision and training of the Posyandu cadres is also limited. The number of Government health staff is insufficient. As an example, there are only 15 midwives for 31 Polindes in the project sub districts. The lack of good transportation in the project area is a constraint to the sustainability of essential activities such as outreach and supervision of the Posyandus. The relatively short duration of the project (3 years) will not allow for implementation of all activities, particularly those aiming at capacity building for sustainability. The groundwork for phasing out has not been developed by LCSP and its key stakeholders. Recommendations The LCSP should accelerate the development of a phase out strategy with the MOH, community and other key stakeholders. This exit strategy can be for FY2003 if the project ends as currently planned, then revised for FY2004-2006 if the recommendations to extend the project for two￾year is granted. The LCSP should define a few key sustainability objectives and develop the appropriate indicators. The DOH should allocate a specific budget for the support of the Posyandus. World Vision 24 LCSP Midterm Evaluation C. Program management (1) Planning, monitoring and evaluation Achievements The LCSP successfully engaged project partners and beneficiaries in the initial baseline assessments and planning activities. The DIP and the FAR reports, finalized and used mostly by WV senior technical and management staff, reflect this participatory process. The LCSP staff produces regular and complete progress reports that include various process monitoring data such as data on training, health education sessions, equipment and BCC materials distributed, and very informative problems and issues. These reports are themselves based on the monthly reports from the field staff, including those from each health motivator. The ADP Pontianak, including the LCSP, is progressively integrating project activities into other development programs, as evidenced by the adoption a new organogram (Appendix O), by the training provided to the ADP motivators and their inclusion in many planning and evaluation activities, and various others significant actions (See Appendix L – A). Challenges The LCSP staff has not systematically used detailed annual, quarterly, or even monthly work plans and budgets as a way to communicate technical aspects of the project and to assign clear implementation responsibilities at all levels, from WVIDN at the national level to the health motivators and other field staff members. There is no specific strategy and action plan for each project area. Even though the two project areas, Mandor and Sengah Temila, are similar in terms of health and government infrastructures different, and are both populated mainly by Dayaks, with only a few Malaynese in Mandor city, there are environmental, cultural, and administrative differences that make it worthwhile to develop and implement sub district or health center specific plans. The integration of the ADPs and the LCSP activities is still not fully realized yet. An example of a problem resulting from this situation is seen in Toho, who would like to receive ITNs from LCSP but is not eligible because it is not part of the specific geographic area of the project. Progress reports and action plans and budget at various levels are occasionally submitted late, creating delays in the administrative and financial management of the project. There is no tracking system for report submission. Also, findings, recommendations or requests from the field do not always receive immediate follow up. The ADP and the LCSP staff still operate with different approaches regarding development versus target-oriented projects. These differences appear primarily when working with the community-based organizations and volunteers, and when choosing priorities for action. World Vision 25 LCSP Midterm Evaluation Recommendations The ADP Pontianack, including the LCSP, should continue integration of LCSP activities as a key strategy for capacity building and sustainability. WVIDN and WVUS should take advantage of the LCSP to further develop the rational and strategies for integrating of child survival activities in the ADP Pontianack development programs. This should specifically include ways to achieve LCSP targets without sacrificing other development principles. WVIDN and WVUS should provide assistance to ADP Pontianack in implementing these strategies and drawing lessons for other ADPs in Indonesia and elsewhere. To that effect, WVIDN and WVUS should consider training health staff about development principles, and training development staff about health. Also, development and health staff from ADP Pontianack should meet quarterly to share lessons learned and best practices with this respect. The LCSP should consider developing area-specific strategies and quarterly action plans. These plans, coordinated by the LCSP Technical Team Leader, might adopt the same overall strategies and timeline but include all the activities and information needed by the persons in charge of their implementation such as the sub district or health center staff, the health motivators; etc. The ADP and LCSP should develop policies regarding implementing project activities in communities outside the project areas. (See the case of KSM group outside the project areas interested in distributing ITNs). The ADP and LCSP staff should ensure a timely submission of all progress and financial reports to ensure timely response and actions from the relevant technical and administrative officers. The time needed to prepare complete but concise reports should be provided to the staff responsible for their preparation. WVIDN and the LCSP should strengthen the use detailed technical work plans to coordinate, delegate and assume responsibility for project activities. (2) Human resources management Achievements The LCSP currently has a very motivated and competent technical and managerial staff. In addition to their respective professional skills, most also have computer and English skills that allow the team to effectively communicate with various partners including preparing technical and financial reports. All the LCSP staff shows high potential for professional development. Between October and February 2002, most of them participated at least once in various relevant training programs, in World Vision 26 LCSP Midterm Evaluation Indonesia and overseas. The specific training events attended by the LCSP staff is provided in Appendix N. Challenges The LCSP has had difficulty to fill the core team positions when the project was relocated in the Landak District. It is only in April 2001 that Technical Team Leader, the Training Coordinator, and seven health motivators were on board. Only the Monitoring & Evaluation Officer and the Finance Officer have been in position in Landak since the beginning of the LCSP. The LCSP has also experienced several instances of resignation of senior and mid-level staff that have affected the continuity of project activities. Five out the 8 health motivators initially recruited resigned. Also, the first Technical Team Leader and Training Coordinator, then the Monitoring and Evaluation Officers, resigned. The LCSP Program Officer based in Jakarta will take a leave of absence for the academic year 2002-2003 and still has to be replaced. The positions for a bookkeeper, for an administrative assistant, and for an additional motivator remain open. Among reasons for this difficulty to hire and retain suitable professionals are that most physicians and nurses tend to want to work for the Government or in clinical settings, and that the location of the project creates a sense of isolation for most staff meeting the current qualifications for the various positions. The WVI salary scale and benefits does not seem to be a major motivation for resignation. There may be some issues of quality of teamwork and job satisfaction at various levels of the LCSP staff that impacts the retention of qualified staff and their performance. Recommendations: As the integration of the child survival activities into the ADP progresses, the ADP Pontianack and LCSP should consider relying more on community development specialists than on health specialists for all relevant activities. WVIDN and WVUS should continue teambuilding efforts and emphasize leadership, teamwork, and professional development as opposed to a more administrative and directive style of management. WVIDN and WVUS should increase responsibility, ownership and accountability of the LCSP staff at all levels. World Vision 27 LCSP Midterm Evaluation (3) Financial management and logistics Achievements The LCSP has been implemented for two of its three years of duration with one third of its initial budget. The table below presents the USAID and WVI contributions to the total grant, and the respective field expenses as of March 1, 2002. USAID WVI Total Total Grant 732,388 240,700 973,088 Field Expenses 200,969 56,974 257,943 The LCSP has been able to procure and distribute cold chain equipment (3 motorbikes, 15 vaccine carriers, and 3 generators) in the three health centers, and a series of BCC material (see list in section I.B.2) to health facilities and health workers in the project area. Challenges The reasons for under spending project funds are not fully understood yet, but probably include the delay in recruitment of project staff and in procurement, and a slow implementation of activities. The ADP and LCSP staff capability in financial and logistic management is not adequate. As a result, there have been several instances of delays in the procurement and distribution of project related goods, poor communication of the problems and weak follow up actions. The monthly and three monthly financial plans are not produced on a timely and adequate manner, resulting in delays in the transfer of funds and eventually in the implementation of activities. The positions for a bookkeeper and an administrative assistant are vacant. Recommendations: Given the delayed start up, the relative under spending, and the opportunity for achieving sustainable results in the Landak District, the MTE recommends a 2-year no-cost extension. The LCSP staff in the field should make thoroughly study the activities that they plan to conduct and their costs before developing and submitting their action plan and budget. The LCSP should recruit a bookkeeper as soon as possible. World Vision 28 LCSP Midterm Evaluation The LCSP should review the job description of the Administrative Officer to include logistics in the roles and responsibilities and direct reporting to the Financial Officer. (4) Information management Achievements The LCSP conducted a baseline and a midterm KPC surveys in September 2000 and July 2002; both are very well done and documented. The LCSP also conducted several studies to investigate specific project implementation and management issues:  Two Health Facility Assessments during the baseline survey in September 2000 and during the PLA assessment in December 2000.  PLA assessments in 15 sub-villages, in December 2000. These assessments were facilitated by 8 candidates for the LCSP health motivator positions after a 3-day PLA training seminar.  An assessment of the quality of EPI and other services in Posyandus using observation checklists and exit interviews (see quarterly report October – December 2001). The LCSP staff has produced good and complete progress reports covering the entire project implementation period and in which information on key process, outputs and special events can easily be found. The LCSP project adapted (malaria education information was added) the MOH RI MCH Handbook and printed 10,000 copies for the West Kalimantan Province, including 5,000 copies for the Landak District. This 50-page, pictorial, data recording and educational booklet ensures continuity of care from the pregnancy to the fifth birthday of the child. When and where the MCH booklet is not available, other forms such as the Growth Monitoring Chart are still used. The DHO and health centers staff approved the use of the Community EPI and Pregnancy Tracking Forms by Posyandu cadres, TBAs, and Polindes. These forms were slightly revised from the original models developed by WV Cambodia to include one column for iron/folate and one for vitamin A supplementation in the pregnancy form. The LCSP introduced these forms through on-the-job training of their users in May 2002. Before, Polindes were using GOI forms and cadres were using a register. The LCSP recently proposed revisions to the Monthly Posyandu Report, which is the GOI form used at the Posyandu level to record local data and activities, and to report to the health centers. These forms are filled in by the health center staff or by the cadres when they can do it. The LCSP also recently tested a scoring system to determine which Posyandus are functional. Preliminary analyses identified some problems and a revised form will soon be tested in the field. World Vision 29 LCSP Midterm Evaluation Challenges There is no analysis of the KPC survey data beyond frequencies. For instance, calculation of confidence intervals and simple cross-tabulations are missing. Also, there is no critical narrative interpretation of the findings including comparisons between baseline and midterm results. The midterm Key Informant Interview also lacks data analysis and interpretation. The LCSP does not use DHO statistics collected from health centers and Posyandus such as the attendance to maternal and child health care services, the number of immunizations provided, the number of vitamin A capsule distributed, and others. Some Posyandu cadres do not know the purpose and benefits of the data they collect. Others cannot fill the forms and registers used in the Posyandus. The MCH booklets have only been introduced in the project area in November 2001 (they are sold between Rp. 3500–5000 if the mother can afford) and are not widely distributed yet. As the community EPI and pregnancy tracking system just began, there is no data available yet and it is not known whether it will be feasible, improve tracking of defaulters, decrease dropout rates, or increase the coverage of children completely immunized. The proposed Community-Based Death and Disease Surveillance system, but its implementation has not started yet. The DIP review and the FAR recommended developing and adopting a series of process indicators but this is not done yet. Problems with delays in the transmission of financial reports are discussed in section I.C (5) in Financial Management. Recommendations The LCSP staff should adopt a limited set of key process indicators, as recommended during the DIP review and FAR, and include them in the quarterly or annual reports, as appropriate. Some simple indicators are the number of functional Posyandus; the number of Posyandu days as compared to those expected; the attendance at these Posyandus days; the number of supervision visits made, the number of trainees by type and topics as compared to those planned, etc. The LCSP should assist the DHO in the management, analysis and interpretation of the data currently collected by the Posyandu cadres (Monthly Posyandu Report) and other health services. Such data provide a direct measure of the activities of the health services that the LCSP intends to improve. The LCSP should develop a guideline for the supervision of Posyandus. The observation checklist and exit interview developed in October 2001 may provide the basis for a set of tools to World Vision 30 LCSP Midterm Evaluation be used during such supervision.9 Systematically collected and analyzed, supervision data may provide data for indicators of the quality of those services. The LCSP should begin and monitor the implementation of the various community-based information systems that it proposed: the CBDDS, the EPI and the Pregnancy Tracking Forms, the revised Monthly Posyandu Report. A first step would be to write down a description of the various tools, the data to be collected, the links between tools, the indicators that can be constructed, and the use of the information generated by these forms. The LCSP should assess the current community-based information system (its timeliness, completeness, validity, cost and usefulness) and build on this experience when introducing the new systems and tools. The simplest and the smallest number of indicators will probably have the highest chance of success and sustainability. The LCSP should write a protocol to assess the feasibility and usefulness of the systems that it introduces by the end of the project. The LCSP should only introduce the CBDDS if the project is extended, because it will take more than a year to assess its feasibility and usefulness. The LCSP should continue facilitating the distribution of the new MCH booklets to pregnant women The LCSP should train cadres, TBAs, and health staff in the use of data that they collect. The health information system and indicators that they use should be clearly defined and documented in order to provide clear instructions and skills during this type of training. The LCSP should carefully check the consistency between the questions in the baseline and midterm KPC and the formulation of the project objectives before finalizing the final evaluation instruments. One of the goals in conducting of a series of survey is to compare the values of the same indicators over time. The sample size and the related sampling error must be taken into account when making such comparisons. (5) Technical and administrative support There has been frequent and constructive visits from the WVIDN National Office (the National Health Advisor and the LCSP Project Officer), the Asia Pacific Regional Office (APRO) Health Advisor, and WVUS to the LCSP site. Some of these visits are opportunities to participate to key project activities such the DIP workshop, the First Annual Review, or the Midterm Evaluation. Others are to work directly with and assisting the project staff and its partners. This intense support is important to maintain strong links between the rather isolated field site and the national and US offices and staff. There seems to be, however, missed opportunities for focused technical assistance on specific aspects of the project that could be provided by internal WV staff or external consultants. This is particular important for a project that set out to be a “demonstration site and multiplier to scale up.” 9 This study was mentioned in the section on capacity building. World Vision 31 LCSP Midterm Evaluation The First Annual Review conducted less than a year before the MTE involved a similar large group of external individuals and a similar, largely participatory, methodology. Many issues identified during the FAR and the related recommendations were still valid at the time of the MTE, and therefore just resurfaced during the various group discussions and interviews of the MTE. On the other hand, the LCSP management team did not get the opportunity to conduct its own in-depth analysis and documentation of the status of project implementation and to reassess the validity and feasibility of the DIP after one year of project implementation. As valuable as having a review by external stakeholders and peers might be at some point during the implementation of a CSP project, it does not preclude the need for an internal annual report. The USAID guidelines for the First Annual Report, which does not specify the methodology to be used, seems to emphasize the internal and accountability purpose of the required First Annual Report. Recommendation WV senior technical staff and the LCSP staff in Pinyuh/Landack should plan in advance the backstopping visits and negotiate at least some of the specific aspects of the project that need to be addressed. This negotiation can be done at least partially through the development and approval of detailed work plans (annual, quarterly, and monthly) and scopes of work. WVUS should emphasize the internal analysis and accountability aspect of the required First Annual Report, and assign the primary responsibility for this exercise to the CSP management team and its local partners. The time and resources needed should be made available to the entire CSP team and partners, and WVUS should provide its assistance if appropriate. World Vision 32 LCSP Midterm Evaluation D. Conclusions and recommendations The LCSP implementation began one year later than expected because of the social unrest and insecurity that took place in April 2000 in the initial project area in Poso, Central Suwalesi Province. Upon approval by USAID, WVIDN relocated the project in the Landak District in July 2000, and conducted the baseline assessment and the preparation of the DIP in September 2000. Most field activities began when the entire project team was on board in April 2001. During the first year, the LCSP began reactivating and creating Posyandus and training their cadres, and provided logistical support to the health centers. The level of activity increased in the second year, with the training program now including TBAs and shopkeepers in addition to the cadres in the reactivated or new Posyandus. Technical approaches The KPC surveys in September 2001 and July 2002 show an increase in the coverage of several interventions: immunization of infant and pregnant women according to the EPI schedule, vitamin A supplementation of postpartum women, use of ORS among children with diarrhea, and early treatment of children with fever and malaria. The two surveys also show an increase in knowledge of mothers of children under 2 years of age with respect to the signs of pneumonia in children. These findings suggest that the activities of the LCSP have been effective. However, the project objective will only be achieved if activities are continued and strengthened. In particular, the vitamin A supplementation of postpartum mothers, the immunization of infant and pregnant women, the use of ORS, the dietary management of diarrhea, health care seeking for children with fever and malaria or with signs of pneumonia, and the use of ITN by children are still well below the objectives. The MTE identified several constraints with respect to these interventions. There are still problems in the quality of the services in the Posyandus such as occasional shortages of vitamin A capsules; weaknesses in the cold chain and occasional vaccine shortages; missed opportunities for tetanus toxoid immunization; and insufficient supply of ORS. Also, it is uncertain yet which distribution mechanism for ITNs will be the most effective in increasing their use by children under five. Finally, the MTE acknowledged the high prevalence rate of diarrhea among children under the age of two and the high malaria transmission in illegal gold mining areas. Crosscutting approaches The LCSP made significant progress in mobilizing and building the capacity of its partners to sustain the project current and future achievements. The LCSP involved partners and community members in project activities such as needs assessment and initial project planning, distribution of ITNs, and meetings with project staff. The LCSP has begun training KSM members in CS activities and involving them in the distribution of ITNs. The LCSP staff also World Vision 33 LCSP Midterm Evaluation work more and more closely with the ADP motivators to reach community members through their general development activities. However, the MTE Team found that much needed to be done to ensure a full participation in and ownership of project activities by the community members and organizations. The number of functional Posyandus increased from 76 in July 2000 to 100 in July 2002, and now 80% of the population in the project area has access to a functional Posyandu. One thousand community members (192 shopkeepers, 185 TBAs, 524 Cadres, 106 KSM members) involved in health activities have been trained in specific CS interventions and related skills. The DHO and HC staff has been trained in microscopy (1 laboratory technician), in training skills for child survival, in malaria control (47 DHO and HC staff), and most of the LCSP and health centers staff will have been trained in IMCI by the end of September 2002. In addition, the LCSP provide logistical support to the health centers for their facility-based and for their outreach activities, including supervision. These activities have improved and will continue to improve the quality of care in the MOH services in health facilities and in the Posyandus. The LCSP has achieved positive changes in specific behaviors primarily through training, equipping with some BCC materials, and encouraging Posyandu cadres to educate their communities. The MTE Team found that the type and use of BCC materials and methods may not always be the most effective for the communities in the project area, and there was no adequate material for ARI. Program management The LCSP staff and activities are progressively and successfully integrated into the ADP Pontianack management structure. This is in accordance with one the goal of WV to integrate health, and child survival in particular, into its general development programs. The LCSP staff has otherwise maintained good project-related management and financial practices, except for some solvable issues of delays in transmission of progress and financial reports. The LCSP now has motivated and competent technical and managerial staff on board, but has experienced difficulties in recruiting and maintaining this staff. All the current LCSP staff shows high potential for professional development, and most of them had opportunities to participate in relevant training events through the LCSP. The LCSP conducted a series of assessment and studies including two KPC surveys, two health facility assessments, and PLA assessments in 15 communities. These studies were well conducted and documented, but may not include full analyses and interpretation of the data. The LCSP began introducing several revised and new community-based health information systems and tools. Although they are all relevant and promising, pilot testing and achieving sustainable implementation of these systems will require time and careful follow-up. WVIDN, WVAPRO and WVUS provided continued and constructive technical and administrative support to the LCSP, through frequent visits to the project area and through other backstopping activities in their respective offices. This valuable support may however have been World Vision 34 LCSP Midterm Evaluation short of the focused technical assistance required for this new project set out to be a demonstration site for later scale up at the Landack District level. The LCSP project has been implemented for two of its three years of duration with one third of its initial budget, probably because of the delay in recruitment of project staff and in procurement, and the slow implementation of activities. RECOMMENDATIONS The specific recommendations made throughout the MTE report are summarized below. They are addressed to the LCSP and partners in Landak, and to World Vision Indonesia, APRO, and US. Recommendations to the LCSP and its partners  Conduct joint visioning exercises at the village level to increase the support from community leaders to the Posyandus and other project activities, and to strengthen the integration of health into other community development activities.  Continue increasing the number of functional Posyandus, and improve the quality of their services using supervision guidelines to assess the performance of the cadres.  Assist the DHO and HC in the management, analysis, and use of the data collected by the Posyandu cadres and health centers staff.  Assist the DHO in the development, adoption, and implementation of policies regarding ORS, vitamin A and vaccine supply.  Support the introduction and implementation of the three components of IMCI (health systems, health facilities, and community) in the Landack District, and particularly the training and use of related behavior change communication materials at the Posyandu level.  Develop a phase-out plan for the last year of the project implementation. Recommendations to WVIDN, WVAPRO and WVUS  Continue offering professional growth opportunities to the LCSP staff, clarifying their job responsibility and accountability, and ensure their satisfaction and performance.  Provide focused technical backstopping and assistance in the following areas: o Definition of explicit capacity building activities for the DHO, health centers, Pancur Kasih Foundation, KSM groups and Posyandus o Development of a behavior change strategy and action plan, based on formative research as appropriate. o Development of a performance improvement plan for community members such as Posyandu cadres o Development of detailed protocols for studies such as in-vivo chloroquine resistance; epidemiology of malaria; and field-testing of new health information systems and forms. o Introduction of credit with education methods within the Pancur Kasih Foundation. World Vision 35 LCSP Midterm Evaluation This focus technical assistance should be documented in short technical memorandum easily attached to quarterly reports for easy communication with the various officers and partners involved in the LCSP implementation.  Assist the LCSP team in the development and adoption of a definite table of project monitoring indicators with clear and realistic targets, baseline data or date at which it will be available, source of data, frequency of reporting, and persons in charge of collecting and reporting the data. These indicators can only be defined and adopted for well-defined strategies and activities. This table should be included in the quarterly and annual reports.  Begin planning for scaling up project activities at the Landak District level. The scale-up plan should be based on detailed data from the 8 other sub districts of the Landak District, and operational data from the 3-year implementation of the LCSP in Mandor and Sengah Temila. One step in this planning process should be the organization of a lessons learned workshop with all the LCSP partners around mid-September 2003.  Request a 2-year no cost extension of the LCSP, from October 2003 to September 2005. This extension is necessary to scale up the activities at the level of the Landak DHO and to achieve the objectives of the project in terms of capacity building and sustainability. World Vision 36 LCSP Midterm Evaluation E. Results highlights The MTE team decided that it was too early in the project implementation to capture the highlights of the LCSP and communicate them to a large audience. One aspect of the project that might provide a very good stand-alone document for World Vision is the Integration of Child Survival in Area Development Programs, and the lessons learned for the broader theme of integrating health in development. This document could begin with a summary of the ADP model developed by World Vision (see DIP, Appendix 6; cite # in IDN and elsewhere), and of the self-help groups in particular. It could describe the management structure adopted by Pontianack ADP to successfully integrate a separate project such as the LCSP and the specific activities conducted to integrate health into other community development programs. Activities with the self-help groups are of particular interest: Training members in CS skills, including BCC, data for decision-making, etc. Role, achievements and limitations in the distribution of ITNs Links with Posyandus Organizational structure adopted by several self-help groups (Saham, for example) who now have a health subdivision Others activities can be described. There are several potential advantages of this strategy (longer-term approach; synergy between the various development programs; sustainability; etc) that could probably be demonstrated by the experience of the LCSP project. Results and lessons learned from this experience could benefit other ADPs in Indonesia and elsewhere, but also other PVOs and development organizations. World Vision 37 LCSP Midterm Evaluation II. Action Plan MTE RECOMMENDATIONS ACTION PLAN Recommendations Responsibility Target Date World Vision 38 LCSP Midterm Evaluation ATTACHMENTS World Vision 39 LCSP Midterm Evaluation A. Baseline Information from the DIP MTE Comments: Most of this section A is copied from the DIP and only slightly edited. A few “MTE comments” are inserted to highlight major changes in the project implementation that occurred after this document was finalized. The outline of this section follows. 1. FIELD PROGRAM SUMMARY 2. PROGRAM GOALS AND OBJECTIVES 3. PROGRAM SITE INFORMATION 4. PROGRAM DESIGN 5. PARTNERSHIPS 6. HEALTH INFORMATION SYSTEM 1. FIELD PROGRAM SUMMARY Estimated Program Effort And USAID Funding By Intervention: Intervention % of Total Effort (a) USAID Funds in $ (b) Immunization 30% $219,716 Vitamin A 10% $65,000 Control of Diarrheal Disease 20% $135,000 Control of Malaria 35% $276,672 Micro enterprise development (MED) 05% $36,000 Total 100% $732,388 Program Site Population: Children and Women (c): Population Age Group Number in Age Group Infants (0-11 months) 10,133 12-23 Month Old Children 12,786 24-59 Month Old Children 25,520 Total 0-59 Month Olds 48,439 Women (15-49 years) 67,822 Estimated annual number of live births in the site: 10,675. Sources of the population estimates above: Government of Indonesia, Health Center Statistics. World Vision 40 LCSP Midterm Evaluation MTE Comments: The Program Site Population estimates for Children and Women in Section 1. A of the DIP (table above) are consistent with the total population estimates given for the entire Landak District (313,000), not with those given for the two sub districts (97,616) in which the LCSP focuses its activities (direct impact area). See Attachment B for the population estimates currently used by the LCSP staff. World Vision 41 LCSP Midterm Evaluation 2. PROGRAM GOALS AND OBJECTIVES Goal or Strategic Objective: The Landak District Child Survival Project’s (LCSP) strategic objective is to assist the Landak District Health Office to reduce mortality and morbidity and nurture growth and development (thriving) among infants, children and mothers through an integrated child survival and rural enterprise project in the Mandor and Sengah Temila sub-districts of Landak district, West Kalimantan Province, over three years. Table 5. Intermediate Results and Indicators of the LCSP Intermediate Results Indicators / Means of Verification 1. Increased use of integrated child and maternal health services by the target population 2. Increased participation and contribution of communities for the prevention and early / complete management of diseases EPI  80% of children 12 to 23 months completely immunized verified by card. (Baseline 23%). Verification: KPC Survey, MOH statistics, Road to Health Card  50% of mothers with children less than 2 years of age received TT2 before the birth of their youngest child (Baseline - 97% of mothers did not have a TT card during the survey). Verification: KPC Survey, MOH data/ statistics, Maternal/TT card  100% of health facilities have cold chain temperature recorded in the expected range and vaccines within expiration date. (Baseline 20%) Verification: Health Facility Assessment, Temperature Recording Forms 1. & 2. As above VITAMIN A  90% of children 12 to 23 months received Vitamin A in the past 6 months. (Baseline 54%) Verification: KPC Survey, Road to Health Card, MOH data/Statistics  50% of mothers with children less than 2 years received Vitamin A within one month of their last delivery. (Baseline estimate 12%) Verification: KPC Survey, MOH Statistics, Maternal/TT card 1. & 2. As above MALARIA  75% of mothers with a child less than 2 years of age who was ill with fever during the past 2 weeks seek treatment for their child (ideally within 24 to 48 hours). (Baseline estimate 26%) Verification: KPC Survey, Household and clinic interviews with mothers whose children are suffering from fever (or have suffered with fever in the past 2 weeks).  50% of households utilizing mosquito nets that have been dipped in the past 12 months (Baseline 0%) Verification: Bednet club records, Interviews with households utilizing Bednets. World Vision 42 LCSP Midterm Evaluation 1. & 2.as above DIARRHEA  75% of children less than 2 years of age who had diarrhea in the past two weeks received oral rehydration therapy. (Baseline 43%) Verification: KPC Survey, Household and Clinic interviews with mothers whose children have diarrhea (or within the past 2 weeks).  75% of children less than 2 years of age who had diarrhea in the past two weeks received the same / amount or more of fluids or breast milk. (Baseline 63%) Verification: KPC Survey, Household and Clinic interviews with mothers whose children have diarrhea (or within the past 2 weeks).  70% of children less than 2 years of age who had diarrhea in the past two weeks received the same / amount or more of food. (Baseline 34%) Verification: KPC Survey, Household and Clinic interviews with mothers whose children have diarrhea (or within the past 2 weeks).  Less than 10% of mothers with child less than 2 years who had diarrhea in the past two weeks report their child received an antidiarrheal medicine. (Baseline 40%) Verification: KPC Survey, Household and Clinic interviews with mothers whose children have diarrhea (or within the past 2 weeks). 3. Equip communities to invest their limited resources in low cost high impact CS interventions and to strengthen their household livelihood. Micro Enterprise Development  20 Self-Help Village Groups trained to deliver CS health education and conducting monthly community CS health education sessions. (Baseline 0) Verification: Village visits to observe self help village groups education; Interviews with community members; Supervision/Activity records of Landak CSP. World Vision 43 LCSP Midterm Evaluation CAPACITY BUILDING 1. Local health care delivery system, NGO partners, and communities equipped and providing CS interventions at all levels (home, community, health services)  Provincial Health Office meeting on bimonthly basis for updates of project process and provincial office providing facilitators for key training events. Verification: Meeting records, interviews with MOH and project staff.  District Health Management Team supported to further strengthen their newly formed district. ADP and Project Manager conduct monthly meeting for planning and priority setting for upcoming activities as well as review of accomplishments to date. Verification: Meeting records, interviews with MOH and project staff.  Strengthening of health center management teams in direct impact areas for routine supervision and use of data for targeting and decision-making. Health center teams have map design demonstrating catchment area including presence of village health volunteers and TBAs and a plan for CS intervention coverage and supervision for their area. Verification: Review of map and plan for service coverage. Supervision records. Review of health information system records and results.  Capacity building of all levels of staff including: health center, sub center, and village post staff for CS interventions. Verification: Training records, interviews with training participants.  Support provided to revitalize the Integrated Service Posts, village health posts and the functioning of the village health volunteers. Verification: # of service posts and health posts, and village health volunteers functioning in direct impact area.  Capacity building of ADP staff in CS interventions and related skills (behavior change communication, quality improvement, data for decision making, credit with education, etc) Verification: Training Records, Interviews with project staff to evaluate CS understanding and its application to ongoing development activities.  Capacity Building/ Organizational Development of Local NGO, including activity plan and operations in accordance with established MOU. Verification: Training Records, review of activity plans.  Capacity building of informal health partners for CS initiatives including Traditional healers, shop keepers, self help village groups, community leaders, households, etc. Verification: Training records, interviews with participants on treatment protocols and key household health practices. World Vision 44 LCSP Midterm Evaluation SUSTAINABILITY 1. District and Provincial Health Offices prioritizing the delivery of CS interventions and engaging with community volunteers for creating an enabling environment for improved maternal and child health. 2. Communities engaging in prevention and early and complete management of common illnesses.  Health center and post staff regularly supervising and encouraging village based volunteers. Verification: Interviews with volunteers, field site visits, records.  Health center staff and village -based volunteers utilizing participatory education methodologies and visual health education materials to promote CS interventions. Shop keepers and traditional healers able to state treatment protocols and danger signs indicating need for referral. Verification: Field and clinic site visits to observe education activities.  Increased coverage for all CS interventions demonstrating high community participation. Verification: KPC survey report and MOH monthly data trends.  At least 70% of the village will establish community based EPI/Pregnancy Register and CBDDS (Community Based Death and Disease Surveillance) Verification: Field site visits to observe register and learn from volunteers about tangible changes in health conditions.  At least 60% of the self-help village groups will spend profits on items/activities that contribute to improved household health and child survival. Groups will be engaged in the promotion of CS messages at the village level. Verification: Field site visit to observe education activities. Interview with self help group members regarding spending of profits  ADP staff trained in key CS interventions and ADP design plans include at least 20 percent of resources being spent on health. Verification: Review of ADP design for core health components. 3. PROGRAM SITE INFORMATION Project Location/Demographics The direct impact area of the Landak District CSP is the two sub districts of Mandor and Sengah Temila, totaling a population of 97,616 in 31 villages. The project will indirectly impact the other 8 sub districts of Landak through health worker training and logistical support bringing the total beneficiary number to 314,917. Landak is a newly established district of West Kalimantan Province making the timing of the CSP strategic and the linkage and reference to the importance of birth, growth and thriving relevant. Landak is located in the center of the 19 districts of West Kalimantan Province, and is the only district bordered by the neighboring country of east Malaysia. The district's capitol World Vision 45 LCSP Midterm Evaluation is Ngabang, 178km or a 5-hour drive from Pontianak, the Province's capitol. To reach Pontianak, one takes a two-hour domestic jet flight from Cengkareng airport in Jakarta to Pontianak. The project office will be based in Sungai Pinyuh (where WV has its ADP office), located within Landak district, 130km or a 3-hour drive from Pontianak. The mountainous winding road to Landak is asphalt paved but the routes leading to the numerous villages within the district territory are either dirt or pebble stone and many passages include river/ stream crossings. The Landak District CSP directly targets the two sub-districts of Mandor and Sengah Temali. Mandor sub-district covers the area of 455 km2, consisting of 24,073 inhabitants and 17 villages and a population density of 45 persons/km2. The Sengah Temali sub￾district is divided into two large areas, Pahuman and Senakin and covers a vast area of 2,848 km2, with a population of 71,442 residing in 14 villages (96 sub-villages). The population growth in Landak is 2.49 (Province is 3.5, National is 1.9) with a population density of 32 persons/km2 (Province 26 persons/km2). The economy is of the project area is very poor with 65% of the population living below the Minimum Physical Need, the poverty level as defined by the Government of Indonesia. The economy is largely agrarian-based, with the staple crop of rice, and the main source of income from rubber plantations (82%). Small mining of gold and mercury is ongoing in the area. The individual rubber ownership is small, combined with an unstable rubber rate, poor access to markets and very little income opportunities outside of rubber, contribute to the poverty of households. Starting October 1997, the impact of the economic crisis including devalued currency and collapse of credit, was profound, wiping out the previous three decades of rapid economic growth. Environmental impacts including global change in climatic conditions and massive forest fires further stressed the nation. The population of the project area includes Dayaks (41%), Malaynese (40%), Chinese (11%), and others (8% - Javanese, Bugisneses, Maduranese, and Timorneses). Most are Christians including Catholics (60%) and Protestants with the religion including a strong mix with local customs and traditional ceremonies (including the healing of illnesses). With the emphasis on traditional spiritual practices, most households seek treatment from local healers before going to the formal health service. There are sacred places in the zone (viewed as sites of safe refuge) where ceremonies are conducted to call on the spirit of ancestors for strength in battle (tribal wars). The female literacy rate of the project area is 84% with 23% of mothers reporting a secondary school education. The national literacy rate for Indonesian adults is 89%. As in other parts of Indonesia, the man is accepted as head of the family, with women taking the responsibility of caring for the children and household. Most women assist their husbands with the agricultural work. With the economic declines and limited market opportunities, women are becoming more involved in small home industries such as weaving of baskets, selling of food items, etc. Men must fight the temptation of investing spare time created by unemployment in the popular past times of gambling, drinking, smoking, etc. Forty percent of women report that they are involved in activities outside the home and leave their children with grandparents (16%), an older child (11%) or their husbands (7%). World Vision 46 LCSP Midterm Evaluation High risk groups in the area include: villages located at far distances from the nearest health post (beyond 10km); villages that are inaccessible during the rainy season during to impassable streams; households with poor food security; villages located in the forest in the high transmission areas for malaria; villages with unprotected water sources and no latrines; low birth weight babies; children whose mothers work away from home and leave their children at home; mothers who are younger than 18 years of age; mothers who give birth with the assistance of an untrained attendant; Children who are provided anti￾diarrheal medicine or who are not given the same or more fluids during diarrhea episodes; etc The constraints to child survival include regional insecurity with periodic violent clashes occurring along ethnic lines. The civilian-military relationship continues to be a potential stress point with the collapse of the Soeharto government. The lack of household income sources constrains food security impacting on child nutrition levels, access to primary education, the ability to pay for medical services as well as transport to access those services, and adequate shelter to provide protection from the environment. Other development constraints are the popular social practices of gambling, drinking and "karaoke" / prostitution, further robbing scarce household resources and development. Profile of Infant, Child and Maternal Mortality, and Fertility Indonesia, over the last three decades, has worked hard to improve the health and welfare of its population, and has been noted to be making remarkable success. It has also done well in reducing its population growth rate by investing heavily in family planning programs. USAID reports it as one of their most successful family planning initiatives in the world. In spite of the encouraging information, there are two problems that must be taken into account: there continue to be pockets of great need, and the recent economic reversal poses serious threats to gains in fertility reduction and improvements in infant, child and maternal mortality. Indonesia is a vast archipelago of close to 14,000 islands scattered across several time zones. It is a mosaic of diverse cultures ranging from the ultra-modern skyscrapers of Jakarta Pusat to the stone-age communities of the highlands of Irian Jaya. Pockets of need like Landak exist throughout the country. Prior to the current economic crisis, the World Bank published its Health, Nutrition and Population Sector Strategy in 1997. This document characterized Indonesia as a country with low access to health services, in spite of an economy growing at an average of 6-8% during the period 1985-1995. The GNP per capita in 1997 was roughly $1,060, which fell due to currency devaluations to less than $300 in the last year. It was noted in the report that over 15 years (1980-95), Indonesia had done well in curbing its population growth: TFR figures had dropped 38% to 2.7 per woman who lived to 45 years, and the national population was growing at a comfortable 1.6% per annum. Full immunization coverage rate was 89% for the whole country, and TT2 coverage was 67%. There were major problems even at that time. The under-five mortality rate (75/1000 live births) was still the highest and the physician to population ratio (0.2 per 1000 World Vision 47 LCSP Midterm Evaluation population) the lowest among ASEAN countries. Secondary school enrollment was still low (43% of the eligible population) and child malnutrition was still high (40% of pre￾schoolers undernourished). 64% of women of childbearing age were anemic. Only 63% of the population had access to clean water. Only 43% of the population had access to health services. The MMR was still high at 390/100,000 live births. The total health expenditure per capita was $14, again the lowest in the region. It is reasonable to assume that the national and regional statistics are deteriorating as a result of the economic crisis and with the multiple internal conflicts which have left many displaced for significant periods of time. The following table shows comparative data between the project area, the province to which it belongs, and to the country as a whole. Table 1. Health Indicators in Indonesia, West Kalimantan, and Landak Indicator Indonesia West Kalimantan Landak Infant Mortality Rate (/1,000 live births) 49 56 - Under-5 Mortality Rate (/1,000 live births) 81 94 - Maternal Mortality Ratio (/1,00,000 live births) 390 520 623 623 Population Growth Rate (% per year) 1.63 2.56 1.6 Literacy Rate (% adults able to read and write) 89 85 84 Total Fertility Rate (children/ woman) 2.65 2.92 Sources: UNICEF, Situation of Women and Children in Indonesia, 2000; West Kalimantan Health Profile, Ministry of Health: 1999 Infant and Under-5 mortality: The top contributors to mortality are malnutrition, malaria, acute respiratory infection and diarrhea. More than 60% of under-five deaths in the project area are infant deaths as is the case for all of Indonesia. Local health center records show that the principal causes of infant and child mortality are still dominated by malaria, ARI and diarrhea. The project site is a high transmission area for malaria. Severe economic constraints at the household level, accompanied by frequent episodes of illness, have contributed to the declining nutritional status of children. Nationally, chronic protein energy malnutrition (affecting 23% of children), Vitamin A deficiency, Iodine deficiency, and anemia are noted nutritional problems. In West Kalimantan, 36% of children under five are reported to have chronic malnutrition and 11% of infants are born with low birth weights (<2.5kg). Landak District reports high levels of anemia among pregnant and lactating mothers, and the area is endemic for goiter. Also of concern in the area is the high number of deaths due to injuries/accidents (mainly vehicle related). World Vision 48 LCSP Midterm Evaluation Table 2. Principal Causes of Under-5 Mortality for Indonesia and Landak District Rank Indonesia % Landak District % 1 Acute Respiratory Infections 30 Malaria 26 2 Diarrhea 18 Diarrhea 22 3 Perinatal conditions 14 Acute Respiratory Infections 19 4 Various infections 7 Bronchopneumonia 13 5 Nervous system disorders 6 Sepsis 12 6 Vaccine-preventable diseases 5 Infectious diseases 3 MTE Comments: According to the above data, in Landak District the proportion of deaths under five due to Acute Respiratory Infections and Bronchopneumonia is 42%, that is, substantially higher than those due to malaria. Malaria is reported to be the single most important cause of death among under-5 children in Landak District, although it is not significant nationwide. Malaria is endemic in many parts of West Kalimantan particularly in the forest areas where rubber is grown. Malaria transmission in the project area appears to be consistent throughout the year contributing to high morbidity and mortality among infants, children and adults. Splenomegaly is a common condition in the CSP site as is malaria-related anemia. Chloroquine-resistance is reportedly not yet a major problem. Pneumonia (ALRI) ranks as the single most important killer of under-5 children in Indonesia and third in Landak District. At highest risk are infants and neonates. There are no known estimates of the number of ALRI episodes per child per year. Exclusive breastfeeding is practiced by 52% of mothers. Inadequate shelter with exposure to the environment is of concern. Diarrhea is among the top three causes of death for children under five. About one-third of households in the project zone have some type of latrine, but more alarmingly, 51% of mothers report disposing their infant's feces in the river. Only one-fifth of households in the CSP area obtain their drinking water from a protected source. Nationally, access to safe water is 77% and access to safe sanitation is 61%. Health center staff report that the majority of diarrhea cases are simple diarrhea, with only a small number of dysentery cases and no epidemics reported. Malnutrition Forty-one percent of under fives in West Kalimantan Province are underweight (-2z scores weight/age) compared to 30% nationally. Although the Landak District does not yet have nutritional data for its catchment area, health officials and the community agree it is a significant problem. Low birth weight in the province is 11% and over half of pregnant mothers are anemic. It is believed that maternal anemia (due to malaria), under nutrition during pregnancy and consequent intrauterine fetal under nutrition are significant contributors to the problem. Due to lack of supplies, a small World Vision 49 LCSP Midterm Evaluation proportion of mothers (21%) received a 90-day supply of iron and folate tablets during their last pregnancy. Vitamin A deficiency is still endemic and remains a threat to child survival in Indonesia. Although the first large-scale trials ever showing the impact of Vitamin A on child mortality were done in Indonesia and major successes have been achieved in reducing Vitamin A deficiency nationwide, the effects of the economic crisis are showing up in the decrease of Vitamin A coverage rates. Baseline survey results revealed only one-third of mothers knew about Vitamin A. However, the district reports a Vitamin A capsule coverage of 86% during the biannual distributions. Eighteen percent of mothers in the project zone reported they themselves having symptoms of night blindness, and 4% report that their children have such symptoms. Vaccine-Preventable Diseases In 1995 and 1996, Indonesia still reported tetanus, diphtheria, pertussis and measles among the top ten killer diseases of children under five. The Landak District has not had any vaccine preventable disease epidemics since it was established, and no cases of AFP have been reported. The national immunization coverage is 55%, and it is 53% in the province and 23% in the project target area. Maternal Health, Mortality Reduction and Family Planning: Indonesia has placed a major emphasis on the reduction of the national MMR in the last fifteen years by designating trained midwives in every village and mandating the construction of village birthing posts (polindes). The care of mothers has greatly improved, resulting in large reductions in the numbers of pregnancy-related deaths throughout the country. The MMR for Indonesia is 390 per 100,000 live births; West Kalimantan is higher at 520 and Landak estimates are still higher at 623. Sixty-eight percent of births in the Province take place in the home, while in a survey in the project area, 74% of mothers had their last delivery attended by a traditional birth attendant. The major causes of maternal mortality are hemorrhage and infections. However, the nutrition of pregnant and lactating mothers constitutes an ongoing concern, as shown by the low proportion of those receiving micronutrient supplementation. Although Vitamin A supplementation for postpartum mothers is still planned, WV Indonesia has started introducing this practice in some of its project areas. Twelve percent of mothers in the project area had a card documenting they had received Vitamin A within 4 weeks of delivery. Fertility: The 1997 DHS showed that Indonesia currently has a national average TFR of 2.65 and West Kalimantan a TFR of 2.92. The reduction in the fertility rate and the national modern contraceptive coverage of 55% is largely due to the continued support of donors, including USAID, for the work of the National Council for Family Planning and its partners. In spite of the predominance of Islam as a religion, high levels of female literacy, concepts related to contraception, limitations on family size and fertility reduction are widespread throughout Indonesia. The project area registers a similar modern contraceptive use of 58%, with the oral pill and injections being the most commonly used methods. World Vision 50 LCSP Midterm Evaluation Current Child Survival and Health Programs in Impact Area: In Landak District, the Ministry of Health carries out the bulk of responsibilities for child survival and maternal health activities, as is the case for Indonesia at large. National policies are being implemented for most child survival interventions, as well as other public health initiatives such as TB control, malaria control, mental health, dental health, occupational health, and school health. However, the quality of services is uneven, and is known to be largely a function of supervision, continuing education of health workers, and logistical support. Under the current economic crisis, support for these services has been reduced. In West Kalimantan, the provincial government counts on a network of health staff and infrastructure for the implementation of health programs and policies. A Provincial Health Officer in Pontianak supervises the work of the District Health Officer. The District Health Officer in Ngabang supervises curative and preventive health activities in the 10 sub-districts of Landak. The District Health Officer coordinates all the health initiatives with the local government coordinator, known as the "Bupati" for all sectors of government service. Landak does not have a hospital so referral cases are sent 87km away from the capitol district of Ngabang to a hospital in Sanggau District. The Landak MOH has the following infrastructure. The health facilities in Landak District are:  14 Health Centers in 10 sub-districts  263 Posyandu (Integrated Service Post The health facilities in Mandor are:  1 Health Center with In-patient service – serving 17 villages,  8 Sub-Health Centers (Ngarak, Salatiga, Kerohok, Bebatung, Sebadu, Sekilap, and Mangkunyit)  13 Polindes (Village Birthing Place)  21 Posyandu (Integrated Health Post)  87 Posyandu Cadres / Voluntary Health Worker  20 Traditional Birth Attendants (TBAs) The health facilities in Sengah Temila are (Source: Health Profile of Pahauman Health):  1 Health Center with In-patient service in Pahuman & 1 Health Center in Senakin – serving 14 villages,  3 Sub-Health Centers in Pahuman (Saham, Sidas, Banying),  3 Sub-Health Centers in Senakin (Runut - Tonang, Seginah – Aur Sampuk, Gombang),  13 Polindes (Village Birthing Place) in Pahuman  55 Posyandu (Integrated Health Post)  344 Posyandu Cadres / Voluntary Health Worker  98 Traditional Birth Attendants (TBAs) – 52 of them are trained. World Vision 51 LCSP Midterm Evaluation Table 3. Functions and number of the various types of health facilities in the project area in Mandor and Sengah Temila and the entire Landak District Facility Function Mandor Sengah Temila Landak Puskesmas/ Health Center Clinic with in-patient, outpatient basic curative and preventive services given by professional health staff. 1 2 14 Pustu/ Sub-Health Center A smaller health center with no in-patient beds, offering basic curative and preventive services. 8 6 - Posyandu/ Integrated Service Post Trained village volunteers provide CS services on assigned days of the month, including the sale of essential drugs (village drug post). 21 55 263 Polindes/ Village Birthing Post Government midwife stands ready to assist in delivery all the time. 13 13 - Village Health Volunteers Volunteer health workers trained by the MOH receive essential drug kits but minimal supervision. Usually about 5 VHVs per "posyandu." 87 344 - Traditional Birth Attendants Polyvalent and can be trained further on several professional tracks (about half have been trained). 20 98 - Doctors Provide curative care; are usually MOH staff operating their own private clinics outside of work. 2 1 - In spite of the apparent adequacy of the above service delivery system, the regularity and quality of services, the maintenance of equipment and buildings, and the updating of technical knowledge of personnel are recognized weaknesses. While health centers often have the requisite buildings and facilities (including staff housing), they are not properly staffed, maintained and/ or are grossly under-utilized. Many village birthing posts are without personnel present during normal hours of operation. Staff interviewed at the health center state their greatest training need is for improved disease management and updating on treatment protocols. Health staff have poor access to health education materials. They also state that inpatient care is of poor quality due to lack of staff and supplies. Although the drug supply at the health centers appeared well stocked, staff World Vision 52 LCSP Midterm Evaluation report that there is frequent shortage of chloroquine and paracetamol. Also there is no transport to support the referral of patients, and individuals must find their own ride to the next higher level of service, even to the referral hospital located outside of the District. Villages that are beyond 10km to the nearest health service site are challenged with issues of access including locating and paying for transport to reach services. The fee for service charge at health centers is minimal and can also be waved if necessary. In these more remote villages, community members utilize traditional healers, the VHV if one is present, TBAs, medicines in the local shops and there own knowledge of home care. Traditional healers are not part of the formal health service and currently do not have any relationship or interaction with the MOH for training or any other purpose. There are roughly three healers per village and they each have their own treatment protocols including diagnosis using a special gem stone, use of herbs and leaves often soaked in water to make a tea, use of foods such as chickens, or restricting certain foods such as meat with blood, and many rituals including prayer and sprinkling of water with a specific leaf over the body of the patient. There are also approximately 5 small shops per village that sell a variety of food and household items. These shops carry basic medicines including aspirin, ORS sachets, cough syrup, chloroquine, and a basic antibiotic. When asked about the dosage of these medications, shop owners are not able to explain the number of tablets, times per day or number of days the medication should be taken. The proposed NGO partner for this project, Karya Kasih Foundation, is the only private NGO carrying out relevant development activities. It is an organization that focuses on community development with the vision of enabling "the Dayak community to select and manage their own social, cultural, economical, and political life with the goal of living together in harmony with a loving spirit and with respect and protection" according to the 1945 National Law. They support a diversity of community-based initiatives including education, small microentreprise, a village health solidarity groups, preservation of traditional culture and protection of natural resources. The organization has 300 staff and 10,000 members with assets of an office, one car and a computer. WV has a renewed MOU with Karya Kasih as of June 2000 (see Annex #5). WV also has relationship with multiple churches in the area for the support of community development. 4. PROGRAM DESIGN The Landak CSP is designed along the four main strategies below 1. Strengthen the quality and coverage of existing child survival programs in the area The project will support but not directly implement the health service or establish a new health system in the area, avoiding the creation of a parallel system with the existing government health system or duplication of efforts. Project effort will be in strengthening existing health system through various training and skills upgrading including provision of job aids and health education materials. Revitalizing dormant World Vision 53 LCSP Midterm Evaluation integrated service posts and village health posts and non-functioning village health volunteers. Skills training in defining catchment population and methods for monitoring coverage of indicators for this population will be emphasized. Monthly planning and evaluation meetings with district health officials and health center staff will help strengthen the coordination for the joint implementation of training and supervision activities. 2. Empower and equip communities for the prevention, early and complete management of common diseases. The role of the household and community in contributing to improved maternal and child health will be emphasized within all project initiatives. Key household emphasis behaviors will be targeted for behavior change communication and health workers and community volunteers will be guided to take action towards mobilizing communities to create an enabling environment for improved health. Exercises such as participatory learning and action, definition of and planning for catchment area and presence of community volunteers with routine supervision and health education inputs will help raise awareness and advocate for the adoption of key household practices. The LCSP will also facilitate participation of non-formal stakeholder such as church elders and community leaders, and select specific groups such as traditional healers, shop keepers, self help village groups, for training to improve the quality of their CS services. 3. Initiate and establish micro enterprises and health linkage The micro-enterprise component of WV’s ADP in the area is an activity focused on poverty reduction greatly valued by communities during this time of economic crisis. Many families need inputs at specific times of the year, to buy fertilizers during the planting season or to transport the recently harvested cloves. The credit unions provide access to loans with appropriate payment schedules. Training is usually provided on the creation of business plans, planning of marketing strategies and timing of sales. The approach to lending is through the establishment of community banking groups built on the basis of the self-help village groups. Additional training on CS interventions and household investments for improved health as well as creation of enterprises that provide health inputs (improved shelter, provision of nutrient rich food items, provision and sell of soap, iodized salt, selling of shoes or mattresses) will be mobilized. A credit with education model will be applied. The ADP will learn from the experience of an ongoing WV Indonesia ADP in Jakarta that has been utilizing a credit with education approach for the past 2 years. The ADP will participate in training opportunities with this project as well as cross exchange visits. 4. Become a demonstration site and multiplier to scale up The long-term vision for the project is to replicate the most effective practices in child survival, maternal health and micro enterprises at multiple levels. First, to the entire Landak District, with a target maternal and child health population of over 400,000. DIP discussions with District and Provincial Health authorities reveal expressed commitment World Vision 54 LCSP Midterm Evaluation and readiness to take learning to a provincial level and to utilize the district as a demonstration site. Within WV Indonesia's ADP network, opportunities will be provided to participate in key best practice workshops as well as on site visits. WV just completed a successful scaling up exercise for all its ADP impact areas in Zambia, Southern Africa utilizing its recent CSP in-country experience. Multiple WV operations in Asia have been successful in taking key successes from their CSP to the entire ADP ministry within their nation. This project has several best practice workshops planned with participants expected from the MOH provincial level, ADP participants from across Indonesia, and WV health personnel (many of whom are also implementing CS grants) from across WV programs in Asia. The USAID local mission in Jakarta and the National Ministry of Health will be visited twice a year to be given updates on project accomplishments and to share success of innovations as well as ongoing priority CS interventions and strategies. Relationship with Other Existing Health-Related Activities: a) Landak District’s and Provincial Ministry of Health - will be the government office that will supervise overall project activities and will be the point of contact for planning and prioritizing activities. All project operations will conform with the District Health Plan and existing MOH protocols. Project activities will be complementary, if not jointly executed, to those of the health facilities in the area. Training will be jointly planned and implemented. Results of project activities will be reported regularly to the District Health Officer (DHO), with a copy to the Provincial Health Officer. In return, the project will ask the DHO to provide the vaccines, medicines, training modules, and some basic resources, such as gasoline allowances, for its ongoing programs. Following the DIP exercise, WV signed a Memorandum of Agreement with the Provincial Health Office, District Health Office and District Chief Coordinator "Bupati". b) Karya Kasih - is the only local NGO operating in the subdistricts. Karya Kasih is an organization that focuses on community development with the vision of enabling "the Dayak community to select and manage their own social, cultural, economical, and political life with the goal of living together in harmony with a loving spirit and with respect and protection" according to the 1945 National Law. The organization has 300 staff and 10,000 members with as assets an office, one car and a computer. They support a diversity of community-based initiatives including education, small microentreprise, a village health solidarity groups, preservation of traditional culture and protection of natural resources, and self-help groups. The LCSP will provide training and support to the Karya Kasih staff to implement credit with education schemes among its self-help group village structures. c) In addition, the project will correspond with other approved child survival projects in Indonesia (both locally and centrally funded) as well as with other WV Asia CSP for mutual exchange of information and materials. The project's impact on WV ADP staff for equipping of skills for mobilizing CS interventions will be a key long-term investment for WV in Indonesia. Not listed here but of key importance, are the communities themselves, community-based volunteers, groups and informal traditional service providers operating in the project area. The key to sustainable improved practices World Vision 55 LCSP Midterm Evaluation and an enabling environment for the growth and development of children rests heavily upon their involvement with the CS initiatives. Entry and participation of eligible women and children All communities and households within Mandor and Sengah Temali sub districts will be mobilized to participate in CS activities. Special emphasis will be placed on communities who currently do not have access to health services (beyond 10 km to nearest service site) or communities where health post or volunteers are currently non￾functional. Women of reproductive age and children less than five years of age make up the core target group. Community-based registers of eligible women, children and newborns used by community volunteers, including TBAs and VHVs, to promote participation in CS activities and follow up on dropouts or high-risk households. However, there are many other key decision makers within the community who hold power for maternal and child health issues who must be targeted to create the required enabling environment. Since most of the people belong to Christian religion, church elders/priest will also be mobilized to encourage participation of eligible women, children and newborn in the program. Self-help village groups (Karya Kasih) and WV ADP staff who are knowledgeable of the project area and the community structures will provide essential guidance for community targeting and engagement. Relationship between the choice of interventions and strategies Causes of death The major causes of death for children in the area are malaria, diarrhea, and pneumonia. The CSP will target malaria and diarrhea prevention and early and complete management of these diseases. [MTE Comments: One recommendation of the DIP review to add ARI as a LCSP intervention was adopted during the FAR] Visits to local villages resonate with the need for improved water and sanitation systems. Working in the project zone readily convinces one of the needs for vector control as the number of mosquito bites in the evening and through the night are almost unbearable. The MOH already has initiated the targeting of pneumonia control, so it will not be a priority of this project. The district is not yet ready to incorporate IMCI protocols, but the project stands ready to support this initiative should the timing and readiness for this adaptation occur within the life of the project period. Given the frequency of illness episodes and the ever declining nutritional status in the project site, the LCSP will also include Vitamin A supplementation, education and promotion of Vitamin A rich food consumption both among children and pregnant and lactating mothers. No emphasis will be given to family planning due to the success of National Family Planning program and fairly high coverage of modern contraceptive utilization (over 50%). Attendance to antenatal clinics is also high; however, the project will seek to address malaria during pregnancy as well as malaria￾related anemia particularly among pregnant women. The MOH is targeting the training of TBAs, a wise investment given that more than 70% of deliveries in the zone are assisted by TBAs. The immunization program will be targeted, as current coverage levels are unacceptable low at 23%. The immunization program will provide an entry to World Vision 56 LCSP Midterm Evaluation strengthening existing health systems and the challenges they face with logistical support and coverage of their catchment areas. Strengths and weaknesses and work plans of existing health service The long time existence of the government health system with a relatively adequate infrastructure, the availability of health staff in most of the villages (village birthing posts), and a readiness to learn and receive skills and to partner with NGOs for improved CS intervention coverage are some of the strengths of the existing health system in the area. However, the existing health system also faces many challenges as they seek to establish their newly formed health district. With budget cutbacks, opportunities for upgrading health worker skills for improved diagnosis and treatment are scarce. There is a basic medicines supply, but shortages of high demand drugs such as chloroquine and paracetamol occur. Also, supervision visits to outlying service posts and community volunteers are infrequent. As all Indonesian citizens struggle to meet their basic household needs, many of the integrated service posts are found unmanned as staff work short hours so they can seek other sources of income. The MOH has not actively mobilized the community to the responsibility and opportunities they have for improving their own health practices at the household level. Health education activities are minimal with very few education materials available to support this activity. Data for decision making with the clear definition of indicators is expressed by the MOH as a welcomed area of support. Given that a foundational health structure is in place, the LCSP will seek to strengthen areas where gaps currently exist including health worker training for upgrading of skills, supervision support, community mobilization training and guidance, logistical support, revitalizing of dormant health posts and volunteers, and information systems strengthening. Health center staff will be provided the opportunity to define their catchment populations and begin to engage with community volunteers operating in these zones. Community Preference Baseline assessments including a KPC survey, Focus Group Discussions and Key Informant Interviews reveal gaps in service areas such as immunization coverage as well as weaknesses in community practices in relation to diarrhea prevention and control and malaria control. Communities readily acknowledge their need for improvement in water and sanitation systems and for assistance with malaria control. Of key importance to communities is the need to strengthen their household livelihood. WV's ADP presence will provide the opportunity for households to access credit and business skills training as well as increase their knowledge and investments for improved maternal and child health. Community leaders, volunteers, shopkeepers, and traditional healers all readily express their need to learn more about CS interventions and promote the growth and development of children. Expertise of the staff World Vision 57 LCSP Midterm Evaluation Through the presence of the WV ADP in the project zone, there are staff who familiar with the project area and trained in community mobilization skills as well as equipped to carry out development activities. Their current activities include strengthening of primary education, opportunities for micro enterprise development, improvements in water and sanitation, basic primary health care initiatives, etc. WV Indonesia at large benefits from the leadership of a strong national health unit that has excellent partnerships with both government, donor and NGO units. The health team has multiple health related grant experience and is motivated, hard working, experienced, and respected. They are well integrated into the Asia Regional Team and receive regular feedback and support from the senior Asia Regional Health Coordinator. Staff from WV Indonesia annually participates in off-site CS best practices in other countries where WV is operating CS grants. New Methods, Strategies or Materials in Proposed Program These include: The Community-Owned EPI and Pregnancy Registers  Community EPI Register. This is similar to the PHC/MOH EPI register, with the major (and crucial) difference being that children are entered by month/year birth, regardless of age at the time of first presenting for immunizations. Hence there is one page for each month of birth. This way, village post volunteers know that all empty spaces for children preceding that are children who are late (this readily identifies the children who require follow-up). They know that all children in pages for the preceding 9 months should have had their measles immunization. In addition, newly born children will be listed by the VHV each month even if they do not present for immunization, and will be tracked down if they don’t come. In contrast, the usual MOH/PHC routine is to list the children sequentially as they present for immunizations. In order to know how many defaulters there are, staff must look at each entry, calculate the age, and then determine whether or not they are behind.  Community Pregnancy Register. The PHC/sub-health centers have a TT register that lists all women (pregnant or not) receiving TT. The Landak CSP will keep this, but will add a pregnancy register to pick up neonatal and maternal deaths and make sure all surviving children have been listed in the Community EPI Register. In this way, village health volunteers can easily identify individuals in need of key preventive services. By linking pregnancy registers with infants EPI registers, maternal and neonatal deaths can be identified. From WV Cambodia’s experience, these two community-owned registers provide an effective and low-cost maternal and child health monitoring system that can be maintained and used by the villagers themselves to set their own priorities for health care. These tools have been designed using materials that are available locally and at low cost; can be maintained by users with an elementary reading level; are in a format compatible with government monitoring systems; and collect only that information that is actually useful for those collecting it. Communities have ongoing access to information such as current census, names of World Vision 58 LCSP Midterm Evaluation individuals in need key services, number of maternal and infant deaths, and migration patterns. The communities using this system have made impressive gains in major health indicators. The development of the two registers in Landak District will include periodic review and evaluation. The MTE will also include a review of the village monitoring system. Findings from sample surveys (such as the KPC survey) will be compared with village registers, and the CSP staff and their PHC/SC and NGO partners will conduct focus group discussions with VHVs regarding their understanding and use of the information. The Community-Based Death and Disease Surveillance System The CBDDS was developed by the Dhaka Integrated Child Survival project in Bangladesh, and has since been replicated in India, the Philippines and Cambodia. The LCSP will also adapt it for its own context. This village-based monitoring system provides a concrete avenue for communities to mobilize for CS interventions and increasing the use of services. It also provides a concrete example of how data can be used at the level it is collected for improved targeting of services and high-risk groups. Insecticide Treated Bednets for Malaria Control While bednets of varied quality and color (even hot pink and purple) are available in local markets for the price of 40,000 to 50,000rp (approx. $4), none of these nets are insecticide treated. WV has multiple experiences in Africa operating insecticide-treated bednet initiatives including the involvement of the private sector and village-based community and women's groups. The LCSP provides the perfect opportunity to share relevant malaria control experiences and best practices. The project team has already made contact with Bayer Pharmaceuticals in Indonesia, the Roll Back National Malaria Control program, and other NGOs with ITN initiatives to evaluate and strategize for the best avenue to increase the promotion of bednets sales accompanied by the promotion of annual reapplication of insecticides. Discussions at the DIP planning exercises with over 130 present, including provincial and district health authorities, community leaders and volunteers show keen interest in launching this effort to control malaria. 5. PARTNERSHIPS The local organizations(s) with which LCSP will work to build technical or managerial capacity are: 1. The Ministry of Health, with the main focus on the Landak District Health Office, and the health services in 2 out of its 10 sub districts. 2. Karya Kasih Foundation - Self-Help Village groups 3. Various community structures, including volunteers and community based groups Current capacity of the MOH in the project area is described in section "Strengths and weaknesses and work plans of existing health service." The capacity of the local partner World Vision 59 LCSP Midterm Evaluation organization "Karya Kasih" is discussed under "Relationship with Other Existing Health￾Related Activities." The project's definition for partnership is a "Mutually edifying relationship with one another to strengthen each other and to carry out their respective missions." A relationship where mutual learning takes place and is based on the highest possible standards of accountability and mutual support. The basis for interactions with partners is for the purpose of expanding capacity and heightening integrity and effectiveness. The overall goal of the partnership is sustainable impact on development among poor communities. In the development context and Indonesia in particular, many great challenges must be confronted including economic crisis, political instability, insecurity resulting from ethnic and religious clashes, growing inequality, etc. Partnerships with local entities are essential for the purpose of: improving efficiency and effectiveness; avoiding duplication of efforts; ensuring consistency of approach and messages; and building in sustainability. Characteristics of effective partnerships that the CSP acknowledges are: Share Common Goal Leveraging Knowledge and Skills through Relationships All have something to teach and something to learn Mutually agreed upon terms of engagement Respect for each other's values The CSP also acknowledges that effective partnerships require: Clear description of roles and responsibilities (Preferable to document agreement) Constant dialogue Require changed behavior on the part of both partners The results of effective partnerships are programs that are responsive to communities and build upon one another's strengths In summary, the CSP does not just look at what it has to offer its partners, but also what can be learned from their partners, whether making reference to the MOH, the community or local NGOs such as Karya Kasih. During the DIP planning exercise, the CSP asked each of its partners to complete and present an organizational self-assessment. This exercise provided a rich experience of increased understanding from which to build. Roles and responsibilities of partners were articulated in various MOUs. The CSP's capacity building and sustainability objectives focus on the strengthening of local partners. In summary, interactions with partners will include: - Training for upgrading of workers skills - Supervision and support for application of skills at work site - Joint planning, target setting and review of past accomplishment meetings to be conducted on monthly basis World Vision 60 LCSP Midterm Evaluation - Joint monitoring and evaluation activities - Sharing and discussion of all data as a basis for decision making and targeting including strengthening of information systems. - Emphasis on additional tools for facilitating community mobilization and participation as well as adoption of positive health practices (Participatory Learning and Action, Community based Surveillance, Behavior Change Communication, Participatory Education Methodologies, Health Education Materials - Guidance and support for integrating CS investments into ongoing micro enterprise development initiatives through the self-help village groups. - Assistance to strengthen link between community and the health service and its appropriate utilization - Assistance to health centers to better define and serve their catchment populations with the goal of increasing coverage of quality services. - Opportunities for cross-site exchange visits to learn from other CS and development initiatives both within and outside of Indonesia. - Assistance to identify, train and create relationship with informal health services such as traditional healers, shop keepers, etc. 6. HEALTH INFORMATION SYSTEM The CSP will build on and strengthen the existing MOH health information system related to child and maternal health. Core principles for the information system are:  Establish indicators first and then determine data to be collected;  Clearly defined catchment population so that denominators for each indicator can be established facilitating measurement and tracking of performance;  Encourage data to be analyzed and applied at the level it is collected before passing it up the system;  Only essential variables that guide decisions for program management and targeting of interventions - determine what is essential to know from what is nice to know; ensure community feedback loops so community can begin to own the problem;  Utilize both qualitative and quantitative data collection methods so that processes and results can be evaluated and there is a explanation of why along with the what;  Provide guidance on incorporation of supervision findings into district and sub district plans for priority activities;  Integrate training records and data to help guide supervision visits and reinforcement of health worker skills following these events.  Establish a balance between facility based data and population based data to ensure all target groups are represented and not just those currently accessing services;  Provide guidance for using data for identifying and following up high-risk groups for CS related morbidity and mortality. A variety of tools will be utilized to facilitate monitoring and evaluation activities: MOH data collection forms KPC survey instruments World Vision 61 LCSP Midterm Evaluation Focus Group Discussions Health Facility Assessments including Client Feedback Interviews Key Informant Interviews Participatory Learning and Action Reports and Visual Designs Training Records Supervision Records Community Based Disease and Death Surveillance System Self help group records The CSP will share its reports and overall analysis of findings for the project with WV Indonesia National Office, WV Asia Regional Office, WV US Office, Sub district, District and Provincial Health Offices and with local partners. Recommendation for midterm and final evaluations Although past experience with guidelines have been positive, they do leave the impression that evaluation teams are lead to focus on survey quantitative results and less on processes and critical analysis. Therefore, a recommendation is being made that participatory evaluations give greater attention to:  Role of partnerships in facilitating or constraining the achievement or objectives.  National Health System policies and economic context that facilitated or constrained the achievement of objectives.  Qualitative assessments that explore the services, activities, information or skills gained during project period from a variety of beneficiaries including mothers, fathers, volunteers, traditional leaders, and MOH counterpart teams.  Explore activities that were not a part of the program, but they felt were of priority or greatly beneficial to improved child and maternal health. World Vision 62 LCSP Midterm Evaluation B. 2002 Demographic and Health Services Profile of the Project Area MANDOR SENGAH TEMILA TOTAL Senakin Pahauman Population Total 27,502 20,737 30,145 78,384 Household 5,254 3,961 5,758 14,973 Pregnant Women 583 440 639 1,662 Infant 647 488 710 1,845 Under five 7,505 5,658 8,225 21,388 Community health workers Village Health Volunteer (Posyandu Cadre) 148 153 223 524 Traditional Birth Attendant, Trained 85 74 26 185 Trained Shopkeeper, Trained 91 93 8 192 Administrative units Sub Village (Dusus) 60 34 44 138 Village (Desas) 17 5 9 31 Health facilities Village Health Post (Posyandu) 28 29 43 100 Maternal Care Post (Polindes): Total 17 4 9 30 - Bidan 9 1 5 15 + Bidan 8 3 4 15 Sub Health Center (Pustu) 7 3 3 13 Primary Health Center (Puskesmas) 1 1 1 3 Source: LCSP, August 2002. World Vision 63 LCSP Midterm Evaluation C. MTE Terms of reference LANDAK CHILD SURVIVAL PROJECT (LCSP) WORLD VISION USAID COOPERATIVE AGREEMENT # FAO-00-99-00027-00 TERMS OF REFERENCE FOR THE MID TERM EVALUATION (July 30 – August 8, 2002) General Objective To conduct a mid term evaluation activity of the Landak Child Survival Project (LCSP) and make recommendations on future child survival activities and capacity building needs in Landak District. The Purpose of the Review The purpose of the Mid Term Evaluation is to identify what is working well, determine the adequacy of responses to the First Annual Review recommendations, identify areas that need improvement, and recommend useful actions to guide the staff through the last half of the program. The evaluation should recognize the achievement of the project and staff, assess progress toward sustainable high quality implementation and monitoring of child survival activities, identify barriers to achievement of goals and objectives, and recommend strategies for future extension and expansion of the project Specific Objectives 1. To assess progress in implementing the Detailed Implementation Plan (DIP) for the period. 2. To assess progress towards achievement of objectives or yearly benchmarks. 3. To assess if interventions are sufficient to reach desired outcomes. 4. To identify barriers to achievement of objectives. 5. To provide recommended actions to guide the program staff through the last half of the program. Attention all be given to the area of progress by intervention, capacity building, impact of program approaches such as partnerships, community mobilization, behavior, etc and sustainability. Evaluation Methodology The Terms of Reference proposes an evaluation strategy that fulfills the criteria established by the USAID Child Survival Mid Term Evaluation Guidelines. The evaluation methodology will include the following: Evaluation Team Leader The team leader Dr. Marc Jean-Paul Debay, PhD, MPH will facilitate the evaluation activities in a participatory manner and ensure that the review process is conducted according to USAID standards. Data Collection and Analysis: The evaluation team leader will be responsible for overall methodology and design of the data collection techniques, facilitating the analysis of the World Vision 64 LCSP Midterm Evaluation data, and providing an assessment of the quality of project implementation based on this data. The data collection technique may include:  WHO 30-Cluster KPC (Knowledge, Practice and Coverage) Rapid Survey;  An internal review based on data generated by the Health and Management Information System  Field visits/observation  Focus group discussion; and key informant interviews with stakeholders  Review of project documents  Others as required by the review team Proposed Evaluation Schedule July 30 Arrival of all review team members in Pontianak and Landak Meeting with the Chief of West Kalimantan Provincial Health Office July 31 Briefing on the activities and develop the MTE Team composition Aug 1 – 3 Field visits - Health Centers, community and partner NGOs. Review of records and files, staff interviews. Discuss principal findings and recommendations. Aug 4 Preparing MTE report draft Aug 5 Presentation and Debriefing in the sub-district level and ADP Pontianak/LCSP Aug 6 Presentation and Debriefing in the District Level and local stakeholders Aug 7 Presentation and Debriefing in the Provincial Level Aug 8 Presentation and Debriefing at the MOH RI Aug 9 – 31 Report Writing Team Composition Review Team Leader:  Dr. Marc Jean-Paul Debay, PhD, MPH (Johns Hopkins University Bloomberg School of Public Health) Coordinators:  Mr. Edi Sianipar (General Manager, Yayasan Wahana Visi Indonesia)  Mrs. Mary Lengkong, DDS, DDPH (National Health Advisor WV Indonesia)  Drs. Untung Sidupa (ADP Manager Pontianak/Landak)  Dr. Andre Tanoe, MHP (Technical Team Leader Landak CSP) Team Members:  Mr. Nithin Madhav (USAID Wahington)  Ms. Molly Gingerich (Health, Population and Nutrition Office - Director, USAID Indonesia)  Dr. Sri Durjati Boediharjo, MSc, DPH (Reproductive & Child Health Program, PHN Office, USAID Indonesia)  Dr. Ascobat Gani, DrPH (Faculty of Public Health, University of Indonesia)  Dr. Dachroni, MPH (Director General of the Health Promotion Directorate, MOH RI)  Dr. Sri Hermianti, MSc (Director of the Family Health Directorate – MOH RI)  Dr. Ina Hernawati, MPH (Chief of the Infant & Preschool Children’s Health Sub Directorate, MOH RI) World Vision 65 LCSP Midterm Evaluation  Dr. Ferdinand Laihad, DMM, MPHM (Chief of Malaria Sub directorate, MOH RI)  Dr. Lukman HL, MBA (Chief of the Maternal and Perinatal Directorate – MOH RI)  Dr. H. Mohammad Subuh, MPPM (Chief of the Maternal & Child Health Division, West Kalimantan Provincial Health Office)  Dr. Kamaruddin (Chief of the Landak District Health Office)  Dr. Fe Garcia, MPH (World Vision US)  Dr. Sri Chander, MPH (Regional Health Advisor, World Vision)  Ms. Esther Indriani, SKM (Project Officer Landak CSP)  Local NGO  Various stakeholders from Landak District Team LCSP – ADP Pontianak:  Drs. Markus Akim (Community Development Coordinator ADP Pontianak)  Ms. Ir. Elisabeth Erna, (Monitoring and Evaluation Officer ADP Pontianak)  Mr. Petrus Ipit, SH (Area Coord. for Sengah Temila Sub-district)  Mr. Ir. Malinus (Area Coord. for Mandor Sub-district& Training Coordinator)  Mr. Sarius (Development Motivator ADP Pontianak)  Mr. Boni, SPd (Development Motivator ADP Pontianak)  Mr. Hendrik Rupang, SKed (Monitoring and Evaluation Officer LCSP)  Mr. Albert Silalahi (Finance Officer LCSP)  Ms. Mariani Aritonang, SKM (Training Coordinator LCSP)  Mr. Bonar (Health Motivator)  Ms. Dini Susanti (Health Motivator)  Ms. Dewi Helpina (Health Motivator)  Mr. Herkulanus Didi (Health Motivator)  Ms. Lina Monika (Health Motivator)  Ms. Muliawati (Health Motivator)  Ms. Priska Noni (Health Motivator) Expected Outcome Dr. Marc Jean-Paul Debay, PhD, MPH, the external review consultant, will be responsible for preparing the final report, which must meet all the requirements outlined in the Mid Term Evaluation. A draft review report will be completed and presented at the conclusion of the review visit at the District, Provincial, and National levels. Following the visit, the consultant will edit and refine the draft document into final form. It will be the responsibility of the consultant to forward the final draft to the country office (Mr. James L Tumbuan), the regional office (Dr. Sri Chander) and the WVUS office (Dr. Fe Garcia) for approval and comments. It is essential that the Mid Term Evaluation report is received by all offices no later than Sep 30, 2002. World Vision 66 LCSP Midterm Evaluation D. MTE Team Members The following people participated in all or most of the MTE discussions, field visits, and presentations. Other representatives from the MOH, USAID, the local NGO and other stakeholders had been invited but did not participate. Review Team Leader:  Dr. Marc Jean-Paul Debay, PhD, MPH (Johns Hopkins University Bloomberg School of Public Health) Coordinators:  Mrs. Mary Lengkong, DDS, DDPH (National Health Advisor WV Indonesia)  Drs. Untung Sidupa (ADP Manager Pontianak/Landak)  Dr. Andre Tanoe, MHP (Technical Team Leader Landak CSP)  Ms. Esther Indriani, SKM (Project Officer Landak CSP)  Dr. Sri Chander, MPH (Regional Health Advisor, World Vision) Team Members:  Dr. Sri Durjati Boediharjo, MSc, DPH (Reproductive & Child Health Program, PHN Office, USAID Indonesia)  Dr. Ferdinand Laihad, DMM, MPHM (Chief of Malaria Sub directorate – MOH RI)  Dr. Fe Garcia, MPH (World Vision US)  Drs. Markus Akim (Community Development Coordinator ADP Pontianak)  Ms. Ir. Elisabeth Erna, (Monitoring and Evaluation Officer ADP Pontianak)  Mr. Petrus Ipit, SH (Area Coord. for Sengah Temila Sub-district)  Mr. Ir. Malinus (Area Coord. for Mandor Sub-district& Training Coordinator)  Mr. Sarius (Development Motivator ADP Pontianak)  Mr. Boni, SPd (Development Motivator ADP Pontianak)  Mr. Hendrik Rupang, SKed (Monitoring and Evaluation Officer LCSP)  Mr. Albert Silalahi (Finance Officer LCSP)  Ms. Mariani Aritonang, SKM (Training Coordinator LCSP)  Mr. Bonar (Health Motivator)  Ms. Dini Susanti (Health Motivator)  Ms. Dewi Helpina (Health Motivator)  Mr. Herkulanus Didi (Health Motivator)  Ms. Lina Monika (Health Motivator)  Ms. Muliawati (Health Motivator)  Ms. Priska Noni (Health Motivator) World Vision 67 LCSP Midterm Evaluation MTE team composition for the field visits TEAM 1 TEAM II TEAM III TEAM IV Marc Debay Sri Durjati Kamarudin Fe Garcia Sri Chander Mary Lengkong Untung Sidupa Andre Tanoe Esther Indriani Hendrik Rupang Ina Herawaty Markus Akim Mariani Subuh Erna Marcelinus Maran Laihad Albert Silalahi Dinkes Propinsi I Dinkes Propinsi I Dinkes Propinsi III Health motivators, Area coordinators, or Development Coordinators of the site visited by the field teams acting as facilitator for the visit August 1, 2002 Bonar Didi Lina Dewi Ipit Boni Malinus Sarius August 2, 2002 Dini Bonar Friska Didi Muliawati Ipit Sarius Boni August 3, 2002 Friska Dini Boni Muliawati Boni Malinus Didi Lina World Vision 68 LCSP Midterm Evaluation E. MTE Methodology The MTE team comprised the LCSP staff members, other ADP staff members, four representatives of the MOH (three from the Provincial Health Office, one from the District Health Office, and the director of the national malaria control program), one representative of USAID/Indonesia, 3 WVUS staff members (the National Health Advisor, the Asia Pacific Regional health advisor, and the WVUS Director of Health), and the external evaluator and team leader (See MTE Team members in Appendix D). Other representatives from the MOH, USAID, the local NGO and other stakeholders had been invited but did not participate. Prior to the MTE, the LCSP project conducted a series of surveys and prepared all the relevant data on project activities. The MTE KPC survey reached 450 mothers of children under 24 months of age, and the Key Informant Interview survey reached teachers, cadres, village leaders, etc. The LCSP staff also prepared a briefing package for the MTE team members including summary information sheets the project operations and accomplishments; most of these documents are included as Appendices in this report. The fieldwork of the MTE ran from Tuesday July 30th through to August 8th, 2002 (See schedule of the MTE in Appendix H). Most of the team resided in Pahauma throughout this period, and adopted a very intensive schedule of meetings and field visits. The LCSP staff had prepared a detailed program of work, which the MTE Team adopted immediately. The LCSP staff also organized all the related logistics very effectively. After orientation of the MTE Team of about 20 members, the first task was to identify the main achievements, issues and problems of the LCSP according to a simple list of topics developed on the basis of the USAID guidelines for the MTE of PVO/CSGP projects. The MTE team divided into three groups based on the technical expertise of their members and their relationship to the LCSP (external evaluator or project staff) to discuss the following topics: World Vision 69 LCSP Midterm Evaluation Group Topic 1: Technical approaches Malaria control Immunization Diarrheal diseases control Acute respiratory infection control Vitamin A 2: Cross-cutting approaches Community mobilization and advocacy Behavior change communication Capacity building Health workers performance improvement Sustainability 3: Program management Planning, monitoring and evaluation Human resources management Financial management Logistics Information management After presenting to the whole MTE team the list of problems and issues that they had identified, each group developed a series of individual and group guideline to be used during the field visits. As the LCSP staff had programmed the field visits in advance for practical reasons of logistics, the groups listed the questions most relevant by type of site and interviewee available, taking into account the sample size and the time constraints. An indicative description of the realized samples of sites and interviewees is presented below with estimates of the corresponding “universes.” The team agreed that the group discussions would be conducted with as many mothers, cadres, shopkeepers, and TBAs would be available at the time of the visit. Sampling unit Realized Sample Universe District Health Office 1 1 Sub District Chief Health Centers 3 3 Polindes 4 30 Pustus 4 13 Posyandu 4 100 Mothers of infants 50 1845 Cadres 20 524 Shopkeepers (trained) 10 192 TBAs (trained) 20 185 Community leaders 5 100 KSM groups 5 32 Credit Union Offices 2 2 For the site visits, the MTE team divided into four groups of similar composition of external evaluators and project staff. The lists of questions by type of site and World Vision 70 LCSP Midterm Evaluation interviewee were used as guidelines during the site visits (See the MTE field visits guidelines in Appendix F). Using all the information available to them, the task of each group was then to formulate answers to the various questions discussed with individuals or groups at each site. At the end of each day, each field visits group prepared a bilingual transcription of its findings and presented them to the entire MTE team. After the three days of field visits, the three groups that identified the questions for the field visits convened again to formulate the major achievements, issues and recommendations related to each topics of the MTE. To accomplish this task, each group had to use the findings from the field visits and all the information available from the KPC and other surveys, the various project documents, and the knowledge, intelligence, and judgment of its members. After each group presented its work to the entire MTE team, a small group of four prepared a summary of these findings to present and discuss during the series of debriefing meeting starting the next day. After the field visits, the Team Leader and the USAID representative conducted a series of individual staff interviews to investigate their motivation and satisfaction, and identify issues that had not been addressed during the group discussions and field visits. Issues of confidentiality were negotiated on an individual basis. The interview guideline is presented in Appendix G, and the very informative findings from these interviews are included in the various sections of the report, as appropriate. The successive presentations and discussions that followed at the sub district, district, provincial and national levels were excellent opportunities to collect feedback from the specific audience, and communicate information about the project activities and plans. Fair attempts were made to tailor the presentation to each specific audience. The first presentation at the sub district levels reached the health staff, village leaders, and all the participants to the MTE KPC survey. At the district level, the presentation was an opportunity for the project staff to discuss the MTE findings and recommendations with the former and the recently appointed District Health Officers, other health staff, community representatives, and village leaders. At the provincial level, the audience primarily included health staff from the PHO and representatives from other NGO working in the area. Finally, the audience for the last presentation at the MOH in Jakarta was exclusively made of representatives of the major national program related to maternal and child health and survival. At each presentation, the LCSP staff displayed various communication materials developed by the project and a sample ITNs. The MTE methodology has the advantage of allowing a large participation of various stakeholders who can each bring valuable information and perspective to the discussions and analyses and learn about the project achievements and experience. The methodology also has limitations such as the intensity of work needed to cover all topics with large groups with different background in a short period of time; the sample of sites and interviewee selected before the MTE did not necessarily correspond to the priority questions and issues identified by the groups; this sample was not necessarily representative; and the composition of the groups was not always balanced in terms of external evaluators versus LCSP staff. World Vision 71 LCSP Midterm Evaluation F. MTE Field Visits Guidelines Health district office 1. How is the vaccine distribution system from the District Health Office to the Health Center? 2. What kind of actions that the District Health Office should take if there is any diarrhea outbreak? 3. Does the District Health Office always give the Vitamin A Capsule according to their demands? 4. According to you, how is the community mobilization carried out by the project – in terms of health? 5. What is the kind of collaboration between the District Health Office and the project so far? What kind of benefits do you feel? 6. What is your future plan to improve the mothers and children’s health if someday the project ends? 7. According to you, did the LCSP prepare the community to be able to continue the child survival activities? 8. How is the commitment from the District Health Office to provide funding for mothers and children’s health? Health center/sub-health center/village birthing post 1. How is the Health Center’s preparedness to manage diarrhera outbreak? 2. Do the Health Centers have the protocol to manage the Diarrhea and Malaria? 3. IS there any collaboration between the TBAs and the Midwive in distributing Vitamin A for post partum mothers? 4. Do the Health Centers always receive supply from the District Health Office according to your request? 5. What is your opinion on the collaboration between the project and the Health Staff? 6. According to you, what are the project activities that supported the Health Center’s activities? 7. What energize you to carry out your daily duties? 8. What is your future plan to continue the child survival program after the project ends? Community leaders 1. According to you, who is the most responsible person for individual’s health? 2. Do you feel that you have been involved in LCSP’s activities? World Vision 72 LCSP Midterm Evaluation Camat/sub-district chief 1. Have you ever heard about LCSP? 2. What is your opinion of the project activities? 3. Is the project activities inline with the sub-district government programs? Mothers attending posyandus 1. Could you please mention the types of immunization? 2. Can you mention the immunization schedule for children and pregnant mothers? 3. Can you mention the benefit of those immunizations? 4. Where do you usually get immunization service? 5. How long (in hour) is the travel time needed to go to Posyandu by foot? 6. Do you know how to prevent diarrhea? 7. Where do you usually get ORS (Oral Re-hydration Solution)? Does that place always have enough supply of ORS? 8. What would you do if your child get diarrhea? 9. (For mothers) In Vitamin A distribution month, does your child get Vitamin A? 10. What do you think about the payment system of ITNs at the moment? 11. In Vitamin A month, does your child always receive Vitamin A? 12. Have you ever received post partum Vitamin A (within 1 month after delivery)? 13. Do you know where to buy ITNs? 14. Do you think using ITN is important to prevent malaria? Please explain. 15. According to you, who is the most responsible person for individual’s health? 16. Do the community feel that you have been involved in the project’s activities? 17. Do you think that the health messages made by the project is easily understood? Cader Posyandu 1. Could you please mention the types of immunization! 2. Could you please mention the immunization schedule for children and pregnant mothers? 3. Could you please mention the benefit of those immunizations? 4. Where do you usually get the immunization service? 5. Do you know how to prevent diarrhrea? Please mention. 6. Where do you usually get the ORS? Does that place always have enough supply of ORS? 7. What is the kind of support from the community leaders to the Posyandu? 8. (For Trained Cadres) Could you please mention the ways to prevent malaria and treat malaria? 9. What is the benefit of the project and cadres’ collaboration? 10. How is the health staff’s attendance in delivering service for the Posyandu? 11. What kind of service did the health staff give during Posyandu day? 12. What is your future plans to improve community’s health if someday the project ends? World Vision 73 LCSP Midterm Evaluation 13. Have you ever been ask to involve in the implementation of health care in this area? 14. Do you make routine and timely reports for your activities? If yes, what kind of report? If not, why? TBA 1. Have you ever given Vitamin A Capsule to post partum mothers? 2. What is the benefit that you feel so far from the collaboration with the project so far? 3. Could you mention what kind of things that should not be done while you are assisting mother to deliver a baby? Shopkeepers 1. What kind of benefits do you feel from the training conducted by the project so far? 2. Have you ever joined any shopkeeper facilitation for the shopkeepers? Is there any protocol to prevent malaria? 3. Do you sell antibiotics? 4. How do you recommend the buyers to use the antibiotics? 5. Do you sell anti-malarial drugs? 6. How do you recommend the buyers to use the anti-malarial drugs? 7. Do you sell ‘obat stelan’ ? 8. How do you recommend the buyers to use the ‘obat stelan’ drugs? Credit unions / self help groups 1. Do you know what kind of activities in health that has been carried out by the project for the community? 2. What kind of benefit does the KSM feel because of the project’s existence? 3. What do you think about the ITNS distribution system at the moment? 4. What do you think about the ITN payment system at the moment? 5. What kind of troubles do you have to distribute / sell the ITNs? 6. Is there any suggestion from the KSM to solve those problems? 7. Is there any problem to request more ITNs from the ADP? What is the KSM’s suggestion to solve the problem? World Vision 74 LCSP Midterm Evaluation G. MTE Individual Interview Guideline for LCSP staff Name: Title: Hire date: 1. What are your main achievements since on the job? 2. Can you describe your job? 3. What are the main difficulties you are facing in your job? 4. How do you collaborate with your colleagues? 5. How does your direct supervisor help you in your job? 6. What are your plans when the project ends? World Vision 75 LCSP Midterm Evaluation H. MTE Schedule DAY DATE ACTIVITY Monday July 29 Briefing at WV Office in Jakarta Tuesday July 30 Arrival in Pontianak - Meeting with the Chief of West Kalimantan Provincial Health Office Arrival in Piniuh and Pahauman - MTE team meeting to develop overall work plan Wednesday July 31 MTE team meeting: - Briefing by LCSP staff - Define MTE evaluation questions Thursday August 1 Field visits by four groups: health centers, community and partner NGOs. MTE team meetings to share discuss principal findings of the day and recommendations. Friday August 2 Field visits and MTE meetings, continued Saturday August 3 Field visits and MTE meetings, continued Sunday August 4 MTE team meetings to discuss general findings and recommendations, and prepare presentations Monday August 5 Presentation and debriefing with LCSP/ADP in Pahauman Tuesday August 6 Presentation and debriefing at the District Level in Nbangang, capital of Landak District Wednesday August 7 Presentation and debriefing at Provincial Health Office in Pontianak, capital of West Kalimantan Province Thursday August 8 Presentation and debriefing at MOH in Jakarta Debriefing at WV World Vision 76 LCSP Midterm Evaluation Program for the MTE presentations and debriefing Sub-district Level, August 5, 2002 Time Agenda Facilitator 09.00 – 09.15 09.15 – 09.30 09.45 – 11.15 11.15 – 12.00 12.00 – 12.15 1. Welcome remarks from the ADP Pontianak Manager 2. Welcome remarks from the Camat Sengah Temila 3. MTE Presentation by MTE Team 4. Discussion on MTE Results/Findings 5. Closing remarks from the Camat Drs. Untung Sidupa Camat Sengah Temila Team Leader MTE Manager ADP Pontianak Bupati Kabupaten Landak District Level, August 7, 2002 Time Agenda Facilitator 09.00 – 09.15 09.15 – 09.30 09.30 – 09.45 09.45 – 11.15 11.15 – 12.00 12.00 – 12.15 1. Welcome remarks from the ADP Pontianak Manager 2. Welcome remarks from MTE Team 3. Welcome remarks from the Bupati Kabupaten Landak 4. MTE Presentation by MTE Team 5. Discussion on MTE Results/Findings 6. Closing remarks from the Bupati Kabupaten Landak Drs. Untung Sidupa Team Leader MTE Bupati Kabupaten Landak Team Leader MTE Manager ADP Pontianak Bupati Kabupaten Landak Provincial Level, August 6, 2002 Time Agenda Facilitator 09.00 – 09.15 09.15 – 09.30 09.30 – 09.45 09.45 – 11.15 11.15 – 12.00 12.00 – 12.15 1. Welcome remarks from the ADP Pontianak Manager 2. Welcome remarks from the MTE Team 3. Welcome remarks from the West Kalimantan Provincial Health Office 4. MTE Presentation by MTE Team 5. Discussion on MTE Results / Findings 6. Closing remarks from the Bupati Kabupaten Landak Drs. Untung Sidupa Team Leader MTE Chief of West Kalimantan Provincial Health Office MTE Team Chief of Family Health Division World Vision 77 LCSP Midterm Evaluation I. List of key findings from MTE KPC Survey Sample Description:  Four hundred and forty nine (449) mother child pairs were interviewed. The mother age distribution was equal between the young ages group (45.2 %  15 – 25 years of age) and the older ages group (54.8 %  26 – 50 years of age). Most of the mother or 40.1 % were primary school graduated, with illiteracy rate of 19 %.  The number of children 0 – 11 months of age was 58.6 % (263/449) and children 12 – 23 months of age was 41.4 % (186/449)  Mostly the husbands are working as Farmer / Rubber Tupper (70.4 %  316/449), the others (30 %  133/449) are working as merchant, officer, or gold miner. 3 (0.6 %) mothers are widows.  60.1 % (270/449) of mothers are not working to take care their children. Only 39.9 % (179/449) of mothers had to work outside home to earn money. 16.2 % (29/179) mothers who work outside took care of their children while they were working. Family Water Source and Sanitation  79.8 % (358/449) of mothers take drinking water from the spring, river, and private/public uncovered well. Only 20.2 % (91/44) of mothers take from protected source of water (piped water or covered well).  50.1 % (225/449) of mothers and their families do not have access to latrine.  60.1 % (270/449) of mothers throw their baby’s disposal in the river, 21.4 % (96/449) of mothers just throw it away and only 16.7 % (75/449) of mothers throw it in to latrine. Breastfeeding and Nutrition  45.5 % (200/440) of mothers reported giving colostrums to their newborn – out of all 449 mothers, 9 mothers never breastfeed their babies.  54.8 % (241/440) of mothers breastfeed their last newborn within first hour after delivery; 86.4 % (380/449) of mothers started breastfeed in the same day after delivery.  51.3 % (60/117) of infants 0 – 4 months were exclusively breastfeed in the last 24 hours  62 % (57/92) of infants 0 – 3 months were exclusively breastfeed in the last 24 hours  High continuation of breastfeeding was reflected in the 92.3 %(169/183) of children 12 – 23 months who were still being breastfed. Note: from all 186 infants 12 – 23 months, 3 infants were never been breastfed. Growth Monitoring and Immunization  69.7 % (313/449) of mothers had Monitoring Card for their children under 24 months; which 84.3 % (264/313) of monitoring cards are Road to Health Card and 15.7 % (49/313) of monitoring cards are Mother and Child Health (MCH) books.  89.5 % (280/313) of children who had monitoring card were weighed at least once in the last 4 months.  37.1 % (116/313) of children who had monitoring Card were weighed once a month in the last 4 month  46.8 % (87/186) of children 12 – 23 months with cards had received complete immunization.  55.9 % (104/186) of children 12 – 23 months with card had received measles immunization.  DPT1 coverage 84 % (263/313) of cards or 58.6 % (263/449) of total; DPT3 coverage 57.5 % (180/313) of cards or 40 % (180/449) of total. The DPT drop out rate was 31.6 % (263 – 180 / 263).  OPV1 coverage 88.5 % (277/313) of cards or 61.7 % (277/449) of total; OPV3 coverage 67.1 % (210/313) of cards or 46.8 % (210/449) of total. The OPV drop out rate was 24.1 % (277 – 210 / 277). World Vision 78 LCSP Midterm Evaluation Vitamin A Deficiency  55.7 % (250/449) of mothers with children under 2 reported they know and mention at least one benefit of Vitamin A.  60.2 % (112/186) of mothers reported that their children 12 – 23 months received Vitamin A High Dose Capsule within the past 6 months.  49.7 % (152/306) of mothers reported that their children 6 – 23 months received Vitamin A High Dose Capsule within the past 6 months.  25.6 % (115/449) of mothers reported received Vitamin A High Dose Capsule within 4 weeks of delivery.  18.3 % (82/449) of mothers reported have symptoms of night blindness  12.9% (58/449) of mothers reported their children have symptoms of night blindness. Diarrhea Case Management  45.2 % (203/449) of mothers reported that their children had diarrhea within 2 weeks prior to the survey.  42.9 % (87/203) of children with diarrhea within past 2 weeks were treated with Oralit (ORS Package) or 19.4 % (87/449) of total  35 % (71/203) of children with diarrhea within past 2 weeks were given anti-diarrhea medicine (pill or syrup) or 15.8 % (71/449) of total  58.6 % (119/203) of children with diarrhea within past 2 weeks were given same amount or more fluids other than breastmilk during their diarrhea episode or 26.5 % (119/449) of total  29 % (59/203) of children with diarrhea within past 2 weeks were given same amount or more foods other than breastmilk during their diarrhea episode or 13.1 % (59/449) of total  61.6 % (125/203) of children with diarrhea within past 2 weeks whose mother sought advice or treatment outside their homes for the diarrhea or 27.8 % (125/449) of total  12.3 % (25/203) of children with diarrhea within past 2 weeks were not given any treatment for their diarrhea episode or 5.6 % (25/449) of total. Pneumonia Case Management  39.9 % (179/449) of mothers ever heard and mentioned at least one symptom of “Pneumonia” or “Paru – paru basah” in local language.  54.1 % (243/449) of mothers reported that their children had cough within past 2 weeks prior to the survey  63.8 % (155/243) of these children experienced rapid or difficult breathing - consistent with the diagnosis of pneumonia, or 34.5 % (155/449) of total.  58.7 % (91/155) of mothers reported seeking treatment for their children with pneumonia sign / symptoms or 20.3 % (91/155) of total Malaria  78 % (350/449) of mothers ever heard and mentioned at least one symptom of “Malaria” or “Dongok Kura’” in local language.  61.7 % (216/350) of mothers who ever heard about malaria mentioned correct main cause of malaria or 48.1 % (216/449) of total  4.5 % (20/449) of mothers reported that they have ITNs (Insecticide Treated Nets) in their house.  41.6 % (187/449) of mothers reported that their children c.  29.9 % (56/187) of these children still had fever at the time of survey or 12.5 % (56/449) of total.  57.8 % (108/187) of mothers reported seeking treatment for their children with fever or 24 % (108/449) of total. World Vision 79 LCSP Midterm Evaluation Prenatal Care  37.2 % (167/449) of mothers have MCH Book or Pregnant Mother’s Card for the youngest child.  14.9 % (67/449) of mothers had at least two TT injection before the birth of the youngest child.  86.9 % (390/449) of mothers had at least one prenatal visit prior to the birth of the youngest child.  82.2 % (369/449) of mothers received iron-folate supplements prior to the birth of the youngest child. Delivery/Immediate Newborn Care  90.2 % (405/449) of mothers reported that they delivered the youngest child in their own home  74.2 % (333/449) of mothers reported that they delivered the youngest child with the help of skilled health personnel (doctor, midwives, or trained TBA)  66.1 % (297/449)`of mothers reported that the umbilical cord of their youngest child was cut by scissor or razor.  90.2 % (405/449) of mothers reported that their babies were placed close with mothers immediately after birth. Postpartum Care  59.9 % (269/449) of mothers had at least one postpartum check up. Source: KPC Survey Report, July 2002. World Vision 80 LCSP Midterm Evaluation J. Project implementation timeline: July 2000 – August 2002 Selected Project Activity 2000 2001 2002 J A S O N D J F M A M J J A S O N D J F M A M J J A Management Decision to change project site X Baseline / Midterm Assessments X X DIP workshop&approval / FAR / MTE X X X X Staff recruitment1 1 2 3 4 * 5 6 7 * * Training Community members (#): Posyandu Cadre (524) X X X X X X X X X PDO Cadre (22) X Shopkeeper (192) X X X X X X TBA (185) X X X X X X Health workers: Micr.(1)/ToT (37)/Malar.(32)/IMCI (30) X X X X LCSP staff:2 1 2 3 4 5 5 6 7 8 9 Other 3 generators given to 3HC X ITNs Ordering and Launching X X 10,000 MCH booklets to PHO&L-DHO X Purchase of TBA kits, 15 vacc. Flasks, and more X 1 Numbering corresponds to the following training and number of trainees: 1: PLA (9); 2: Accounting (1, Bangladesh); 3: Training of Trainers (14 ); 4: Malaria (11); 5: IMCI (4); 6: Local Capacity for Peace (5); 7: Monitoring and Evaluation (3, Thailand); 8: KPC surveys (1, Cambodia); 9: Behavioral Change Communication (2, South Africa). 2 Numbering corresponds to the hiring dates of the following staff members (Project Officer and Financial Officer were hired previously): 1: Monitoring and Evaluation Officer; 2: TTL,then Training Coordinator; 3: TTL and 8 Health Motivators; 4: 2 Health Motivators and 1 Administrative Assistant; 5: 3 Health Motivators; 6: Training Coordinator; 7: Project Officer. World Vision 81 LCSP Midterm Evaluation K. Milestones of the Landak Child Survival Project May 20, 1999  USAID approved REACH (Rural Entrepreneurs and Advocates for Child Health) Project funding June 30, 1999  Dr. Chander (Regional Health Adsvisor) and Mrs. Mary Lengkong (WVI National Health Advisor) visited REACH project. July 15-20, 1999  REACH Technical Review meeting in Washington, from WVI: Mrs. Mary. Score of 83.1. Oct 4-8, 1999  Mr. Greg Kearns (WVUS PO for Indonesia), Mrs. Mary, Mr. Eddy Sianipar (GM Wahana Visi Indonesia), and Dr. Amsal (REACH Project Team Leader) went to Poso for REACH Project’s preparation Nov 1999  Recruitment of staff (midwives and nurse), Office set up, Qualitative data collection using PLA Dec 1999  Recruitment for Monev Officer (Ms. Elfrida) Jan 24-28, 2000  Dr. Chander and Mrs. Mary went to Poso for project backstopping and preparation for Baseline Survey Feb 14-19, 2000 Feb 20-24, 2000 Feb 25, 2000  KPC Survey Training followed by 40 participants  Mrs. Mary and Ms. Esther Indriani (PO for REACH) visited Poso to facilitate the Baseline Survey  Baseline Survey in 30 dusuns, 418 respondents  Presentation of preliminary baseline survey results to local government units in project areas  Recruitment of 6 midwives/paramedics, Training Officer (Dr. Ariston), and Finance Officer (Albert) March 4-10, 2000 March 28-29, 2000  Regional Quality Improvement Workshop in Dhaka, Bangladesh (attended by Dr. Amsal, Mangapul, Esther, and two local government partners i.e. Imam Subekti, MPH and Dr. Natsir Borman)  DIP workshop in Tentena, North Pamona. Attended by 140 participants, Dr. Fe Garcia, Greg Kearns, Rachel Brumbaugh, Dr. Chander, Mrs. Mary and Ms. Esther went to Poso April 16, 2000 April 30–May 12, 2000  Social riots in Poso. Staff evacuated to Palu.  DIP report writing in process  WVUS gave postponement for the DIP submission before on May 15 to July 15, 2000  Dr. Ariston joined the IMCI training in Makasar, organized by MOH RI. May 2000  Big social riot in Poso. Staff evacuated to Palu. June 2000  Staff evacuated to Jakarta  During the project suspension, the core team worked in other projects  Finishing of the REACH DIP Report. July 5, 2000 July 10, 2000 July 24-28, 2000  Mrs. Mary and Mr. Mangapul (ADP Poso Manager) went to Poso to do security assessment and feasiblity to continue the REACH Project.  Management meeting on the situation of Poso. The conclusion was to move the REACH project to another needy area with WVUS ADP.  Mrs. Mary and Ms. Esther went for field assessment to Pontianak District as the proposed new project area.  Informed Greg Kearns and Dr. Chander about the latest situation Aug 2000  Administrative preparation for the change of project site (asking approval from the West Kalimantan Provincial Health Office), reported to Dr. Fe Garcia, Greg Kearns, Dr. Chander and Dr. Sri Durjati (USAID-Jakarta).  Preparation for Baseline Survey  Recruitment process for the new Team Leader, Monev Officer, Training World Vision 82 LCSP Midterm Evaluation Coordinator, and Health Motivators. All the REACH staff were absorbed to other projects except the Finance Officer. Sep2000 Sep18-22, 2000 Sep25-27, 2000  Change of project name from REACH to Landak Child Survival Project  Training on Baseline Survey, attended by 24 participants.  Baseline Survey in 30 sub-villages, 252 respondents.  Mrs. Laura Grosso (new WVUS PO for Indonesia), Ms. Anne Henderson, Dr. Chander, came to project site to backstop the DIP & Sustainability Workshop preparation.  Recruited Mr. Hendrik Rupang as Monev Officer.  Recruitment process for the Health Motivators; 8 candidates were involved in the Baseline Survey as recruitment process. Oct 3-4, 2000 Oct 25-28, 2000  DIP & Sustainability Workshop, attended by 120 participants.  Social unrest in Pontianak.  Finishing of the Landak CSP DIP Report. Nov 2000  Approval of Landak CSP DIP Report.  Recruitment process. Dec 8-19, 2000 Dec 13-20, 2000  Esther went to Pontianak/Landak to conduct the PLA and the 2nd HFA.  DIP Review in Washington. From WVI: Mrs. Mary, Mr. Eddy S, Mr. Sisbandi (ADP Pontianak Manager).  Recruitment process. Jan 2001  One staff from Landak District Health Office (the Head of CDC) joined training for microscopist in Jakarta. Feb 2001  Preparation for Cadres Training.  Due to difficulties in recruiting MPH, we recruited Dr. Ronald as Project Team Leader March 9, 2001  Meeting on LCSP Staffing, attended by Mr. Laura Grosso, Mary, Eddy S, Mr. Sisbandi, Dr. Andre Tanoe, MHP, Dr. Ronald Gunawan, Lucy (PO Wahana Visi), and Ms. Esther in National Office Jakarta.  In reference to the proposal/USAID guideline, the Team Leader should be a person with MPH degree, so we appointed Dr. Andre Tanoe as the LCSP Technical Team Leader, and Dr. Ronald with his agreement as the Training Coord.  Recruitment of 8 Health Motivators. April 10-11, 2001 April 16-21, 2001  First LCSP training. Two batches of TOT for Health Staffs conducted simultaneously. Revise the DIP Workplan by inserting ARI component.  Dr. Andre, Ms. Esther, and Mr. Albert joined the ADP Operation in Puncak. May 2001  Malaria Training, facilitated by Dr. Ferdinand Laihad (CDC MOH RI).  The core team had a two-day meeting in Ngabang to discuss DIP and Workplan. ARI Intervention was inserted in the Logframe. June 3-16, 2001 June 25-27, 2001  Inter-country IMCI Training in Semarang, attended by Mrs. Mary.  Mrs. Laura Grosso and Mary visited the project.  Sought information for Deltamethrine and Impregnated Bed Nets. July 22-28, 2001  IMCI Training in Surabaya, attended by Dr. Andre, Dr. Ronald and Ms. Esther.  Printed 10,000 MCH Handbook. Aug 6-10, 2001 Aug 29-31, 2001  Mrs. Mary and Ms. Esther visited the project for backstopping and preparation for FAR in Sep2001.  Dr. Chander and Ms. Esther conducted the second backstopping visit for FAR preparation.  10,000 MCH Handbook was given to West Kalimantan Province, and 3,000 of them were allocated to the Landak District.  Recruitment process for 2 more health motivators and 1 Admin Assistant. World Vision 83 LCSP Midterm Evaluation Sep4, 2001 Sep19-24, 2001 Sep25, 2001 Sep26, 2001  Mr. Sisbandi resigned from WVI.  First Annual Review of Landak CSP.  FAR Team Leader was Prof. Ascobat Gani, DrPH.  The FAR Members: Dr. Chander, Dr. Sri Durjati –USAID Jakarta, Dr. Ina Hernawati (Under five Children’s Health – MOH RI), Mrs. Mary, Ms. Esther, LCSP Core Team members.  Presentation at the Bupati office, and handed over 3 generators for all 3 Health Centers in LCSP areas.  Presentation at the Depkes RI Jakarta.  FAR Report writing. Oct 1, 2001 Oct 31, 2001  Dr. Ronald (Training Coord) resigned from Landak CSP, and currently working for the Relief Dept.  Dr. Andre resumed the Training Officer’s tasks.  Submission of FAR Report by Prof. Ascobat.  2 health motivators resigned (Novi wanted to have clinical work, and Titien wanted to get married). Nov 5-23, 2001 Nov 24, 2001  Regional Workshop on Monev in Reproductive Health, Bangkok. Attended by Dr. Mohammad Subuh (West Kalimantan Provincial Health Office), Hendrik and Esther.  Dr. Andre went to Cambodia for the “Regional Collaboration to Build Field Capacity to Conduct KPC Surveys” until 24 Dec 2001.  Purchased 2,000 Permanet ®, asssited by WVUS (Laura Grosso and Bob Wilson).  Recruited 3 more health motivators. Dec 12-16, 2001  Dr. Sri Chander and Ms. Esther Indriani went to Landak to follow up the First Annual Review Recommendations and for project backstopping. Jan 8 –12, 2002 Jan 25, 2002  Mrs. Mary Lengkong and Ms. Esther Indriani backstopped the LCSP and disucssed about the MED plan, including plan to distribute ITNs  Collecting data on the Traditional Birth Attendants (TBAs) in the project areas (90 TBAs)  Finalizing the FAR Report.  Arrival of 2,000 ITNs in Pontianak Port Feb 2-8, 2002  Mrs. Mary Lengkong and Mr. Untung Sidupa (ADP Pontianak Manager) joined the Behavior Change Communication (BCC) Workshop in Johannesburg, South Africa, organized by CORE and CSTS – both paid by the LCSP  Arrangement of Tax Exemption Letter from the Ministry of Social Affairs for the 2,000 ITNs.  LCSP and ADP Pontianak conducted the SURPINAS (National Ministry Info System from WVI)  FAR Report was sent to MOH RI Mar 21, 2002  Mrs. Mary Lengkong and Ms. Esther Indriani joined the meeting in USAID with Ms. Lynn Krueger (Deputy Team Leader for USAID Office of Health, Population and Nutrition) and Dr. Sri Durjati (Reproductive and Child Health Programme)  Continued process to release 2,000 ITNs from the Pontianak port. Difficulties were occurred during the document arrangement because the Pontianak Tax Dept never managed Tax-Free imported goods before April 4, 2002 April 29-May 4 ‘02  The project were able to release the 2,000 ITNs from the Pontianak Port  Mrs. Mary Lengkong and Ms. Esther Indriani backstopped the LCSP  Meeting with Dr. Sri Durjati (USAID Jakarta)  Recruitment process for LCSP Training Officer  Purchased 100 TBA Kit and 15 Vaccine Flasks. Produced 1,000 Cadres T- World Vision 84 LCSP Midterm Evaluation shirt promoting Exclusive Breastfeeding and 200 TBA T-shirt promoting Vitamin A Capsule Distribution May 28, 2002  Launching of the ITNs in Landak District. Attended by the District Health Office Chief, community leaders, and KSM (Self Help Groups) representatives in ADP Pontianak’s area, and the representative of Permanet® Jakarta, Mr. Aidil Zarkasih.  Recruitment and placement of Ms. Marianni Aritonang  Recruitment process for LCSP Project Officer as Ms. Esther Indriani would take MPH Course in Netherlands June 2002  Preparation for MTE (TOR, consultant, etc)  Recruitment of Dr. Fransiska Kaligis (PO LCSP) July 2002  KPC Survey Training (16-20 July ’02) and KPC Survey Data Collection (22-25 July ’02).  MTE of Landak CSP (30 July – 8 Aug ’02) with the External Team Leader of MTE: Dr. Marc Debay, PhD (John Hopkins University), and the external team members: Dr. Fe Garcia (WVUS), Dr. Sri Chander (WV APRO), Dr. Sri Durjati (USAID Jakarta), Dr. Ferdinand Laihad (Malaria sub-dit MOH RI).  Dr. Fransiska resigned due to her desire to take specialist course on medical science Source: LCSP, August 2002. World Vision 85 LCSP Midterm Evaluation L. Follow up on recommendations from First Annual Review Recommendation Accept / Reject Person(s) Responsible Dateline Follow up A Accelerate integration of LCSP with ADP to enhance sustainability and maximize synergy 1. Re-organize organogram to reflect: i Close functional integration between LCSP and ADP staff Accept Team Leader Nov ‘01 One LCSP motivator handles the ADP sponsorship program. ITN program uses KSMs as distributors. LCSP involved in KSM quarterly meeting, especially in health sessions ii ADP Manager taking charge of LCSP, including financial management Accept Team Leader & ADP Sept ‘01 Every fund request has to go through ADP Manager. iii Recruitment of 2 senior PHNs, one in each kecamatan, and working closely with 2 Area Coordinators Accept Team Leader Nov ‘01 This is not necessary anymore since the aim of increased cooperation with health centers is already partially achieved as demonstrated by : - High number of training sessions, where most of the trainers are from health centers - Improved Posyandu sessions, which are now rarely held without a health center staff - Health centers started to invite ADP to give some ideas in their internal meeting.. iv Form geographic teams of 1 ADP motivator and 1 health motivator to cover a cluster of desas in each kecamatan. Accept Team Leader Jan ‘02 Done for : - TOGAs (family herbal medicine garden) and fish ponds projects where the ADP motivator helps LCSP motivator (in process) - ITN distribution, which was coordinated by pairs of LCSP and ADP motivators 2. Conduct a team building exercise with ADP & LCSP staff Accept Team Leader & ADP Jan ‘02 Done by : - AKU (Business Feasibility Analysis) training - Christmas celebration - LCP training World Vision 86 LCSP Midterm Evaluation Recommendation Accept / Reject Person(s) Responsible Dateline Follow up B Develop Joint Plan to increase CS/RH coverage 1. Negotiate with Kepala Dinkes and Mandor PHC Chief to accelerate the re-functioning / establishment of additional posyandus & posyandu kaders in Mandor kecamatan. Accept Team Leader Oct ‘01 6 New Posyandu were established in Mandor Sub Distric, 2 in process 2 New Posyandu were established in Sengah Temila Sub District Support in re-functioning 6 Posyandu 2. Introduce the latest tracking registers for new pregnancies at desa/dusun level and include 2 additional columns for iron/folate and post-partum vitamin A supplementation Accept Team Leader & Monev Dec ‘01 Started in May 2002 Posyandu’s Cadres are using the forms 3. Institutionalize local area monitoring by each Pustu and Polindes to increase the enrollment of new pregnancies and newborns Accept Monev Dec ‘01 Started in May 2002 TBAs are also helping them to fulfill the task. 4. Assist bidan desa/pustu staff to identify and work with all dukun bayis (TBAs) in their catchments areas. Accept Team Leader Oct ‘01 Combined with TBAs training activites Relationship with TBAs is closer than before LCSP invited many formerly unidentified TBAs to join the training. 5. Discuss and work out plan with Dinkes/Puskesmas staff to decrease missed opportunities for immunization Accept Team Leader Oct ‘01 All newborn can now get BCG immunization anytime A motorcycle was given for each health center to help staff go to posyandus Motivate Posyandu cadres to confirm whether health staff is available, and look for replacement otherwise. 6. Work with Kepala dusuns and Kepala desas/Posyandus to ensure that each new birth is registered and each newborn is enrolled in the EPI register. Accept Team Leader & Monev Dec ‘01 Still in process. The Kepala Dusun/ Desas/ Posyandus/ TBAs/ Bidans are registering newborns 7. Ensure that each pregnant woman receives the new MCH booklet Accept Team Leader May ‘02 Because only midwifes or nurse can give the MCH Book, Posyandu cadres motivate pregnant women to go to health centers, village delivery posts, or Posyandus World Vision 87 LCSP Midterm Evaluation Recommendation Accept / Reject Person(s) Responsible Dateline Follow up C Increase quality of Posyandu Sessions and Posyandu kaders 1. Work with Puskesmas staff to ensure that no immunization eligible child or pregnant women is turned away because of vaccine shortage/unavailability. Accept Team Leader Mar ‘02 Done by helping health center recount target numbers for each posyandu so they can prepare the right vaccine amount (except if the vaccine are not available!) 2. Conduct exit interviews of child caregivers/mothers who have received Posyandu services Accept Monev Oct ‘01 Carried out in November 2001 and February 2002 3. Develop and use supervisory checklists to assess quality of immunization services at Posyandu level. Accept Monev Oct ‘01 Checklists available in Posyandus 4. Train Posyandu kaders to enhance caregivers’ ability to recognize danger signs of pneumonia, diarrhea and malaria that require immediate careseeking in children under-5 Accept Team Leader May ‘02 Done in every Posyandu Cadre training sessions 5. Assist Polindes/Pustu/ Puskesmas to provide regular refresher training of Posyandu kaders. Accept Team Leader May ‘02 LCSP has carried out 2 batches of Posyandu Cadre refresher training in Mandor Area. 6. Initiate joint quarterly supervisory visits with Pustu/Polindes staff to Posyandu kaders using checklists for observation of practice. Accept Monev Dec ‘01 Done - but still without checklist. The checklist is being developed. 7. Train Posyandu kaders to disseminate the 16 key family practices of community IMCI to care-givers Accept Team Leader Mar ‘02 Some of the key family practices were taught to the care givers (see Section I.B.2 in this MTE report) World Vision 88 LCSP Midterm Evaluation Recommendation Accept / Reject Person(s) Responsible Dateline Follow up D Assist Landak District to increase quality of service at Puskesmas/Polindes/Pustu level 1. Develop jointly with Depkes/Jakarta, Provincial Dinkes, and Landak Dinkes a district-wide IMCI implementation plan. Accept Team Leader Apr ‘02 Done 2. Facilitate IMCI training of staff of 3 Puskesmas of Mandor and Sengah Temila kecamatan, with regular follow-up visits thereafter using supervisory checklists Accept Team Leader Monev MOH May ‘02 1 midwife from Pahauman Health Center attended the IMCI training in Provincial Health Office Another IMCI training will be conducted around Mid August 2002. 3. Facilitate IMCI training of staff of Polindes/Pustu level in Mandor and Sengah Temila kecamatans, with regular follow-up visits thereafter using supervisory checklist Accept Team Leader Monev MOH June ‘02 5 midwifes from 6 Polindes attended the IMCI training in the Provincial Health Office Another IMCI training will be conducted around Mid August 2002 4. Together with Landak Dinkes, introduce (1) Joint supervisory schedule for Puskesmas, Pustu and Polindes; (2) Quality of Care checklist; (3) Monitoring indicators of facility performance and supervision of Posyandu kaders; and (4) Monitoring indicators of ability of care-givers to recognize danger signs Accept Team Leader Monev MOH Oct ‘01 The joint supervisory visit schedule has been made and will starting in August 2002. LCSP will go to the field using the checklist in LCSP area only World Vision 89 LCSP Midterm Evaluation Recommendation Accept / Reject Person(s) Responsible Dateline Follow up E Develop and Operationalize Capacity-Building Objectives, Targets and Indicators 1. Arrange with Landak Dinkes a rotating secondment of 2 PHNs from puskesmas in Mandor/Sengah Temila to the LCSP team. Accept Team Leader MOH Dec ‘01 2. Facilitate the KSMs and LSMs to include health in their mandate. Accept Team Leader ADP Nov ‘01 3. Facilitate the LSMs in conducting an organizational development (OD) exercise Accept Team Leader ADP Mar ‘02 4. Jointly develop and use an Organizational Capacity Assessment (OCA) tool for KSMs. Accept Team Leader ADP Jan ‘02 5. Jointly develop and use an Organizational Capacity Assessment (OCA) tool for LSMs. Accept Team Leader ADP Feb ‘02 F Jointly implement the marketing and use of long-lasting insecticide-treated nets Jointly develop a social marketing plan with the KSMs, local health staff, Posyandu kaders for long lasting ITNs Accept Team Leader, ADP December ‘02 The ITN distribution system is carried out. Source: LCSP, August 2002. World Vision 90 LCSP Midterm Evaluation M.Training events attended by community members DATE AREA PARTICIPANT # June 7-9, 2001 Sengah Temila (Senakin) Posyandu Cadre 25 June 12-14, 2001 Mandor Posyandu Cadre * 25 June 25, 2001 Sengah Temila (Senakin) Shopkeeper 11 June 28-30, 2001 Sengah Temila (Senakin) Posyandu Cadre 28 July 23-25, 2001 Sengah Temila (Senakin) Posyandu Cadre 33 August 10-12, 2001 Sengah Temila (Pahauman) Posyandu Cadre 29 September 11-13, 2001 Mandor Posyandu Cadre * 89 January 17-19, 2002 Sengah Temila (Pahauman) Posyandu Cadre 30 January 18, 2002 Mandor Shopkeeper 20 January 28, 2002 Mandor Shopkeeper 11 January 30, 2002 Sengah Temila (Senakin) Shopkeeper 16 February 1, 2002 Sengah Temila (Senakin) Shopkeeper 7 February 2, 2002 Sengah Temila (Senakin) Shopkeeper 25 February 19, 2002 Mandor Shopkeeper 13 February 21, 2002 Mandor Shopkeeper 12 February 26, 2002 Mandor TBA Training I 17 March 4, 2002 Sengah Temila (Senakin) Shopkeeper 9 March 6, 2002 Mandor TBA Training I 25 March 7, 2002 Sengah Temila (Senakin) Shopkeeper 25 March 11-13, 2002 Sengah Temila (Senakin) Posyandu Cadre 33 March 11-13, 2002 Sengah Temila (Senakin) Posyandu Cadre 34 March 12-14, 2002 Sengah Temila (Pahauman) Posyandu Cadre 37 March 12-14, 2002 Sengah Temila (Pahauman) Posyandu Cadre 33 March 20, 2002 Mandor TBA Training I 26 March 20, 2002 Mandor TBA Training I 17 March 21, 2002 Mandor Shopkeeper 14 March 27, 2002 Mandor TBA Training II 17 April 2, 2002 Mandor TBA Training II 40 April 2-4, 2002 Sengah Temila (Pahauman) Posyandu Cadre 26 April 3-5, 2002 Sengah Temila (Pahuman) Posyandu Cadre 28 April 18, 2002 Sengah Temila (Senakin) TBA Training I 27 April 26, 2002 Mandor TBA Training II 4 April 30, 2002 Mandor Shopkeeper 10 April 30, 2002 Mandor Shopkeeper 11 May 1, 2002 Mandor TBA Training III 39 May 2-4, 2002 Sengah Temila (Pahauman) Posyandu Cadre 40 May 3, 2002 Sengah Temila (Senakin) TBA Training I 12 May 17-18, 2002 Mandor Posyandu Cadre Refresh 45 World Vision 91 LCSP Midterm Evaluation DATE AREA PARTICIPANT # May 31, 2002 Sengah Temila (Senakin) TBA Training II 27 June 1, 2002 Mandor TBA Training IV A 36 June 3, 2002 Sengah Temila (Senakin) TBA Training I 17 June 4, 2002 Sengah Temila (Senakin) TBA Training II 27 June 5, 2002 Sengah Temila (Pahauman) TBA Training I 26 June 11-13, 2002 Mandor Posyandu Cadre 34 June 17-18, 2002 Mandor POD Cadre 22 June 19, 2002 Mandor TBA Training IV 12 June 21, 2002 Mandor TBA Training III 26 June 21, 2002 Pahauman Shopkeeper 8 June 26-27, 2002 Mandor Posyandu Cadre Refresh 54 June 28, 2002 Sengah Temila (Senakin) TBA Training II 18 June 29, 2002 Sengah Temila (Senakin) TBA Training I 18 July 2, 2002 Mandor TBA Training IV B 41 * Also include KSM and Kades participants. Source: LCSP, August 2002. World Vision 92 LCSP Midterm Evaluation N. Training events attended by LCSP staff DATE TRAINING EVENT LCSP STAFF December 11-13, 2000 PLA (Participatory Learning for Action) Mr. Hendrik Rupang (LCSP Monev Officer) Ms. Esther Indriani (LCSP Project Officer) Ms. Dini Susanti (Health Motivator) Ms. Titien Zurianti (Health Motivator) Ms. Lina Monika (Health Motivator) Ms. Novianti (Health Motivator) Ms. Petronella (Health Motivator) Ms. Dewi (Health Motivator) Mr. Michael Thommy (Health Motivator) February 2001 Grant Accounting Workshop in Bangladesh Mr. Albert Silalahi (Finance Officer) April 10-11, 2001 Training of Trainers (TOT) for Health Staff Ms. Dini Susanti (Health Motivator) Ms. Titien Zurianti (Health Motivator) Ms. Lina Monika (Health Motivator) Ms. Novianti (Health Motivator) Ms. Petronella (Health Motivator) Ms. Dewi (Health Motivator) Mr. Michael Thommy (Health Motivator) May 19, 2001 Malaria Training Dr. Andre Tanoe (Technical Team Leader) Dr. Ronald Gunawan (Technical Training Coordinator) Mr. Hendrik Rupang (LCSP Monev Officer) Ms. Esther Indriani (LCSP Project Officer) Ms. Dini Susanti (Health Motivator) Ms. Titien Zurianti (Health Motivator) Ms. Lina Monika (Health Motivator) Ms. Novianti (Health Motivator) Ms. Petronella (Health Motivator) Ms. Dewi (Health Motivator) Mr. Michael Thommy (Health Motivator) June 2001 Training of Trainers (TOT) in Utilization of MCH Book Dr. Ronald Gunawan (Technical Training Coordinator) June 3-16, 2001 Inter-country IMCI Training Mrs. Mary Lengkong, DDS, DDPH (National Health Advisor) July 22-28, 2001 IMCI Training Dr. Andre Tanoe (Technical Team Leader) Dr. Ronald Gunawan (Technical Training Coordinator) Ms. Esther Indriani (LCSP Project Officer) September 17-19, 2001 Local Capacity for Peace Building - WVI Dr. Ronald Gunawan (Technical Training Coordinator) Ms. Dini Susanti (Health Motivator) World Vision 93 LCSP Midterm Evaluation DATE TRAINING EVENT LCSP STAFF Ms. Titien Zurianti (Health Motivator) Ms. Lina Monika (Health Motivator) Ms. Novianti (Health Motivator) November 5-23, 2001 Regional Workshop on Monev in Reproductive Health, Bangkok. Mr. Hendrik Rupang (LCSP Monev Officer) Ms. Esther Indriani (LCSP Project Officer) Dr. Mohammad Subuh (Chief of the Family Health Dept. of the West Kalimantan Provincial Health Office) November 24 – December 24, 2001 Regional Collaboration to Build Field capacity to Conduct KPC Survey, Cambodia Dr. Andre Tanoe (Technical Team Leader) February 4-8, 2002 BCC (Behavior Change Communication) Workshop in Johannesburg, South Africa Drs. Untung Sidupa (ADP Pontianak Manager) Mrs. Mary Lengkong, DDS, DDPH) Source: LCSP, August 2002. World Vision 94 LCSP Midterm Evaluation O. ADP Pontianack – Landack Child Survival Project Organizational Chart * Last Revision: July 29, 2002 YAYASAN WAHANA VISI INDONESIA ADP PONTIANAK – LANDAK CHILD SURVIVAL PROJECT ORGANIZATION CHART Yayasan Karya Kasih Yayasan Pancur Kasih ADP Manager Mr. Untung Sidupa Finance Officer Mrs. Johana Tobo CSP Tech. Team Leader Dr. Andre Tanoe, MHP Admin Assistant Ms. Reni Book Keeper Vacant Driver Mr. Said Health Motivator Ms. Dini Health Motivator Ms. Muliawati Health Motivator Ms. Asih Health Motivator Ms. Dewi Health Motivator Mr. Herculanus HealthMotivator Mr. Bonar Health Motivator Ms. Lina Health Motivator Ms. Friska SR Coordinator Mr. Deni Teman CD Coordinator Mr. Markus Akim Mandor Area Coord Vacant CD Motivator Mr. Malinus CD Motivator Mr. Bona CD Motivator Mr. Anto Sengah Temila Area Coord Mr. Petrus Ipit CD Motivator Mr. Sarius CD Motivator Mr. Boni CD Motivator Ms. Setiana Toho Area Coord Mr. Sutejo CD Motivator Mr. Tri Johan CD Motivator Mr. Amon CD Motivator Mr. Herdi Book Keeper Ms. Cecilia Driver Mr. Agung Purchasing Mr. Alexander Lini Guard Mr. Hadi GS Vacant Training Coordinator Ms. Mariani Finance Officer Mr. Albert Silalahi Monev Officer Ms. Elisabeth Erna Monev Officer Mr. Hendrik Rupang Local Capacity for Peace Coord Ms. Maria Yustiningsih Training Coordinator Mr. Malinus Source: LCSP, August 2002.