Final Evaluation of the Kean Svay Child Survival (Follow On) Project December 8–19, 2003 Report Prepared by Marc Debay On behalf of the FE Team FAO-A-00-00-00038-00 i Table of contents A. Summary .................................................................................................................................1 B. Assessment of Results and Impact..........................................................................................5 1. Results: Summary Charts....................................................................................................5 2. Results: Technical Approach ..............................................................................................8 a. Project Overview.............................................................................................................8 b. Immunization ................................................................................................................11 c. Nutrition........................................................................................................................13 d. IMCI..............................................................................................................................16 e. Birth Spacing.................................................................................................................20 3. Results: Cross-cutting approaches ....................................................................................22 a. Community mobilization...............................................................................................22 b. Communication for Behavior Change ...........................................................................23 c. Capacity Building Approach.........................................................................................26 (i) Strengthening WV US and WV Cambodia...........................................................26 (ii) Strengthening local partner organizations.............................................................27 (iii) Strengthening Health Facilities.............................................................................28 (iv) Strengthening health worker performance ............................................................30 (v) Training .................................................................................................................30 d. Sustainability Strategy...................................................................................................31 C. Program Management ...........................................................................................................33 a. Planning.........................................................................................................................33 b. Project staff training......................................................................................................33 c. Supervision of project staff ...........................................................................................33 d. Human resources and staff management.......................................................................33 e. Financial management...................................................................................................34 f. Logistics........................................................................................................................34 g. Information Management ..............................................................................................35 (i) Monthly Activity Reports......................................................................................36 (ii) Supervision and COPE reports..............................................................................36 (iii) Intermittent surveys and assessments....................................................................36 (iv) CBDDS..................................................................................................................37 h. Technical and Administrative Support..........................................................................39 i. Management Lessons Learned......................................................................................40 D. Conclusions and Recommendations......................................................................................41 1. Child health and survival...................................................................................................41 2. Capacity building ..............................................................................................................43 3. Data and documentation....................................................................................................43 E. Results highlights..................................................................................................................46 ii APPENDICES...............................................................................................................................47 Appendix A Final Evaluation Team Members ..........................................................48 Appendix B Final Evaluation Methodology..............................................................50 Appendix C Final Evaluation Documents Reviewed.................................................54 Appendix D Final Evaluation Schedule .....................................................................55 Appendix E Timeline of project implementation.......................................................56 Appendix F Update on MTE recommendations ........................................................57 Appendix G Lessons Learned Workshop Agenda .....................................................66 Appendix H Lessons Learned Workshop List of Participants...................................67 Appendix I VHV Knowledge assessment..................................................................69 iii List of Tables Table 1 Achievements of Intermediate Results and sub Results of the KSCSP.............................5 Table 2 Achievements of end-of-project objectives in Leuk Dek ..................................................6 Table 3 Capacity building and sustainability objectives.................................................................7 Table 4 Demographic and health services profile of the four implementation areas of KSCSP....9 Table 5 Relative support provided by KSCSP to its five implementation models.......................10 Table 6 KSCSP achievements by implementation model and intervention..................................42 List of Figures Figure 1 Percentage of children 12-23 months completely immunized by age one .....................12 Figure 2 Percentage of mothers of children 0-11 months who received at least 2 doses of Tetanus Toxoid during last pregnancy................................................................................................12 Figure 3 Percentage of children 0-23 months who were breastfed within one hour after birth....13 Figure 4 Percentage of children 0-3 months exclusively breastfed during the last 24 hours........13 Figure 5 Percentage of children 6-23 months who received one dose of vitamin A during the last six months .............................................................................................................................14 Figure 6 Percentage of mothers of children 0-23 months who received 1 dose of vitamin A within four weeks post partum..............................................................................................14 Figure 7 Percentage of children 0-23 months with diarrhea the last two weeks who received ORT ...............................................................................................................................................16 Figure 8 Percentage of children 0-23 months with diarrhea in the last two weeks who received an antidiarrheal medicine or antibiotic ..................................................................................16 Figure 9 Percentage of children 0-23 months with cough and difficulty breathing the last two weeks who were brought to a trained provider.....................................................................17 Figure 10 Percentage of caretakers of children 0-23 months who can name at least one danger sign of DHF (cold extremities, bleeding, abdominal pain, weakness).................................17 Figure 11 Percentage of children 0-23 months with diarrhea/sick in the last two weeks who received the same amount or more food ...............................................................................19 Figure 12 Percentage of children 0-23 months with diarrhea/sick in the last two weeks who received the same amount or more fluids other than breast milk..........................................19 Figure 13 Quality of case management of childhood illness in Kean Svay and Kampong Thom ODs........................................................................................................................................20 Figure 14 Percentage of mothers of children 0-23 months, not pregnant and who do not desire a child within two years, who are using a modern method of contraception...........................21 Figure 15 Method-specific contraceptive prevalence among mothers of children 0-23 months, not pregnant and who do not desire a child within two years in Leuk Dek ................................21 Figure 16 Number of men and women who received health education from VHVs in Leuk Dek, FY01 to FY03........................................................................................................................24 Figure 17 Average knowledge score on various topics for VHVs in Leuk Dek...........................25 Figure 18 Knowledge score on CDD, DHF and ARI among mothers attending health centers in Leuk Dek...............................................................................................................................25 Figure 19 Trends in mortality under five as reported by the CBDDS in Leuk Dek .....................38 iv Acronyms AD Administrative District ADP Area Development Program ARI Acute Respiratory Infections BCC Behavior Change Communication CBDDS Community-Based Disease and Death Surveillance CS Child Survival DIP Detailed Implementation Plan EPI Expanded Program of Immunization FBC Feedback Committee FE Final Evaluation HAHP HIV/AIDS and Health Program (WVC) HCMC Health Center Management Committee HIS Health Information System IMCI Integrated Management of Childhood Illness KPC Knowledge, Practices and Coverage KSCSP Kean Svay Child Survival Project LQAS Lot Quality Assurance Sampling MCH Maternal Child Health MOH Ministry of Health MTE Midterm Evaluation NGO Non-Governmental Organization OCA Organizational Capacity Assessment OD Operational District ORT Oral Rehydration Therapy PVO Private Voluntary Organization RH Reproductive Health TBA Traditional Birth Attendant USAID United States Agency for International Development VDC Village Development Committee VHSG Village Health Support Group VHV Village Health Volunteers WV World Vision WVAPRO World Vision Asia Pacific Regional Office WVC World Vision Cambodia WVUS World Vision United States 1 A. Summary The Kean Svay Child Survival (Follow On) Project (KSCSP Phase 2; FY 2001-2003) aimed at applying lessons learned from the previous Kean Svay Child Survival Project (KSCSP Phase 1; FY 1997-2000) to other areas in Cambodia and at building the related capacity of the Ministry of Health (MOH) and World Vision Cambodia (WVC). A particular interest was the development of models of integration of child survival into the World Vision’s Area Development Programs (ADP) in Cambodia. The technical interventions of the two phases of the KSCSP were immunization, nutrition, case management of childhood diseases and birth spacing. The local partners of WVC in the implementation of the KSCSP were the government health services (the Operational District—OD and the Health Centers—HC) and the communities (primarily the Village Health Volunteers—VHVs and the Health Center Management Committees—HCMC members) in the Kean Svay and Kampong Thom ODs. In three years of implementation, the KSCSP has been successful in achieving all its child health and survival objectives in the direct impact area, the Leuk Dek Administrative District (AD; population 52,779). Below are highlights of key project indicators from the baseline and final Knowledge, Practice and Coverage (KPC) surveys in Leuk Dek. Results of the Key Interventions of the KSCSP in Leuk Dek between November 2000 (baseline) and August 2003 (end of project) # Key Project Indicator Base Line End of Project 1 % of children 12-23 months who had been fully immunized before age 12 months 71% 98% 2 % of mothers of infant 0-11 months would received two doses of Tetanus Toxoid during last pregnancy 26% 99% 3 % of infants 0-4 months who were exclusively breastfed in the last 24 hours 7% 43% 4 % of children 6-23 months who received a dose of vitamin A within last six months 26% 92% 5 % of mothers of infants 0-11 months who received a dose of vitamin A within 2 months of delivery 2% 98% 6 % of families with children 0-23 months who reported use of iodized salt 22% 66% 7 % of children 0-23 months with cough and difficult or rapid breathing in the past 2 weeks were brought to a trained health provider 66% 100% 8 % of children 0-23 months with diarrhea in the past two weeks who were treated with ORT 53% 100% 9 % of children 0-23 months with diarrhea/sick in the past 2 weeks who were given the same amount or more food during their diarrhea/sickness episode 55% 81% 10 % of children 0-23 months with diarrhea/sick in the past 2 weeks who were given the same amount or more fluids during their diarrhea/sickness episode 75% 83% 11 % of caretakers of children 0-23 months who can name at least one danger sign of DHF 21% 97% 12 % of women with children 0-23 months, who are not pregnant and desire no more children in next two years or are not sure, who are using modern contraceptives 6% 51% 2 The KSCSP was also implemented in two other ADs of the Kean Svay OD and in the Kampong Thom OD (in three of the five ADs, considered as one implementation area). The first panel in the table below summarizes the support that the KSCSP provided to the three levels of the health system (OD, HC, and community) during the two phases of the project in Kean Svay AD and in the three other implementation areas in Phase 2. The presence of an ADP and the type of support received from the KSCSP is also specified. These various types and relative intensity of support characterize five KSCP implementation models. The second panel of the table provides a qualitative assessment of the achievement of the project with respect to the key technical interventions as measured through a series of KPC surveys throughout the seven years of implementation (“+” refers to a positive achievement of the intervention objective as stated in the DIP or equivalent). 3 Implementation models Kean Svay OD Kean Svay AD Phase 1 Phase 2 Leuk Dek AD Lovea Em AD Kampong Thom OD (3 ADs1 ) Support Type of support Level of Health System Operational District +++ +++ +++ +++ + Health Centers +++ ++ +++ + + Mobile team +++ - - - - Outreach services - + +++ - - Community +++ + +++ - - Area Development Program Technical - - +++ - ++ Financial - - +++ - + Achievement Intervention Immunization 1. Complete immunization of infants + - + + - 2. Tetanus Toxoid in pregnant women + - + + N/A Nutrition 3. Exclusive breastfeeding + + + + N/A 4. Vitamin A in children 6-23 months + - + - N/A 5. Vitamin A in post partum mothers + - + + N/A 6. Use of iodized salts + - + - + IMCI 7. Use public health facility when ARI + + + + N/A 8. ORT when diarrhea + + + + N/A 9. Same or more food when sick + + + + + 10. Same or more fluids when sick + + + + + 11. Knowledge of danger sign of DHF + + + + N/A Birth spacing 12. Contraceptive prevalence + - + + N/A Note: Support: - none; + low; ++ medium; +++ high. Achievement: + positive (objectives achieved or equivalent result); - poor achievement; N/A: no available data. 1 Prasath Sambo, Prasath Balang, and Sandan. 4 Preliminary conclusions of this comparison of the various implementation models follow. - All the objectives were met in Phase 1 in Kean Svay and in Phase 2 in Leuk Dek, the two models of relatively intensive project support. This finding suggests that providing support to health centers for outreach activities (Leuk Dek, Phase 2) achieved equal or better results than a more expensive mobile team approach (Kean Svay, Phase 1). - The results achieved in Kean Svay AD in Phase 1 were not always sustained in Phase 2 when the intensive support to this area decreased—a concern for the sustainability of the project. - More intensive support to outreach and community mobilization in Leuk Dek produced better results for some but not all interventions than in Lovea Em. - Direct support to the OD in Lovea Em achieved better results than indirect support to the Kampong Thom OD through the ADPs, two implementation areas with otherwise limited project support. Following on its success in the Kean Svay AD, the KSCSP was able to further build the capacity of the Kean Svay OD to implement child survival activities in other areas. During Phase 2 of KSCSP, all local project partners were systematically involved and participated actively. The capacity of the Kean Svay and Kampong Thom OD and HC staff improved in terms of knowledge, clinical and managerial practices, behaviors towards to the clients and support systems. At the community level in Leuk Dek, VHVs and community leaders gained ownership of the KSCSP-supported activities and will continue their implementation. The ADPs in Leuk Dek and in Kampong Thom have integrated some child health and survival activities, approaches and tools into their plans. Below are a few specific recommendations to WV. 1. WVC should investigate and document the nature and reasons for : · The success of the KSCSP regarding selected interventions like birth spacing, postpartum vitamin A, and exclusive breastfeeding. · The persistent high use of anti-diarrhea medication in the project areas. · The decline in the coverage of vitamin A supplementation among children 6-23 months observed in Lovea Em. 2. WVC, WVAPRO and WVUS should: · Complete the analysis of the cost-effectiveness of the various models of implementation of child survival proposed in the DIP and initiated during the FE. · Review the status of the CBDDS in Kean Svay before deciding how to turn it over at the end of project. · Assess of the monitoring tools introduced during the KSCSP to draw conclusions about their use within World Vision and other organizations working in health and child survival. 5 B. Assessment of Results and Impact 1. Results: Summary Charts “The KSCSP extension project’s strategic objective is to assist the Kean Svay OD, private sector and community partners to accomplish, sustain, document and replicate promising practices to reduce infant and child mortality and morbidity through an innovative mainstreaming child survival and reproductive health improvement project in the primary impact area (namely, Leuk Dek District, where WV Cambodia’s ADP is located) of the Kean Svay Operational District (OD) in Kandal Province in Cambodia over a three year period [DIP page 50].” This strategic objective is achieved. Table 1, adapted from the DIP (pages 50-51), provides an overview of the Intermediate Results and sub Results of Phase 2 of the KSCSP extension project. Overall, all results are achieved except SR # 2.2. Table 1 Achievements of Intermediate Results and sub Results of the KSCSP Intermediate Results (IR) Achieved Sub-results (SR) Achieved IR # 1 - Coverage: Increased use of high￾impact child survival and reproductive health services by the target population (Leuk Dek) YES SR # 1.1: Increased Coverage of Essential Care of the Sick Child SR # 1.2: Increased Coverage of Immunization SR # 1.3: Prevention of Malnutrition/Vitamin A and Iodine Deficiency SR # 1.4: Improved Reproductive Health Coverage YES YES YES YES IR # 2 - Quality: Increased Quality of Case Management of Childhood Illness in Health Centers (all areas) YES SR.# 2.1 Strengthened Health Center Case Management of Childhood Illnesses SR.#.2.2 Improved Private Practitioners’ Case Management of Childhood Illnesses YES NO IR # 3 -Capacity Building: Enhanced capabilities of individuals, families and communities to protect and provide for their own health (Leuk Dek) YES SR # 3.1: Improved Caretaker and Family Practices SR # 3.2: Capacity building of community institutions SR # 3.3: Capacity of WV ADP staff SR # 3.4: Capacity of WV HQ staff YES YES YES YES IR #4 - Sustainability: Enhanced sustainability of child survival and reproductive health services and support systems (Leuk Dek) YES SR # 4.1: Sustainability initiatives in place SR # 4.2: Increased community participation SR.# 4.3: Increased capacity of community organizations to sustain long-term viability of health development processes/impact YES YES YES 6 Table 2 presents the values of the key indicators of the KSCSP found in the three KPC surveys conducted in Leuk Dek, the direct impact area, at baseline (BL, November 2000) and for the midterm (MTE, January 2002) and final evaluations (FE, August 2003). These values are presented above the targets set in the DIP (pp. 61-65) for FY01, FY02, and FY03, respectively, with a mention of the end-of-project achievement. Table 2 Achievements of end-of-project objectives in Leuk Dek Results / Targets # Key Indicator BL MTE FE Objective Achieved 1. Immunization 1 % of children 12-23 months who had been fully immunized 71% 97% 98% before age 12 months ( have received card-documented doses of BCG, OPV3, DTP3 and measles vaccines before age 12 months) 75% 80% 85% YES 2 % of mothers would received two card-documented doses of 26% 87% 99% Tetanus Toxoid before the birth of their youngest child 0-11 months 40% 55% 75% YES 2. Nutrition 3 % of infants 0-4months who were exclusively breastfed in the 7% 28% 43% last 24 hours 20% 60% 90% YES 2 4 % of children 6-23 months who received a card-documented 26% 97% 92% dose of vitamin A within the last six months 40% 60% 80% YES 5 % of mothers of infants 0-11 months who received a card- 2% 87% 98% documented dose of vitamin A within 2 months of delivery 30% 60% 80% YES 22% 9% 66%1 6 % of families with children 0-23 months who reported use of iodized salt 25% 35% 45% YES 3. IMCI 7 % of children 0-23 months with cough and difficult or rapid 66% 77% 100% breathing in the past 2 weeks were brought to a trained health provider 70% 75% 80% YES 8 % of children 0-23 months with diarrhea in the past two weeks 53% 93% 100% who were treated with ORT 60% 70% 80% YES 9 % of children 0-23 months with diarrhea /sick3 in the past 2 55% - 81% weeks who were given the same amount or more food during their diarrhea/sickness episode 50% 65% 80% YES 10 % of children 0-23 months with diarrhea/sick3 in the past 2 75% - 83% weeks who were given the same amount or more fluids other than breast milk4 during their diarrhea/sickness episode 60% 75% 80% YES 11 % of caretakers of children 0-23 months who can name at 21% 68% 97% least one danger sign of DHF (cold extremities, bleeding, abdominal pain, weakness) 30% 40% 60% YES 4. Birth Spacing 12 % of women with children 0-23 months, who are not pregnant 6% 64% 51% and desire no more children in the next two years or are not sure, who are using a modern contraceptive method 10% 20% 30% YES 1 The indicator in August 2003 was the positive test of the presence of iodine in the salt used in the household. Although more reliable, it cannot be compared to the self reported use of iodized salts. 2 Strictly speaking, this objective as stated in the DIP was not achieved. However, the value of 90% was clearly unrealistic and 43% is quite a remarkable achievement. 3 Prior to 2003, all surveys asked this question only in relation to diarrhea in the last 2 weeks. In 2003, following the IMCI approach, the question was asked in relation to any sickness. 4 The 2000 (BL) survey specifically asked about breastfeeding while the 2003 survey did not. 7 Table 3 lists the capacity building and sustainability objectives defined for the KSCSP during the First Annual Review in September 2001 with a mention on the achievement at the end of the project. Table 3 Capacity building and sustainability objectives Objective Objective Achieved World Vision Cambodia 1. Replicate and integrate essential KSCSP interventions into WVC ADPs YES 2. Increase competencies of WVC health and other program staff to enable successful integration of CS interventions into ADPs YES 3. Strengthen knowledge and skills of WVC staff in project design, planning, monitoring and evaluation YES 4. Enhance grant management skills, systems and capacities throughout WVC YES Ministry of Health 1. 100 % of Leuk Dek health center outreach activities managed solely by MoH (Operational District/Health Center). YES 2. Increased quality of Case Management of Health Center Staff in LD, LE, KS, and KgT. YES 3. To strengthen the technical and managerial capacity of KS and KgT OD to conduct quality supervision of Child Survival Interventions. YES 4. Health Center Management Committee (HCMC) functioning in two Health Centers in Leuk Dek. YES 5. Increased organizational capacity of Kean Svay Operational District to manage and sustain child health programs. YES Community Mobilization 1. Health Center Management Committee and Feedback Committee functioning in two HCs in Leuk Dek. YES 2. Families with children 0 – 23 months are practicing selected C/IMCI family practices. (Home care, men’s participation and health seeking behavior.) YES 3. Increased capacity of nine of the forty-five VDCs in Leuk Dek, and the majority in Kampong Thom to design, manage, and implement child health activities. YES Sustainability 1. Integrate CS interventions into WV ADP design and activities YES 2. Phased transfer of program responsibility to OD staff and community YES 3. Build MoH capacity by equipping and training staff YES 4. Strengthen community capacity to identify and respond to health needs YES 5. Strengthen equitable cost-recovery mechanis ms and other means of financial support YES 6. Explore opportunities to involve key private sector providers in quality CS services NO 7. Use organizational and community structures to advocate for healthy public policy YES 8 2. Results: Technical Approach a. Project Overview The Kean Svay Child Survival (Follow On) Project (KSCSP1 Phase 2; FY 2001-2003, plus 5 months no-cost extension) aimed at applying lessons learned from the previous Kean Svay Child Survival Project (KSCSP Phase 1; FY 1996-2000) to other areas in Cambodia and at building the related capacity of the Ministry of Health (MOH) and World Vision Cambodia (WVC). A particular interest was the development of a model of integration of child survival into the World Vision’s Area Development Programs2 (ADP) in Cambodia. The technical interventions of the two phases of the KSCSP are immunization, nutrition, case management of childhood diseases and birth spacing. The main objectives related to these interventions are presented in Table 2. The local partners of WVC in the implementation of the KSCSP are the government health services (the Operational District—OD and Health Center—HC teams) and the communities (primarily the Village Health Volunteers—VHVs and the Health Center Management Committee—HCMC members) in the Kean Svay and Kampong Thom ODs. The KSCSP build the capacity of its local partners through the provision of extensive training to the OD, HC, and VHSG, through the establishment of systems to improve health services performance (supervision, training and follow-up; information system, community mobilization, etc.). Within WVC, the ADP staff in Leuk Dek and in the three ADs in Kampong Thom and the HIV/AIDS and Health Program (HAHP) and management staff provided support to the KSCSP staff; these WVC staff members directly or indirectly benefited from training and other capacity building opportunities provided by the KSCSP. The KSCSP was implemented in the three ADs of the Kean Svay OD and in three of the five ADs of the Kampong Thom OD,3 thereby defining four implementation areas. Table 4 presents selected elements of the demographic and health services profile of these four implementation areas. ______________________ 1 In this report, the term KSCSP refers to the Phase 2 of the project (sometimes referred to as the “KSCSP Follow On” or the “KSCSP Extension” in other documents). 2 ADPs are World Vision’s programmatic unit of integrated development based on a 10-15 year commitment to improve child well being and reduce poverty in defined geographical areas. 3 The three ADs in which the KSCSP is implemented, considered as one implementation area in the discussion below, are Prasath Sambo, Prasath Balang, and Sandan. 9 Table 4 Demographic and health services profile of the four implementation areas of KSCSP Kean Svay OD Demographic and health services profile Kean Svay Leuk Dek Lovea Em Kampong Thom OD (3 ADs1 ) Total population2 118985 52779 71133 119858 Children < 1 4045 1794 2419 4075 Children < 5 20227 8972 12093 20375 Women 15-49 26177 11611 15649 26369 Communes 9 7 15 21 Villages 36 24 43 205 Health Centers 7 4 6 12 VHV3 64 50 65 78 1 Prasath Sambo, Prasath Balang, and Sandan. 2 Sources: 2003 Population Statistics of Kean Svay OD and Kampong Thom Provincial Health Department. 3 Source: DIP, except for Leuk Dek and Kean Svay for which KSCSP maintains lists of the VHVs that it supports. In each project implementation area, the KSCSP provided the following types support to the three levels of the health system. Level of the Health System Type of Support 1. Operational District - Health Management system - Training of trainers - Planning, monitoring and evaluation 2. Health Center - Technical and management training - Supportive supervision from OD - Mobile teams or outreach activities 3. Community - Training of VHVs and other community members - Follow-up and incentives to VHVs - Creation of HCMC and training of its members Table 5 presents rough estimates of the relative support provided at each level of the health system in each KSCSP implementation area, and during each phase of the KSCSP in the Kean Svay AD. These different types of support characterize five implementation models. The more intensive model was implemented in Kean Svay AD during Phase 1 of the KSCSP and included mobile teams and a direct involvement of the KSCSP staff in service delivery. During Phase 2, no support was provided to mobile teams in Kean Svay AD and the HCs continued outreach and community health activities. The most intensive model during Phase 2 of the KSCSP was implemented in Leuk Dek AD, the direct impact area of the project, where support was provided at the three levels of the health system. In Lovea Em AD, the KSCSP provided support to the health centers but not to the outreach or community health activities. The project support to the Kean Svay OD benefited its 3 ADs. Finally, in Kampong Thom the KSCSP only provided limited support to the OD and the HCs. 10 Table 5 also presents rough estimates of the relative technical and financial support provided to the ADP in two of the KSCSP implementation areas. The KSCSP provided more intensive support to the ADP in Leuk Dek than to those in Kampong Thom. In Leuk Dek, however, the ADP began in 2000 and is still at an early stage of its development. In Kampong Thom, the three ADPs are already more established and include other health programs involving VHVs and HC staff. The implementation of the KSCSP within an ADP area was expected to facilitate the process and increase the effectiveness and sustainability of the project. It was also expected to provide the opportunity to develop the best strategies to integrate child survival activities in all ADPS in Cambodia. Table 5 Relative support provided by KSCSP to its five implementation models Implementation model Kean Svay OD Kean Svay AD Type of support Phase 1 (1) Phase 2 (2) Leuk Dek AD (3) Lovea Em AD (4) Kampong Thom OD (3 ADs1 ) (5) Health System Operational District +++ +++ +++ +++ + Health Centers +++ ++ +++ + + Mobile team +++ - - - - Outreach services - + +++ - - Community +++ + +++ - - Area Development Program Technical - - +++ - ++ Financial - - +++ - + Note: Intensity of support: - none; + low; ++ medium; +++ high 1 Prasath Sambo, Prasath Balang, and Sandan. The discussion of results by technical approaches in the following sections presents data from the KPC and other surveys conducted in four implementation areas. During Phase 1 of KSCSP in Kean Svay AD, surveys were conducted at baseline in April and October 1996,4 at midterm in July 1998 and at the end in June 2000. This last survey is considered as the baseline of the KSCSP Phase 2 in Kean Svay AD, and was followed by a final survey in August 2003. In November 2000, one distinct KPC survey was conducted in the two other implementation areas in Kean Svay OD, Leuk Dek and Lovea Em. In January 2002, a midterm survey was conducted only in Leuk Dek, the direct impact area of the KSCSP Phase 2. In August 2003, end-of-project surveys were ______________________ 4 The April 1996 survey was conducted when WVC began child survival activities in the area before the beginning of the USAID-funded project that started in FY1996. Another survey was conducted in October 1996 and constitutes the actual baseline of the project. 11 conducted in the three ADs of the Kean Svay OD. One KPC survey was conducted in November 2000 in Kasach Kandal as a control area but was not repeated at the end of the project. In Kampong Thom, a few child survival indicators equivalent to those of the KPC surveys were integrated in the household surveys conducted by the ADP in two ADs, Prasat Ballang and Prasat Sambo, in May 2000 and in May 2003. Most of the other data discussed in the following sections come from Leuk Dek except for supervision data from the KS and KT ODs. The methods and tools of these data are discussed in section C.g. b. Immunization The two graphs below show that in Leuk Dek the KSCSP objective5 for immunization of infants against childhood diseases was already met at midterm and that this result was maintained until the end of the project. The increase in the percentage of children 12-23 months completely immunized by age one was similar in Leuk Dek and Lovea Em, but it slightly decreased in Kean Svay during phase 2 of the KSCSP.6 In Kampong Thom, the percentage of children 12-23 months who received DPT37 rapidly increased from 12% to 56% between May 2000 and May 2003 in one AD and remained stable around 20% in another. The increase in the proportion of mothers with a child under two who received at least 2 doses of Tetanus Toxoid (TT2) during their last pregnancy increased from 26% to 80% in Leuk Dek and nearly as much from 3% to 58% in Lovea Em, a large increase similar to what had been observed in Kean Svay during Phase 1 of the KSCSP. However, this proportion decreased from 96% to 62% in Kean Svay during Phase 2. ______________________ 5 The DIP set intervention-specific objectives for the Leuk Dek AD only. Targets for key indicators can be seen in Table 2. 6 p = 4.7% 7 This indicator is used as a proxy for the percentage of complete immunization, which is not available in the KPC surveys in Kampong Thom. The coverage of DPT3 is always equal or higher than the coverage of complete immunization. 12 Figure 1 Percentage of children 12-23 months completely immunized by age one Figure 2 Percentage of mothers of children 0-11 months who received at least 2 doses of Tetanus Toxoid during last pregnancy 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em Kg Thom PB Kg Thom PS Note: In Kampong Thom, percent of children 12-23 months who received DPT3. 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Kean Svay Leuk Dek Lovea Em The KSCSP monthly reports on immunization activities show that in FY 2003 a total of 1024 children under 12 months were completely immunized in Leuk Dek. Using population figures from Kean Svay OD (see Table 4), this number corresponds to a coverage of 57%. Assuming that the sampling and reporting of the immunization status in the KPC survey is accurate, the discrepancy between the two estimates suggests a major overestimation of the population figures used for Leuk Dek, an underreporting of the number of immunizations by the HCs, or that a large number of children are getting immunized outside of the 4 HCs. In FY 01, FY 02 and FY 03, the VHVs in Leuk Dek reported 8, 2, and 0 cases of measles, respectively (see section C.g(iv)). This finding is consistent with the sharp increase in immunization coverage at the beginning of the project implementation. The HC and OD staff interviewed during the field visits attributed the success of the immunization intervention to the strong support from the OD, the development and implementation of clear Plans of Action, and the intense participation of the communities and the HC staff. They also recognized important constraints such as the large area to be covered, the occurrence of natural disaster such as flood, the side effects of vaccines, and the poor funding of the immunization services by the government. They also recognized that after the mobile teams were discontinued in Kean Svay AD at the end of Phase 1 of 13 the KSCSP, there had not been enough funding from the government and regular supervision. Also, the practice of attaching the mother’s card to the child’s card (the yellow card) was not followed in Kean Svay AD as well as in Leuk Dek, and this may explain at least part of the decline of the TT2 coverage. c. Nutrition The two graphs below show the improvement in breastfeeding practices achieved in the four implementation areas of the KSCSP. The percentage of newborn breastfed during the first hour after birth increased from about 5% at baseline to 89% in Leuk Dek, 29% in Lovea Em, but it decreased to 41% in Kean Svay after having reached 66% at the end of Phase 1. In Kampong Thom, this percentage increased from the same levels as in Kean Svay at baseline to 51% in Prasat Ballang and to 22% in Prasat Sambo. There was no target for this indicator in the DIP. The proportion of infants 0-3 months exclusively breastfed increased from less than 10% at baseline to 43% in Leuk Dek and 20% in Lovea Em, and it kept increasing to 16% in Kean Svay after having reached 10% at the end of Phase 1. The end-of-project target for this indicator was unrealistically set at 90%, and the achievement in Leuk Dek of 43% is quite remarkable. Figure 3 Percentage of children 0-23 months who were breastfed within one hour after birth Figure 4 Percentage of children 0-3 months exclusively breastfed during the last 24 hours 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em Kg Thom PB Kg Thom PS 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em Figure 5 shows that the KSCSP objectives for vitamin A supplementation of children under five were achieved in Leuk Dek, with an increase from 26% to 91% in the coverage of this intervention among children 6-23 months. In Kean Svay AD this 14 coverage remained stable around 40% while it decreased from 59% to 34% in Lovea Em. The FE Team8 discussion did not clarify the reasons for this striking difference which could involve an issue of recording (in Lovea Em, 98% of children have an immunization card and the vitamin A supplementation coverage based on mothers’ recall in 43%) or be related in programmatic differences (role of VHVs and implementation of campaigns). The percentage of mothers of children under two who received one dose of vitamin A post partum increased dramatically from nearly 0% to 98% in Leuk Dek and to 74% in Lovea Em, but it decreased from 91% to 72% in Kean Svay AD. Figure 5 Percentage of children 6-23 months who received one dose of vitamin A during the last six months Figure 6 Percentage of mothers of children 0-23 months who received 1 dose of vitamin A within four weeks post partum 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em The KSCSP collected data on the number of Vitamin A capsules distributed by VHVs to postpartum mothers in Leuk Dek. These data allow the calculation of the coverage of this intervention, which increased from 57% in FY01 to 84% in FY03. The discrepancy with the August 2003 KPC reports of 98% coverage may result from overestimated population figures used in the CBDDS calculations, as noted in the previous section. The VHVs have been successful in educating villagers on the need for iodine. They have continued selling iodized salt in Kean Svay and begun in Leuk Dek in March 2002. In 2003, the VHV sold an average of 490 and 2075 kg of iodized salt in Kean Svay and in Leuk Dek, respectively.9 The higher amount of salt sold in Leuk Dek where the ______________________ 8 See Appendix B for the FE methodology and composition of the team. 9 See Monthly Reports, Compiled for FY 2003, WVC-KSCSP. 15 population is smaller but otherwise similar to that of Kean Svay shows the importance of a close follow up of the VHVs for the success of this intervention. In August 2003, the proportion of households with salt that tested positive for iodine at the time of the survey was 66% in Leuk Dek, 27% in Kean Svay and 23% in Lovea Em.10 In Kampong Thom, this indicator was 52% and 43% in the two ADs were a KPC was conducted in May 2003. Following a recommendation from the DIP review, the KSCSP conducted anthropometric measurements during the January 2002 KPC survey, and found that 27% of children under two were underweighted (< -2SD weight-for-age) and 3.7% of them severely (<-3SD). The August 2003 KPC survey did not include anthropometric measurements and the FE Team members attempted to check the nutritional status of children under five seen in HCs during the field visits in KS OD. Among 400 IMCI cards reviewed in the four HCs visited, the proportion of children classified as severely malnourished or pale/underweighted was 2.5% (0.7% for children under 2 years and 6.6% for children 2-5 years). Assuming that the nutritional assessment reported on the IMCI card is equivalent to a classification into moderate or severe malnutrition based on weight-for-age indicator, this finding suggests that the protein-energetic nutritional status of children under five brought to the HCs in KS OD is within the normal range. The OD and HC staff interviews during the field visits and the FE Team discussion established that the success of the nutrition interventions was due to the training provided by the KSCSP to the HC and OD staff, TBAs, and VHVs, and to the good participation of these trainees in the nutrition activities. The quality of the counseling about breastfeeding provided to mothers by HC staff and VHVs, but particularly by midwives and TBAs at the time of delivery, seems to have been very important in increasing the quality of the practices of breastfeeding. Subsidies for the iodized salt seem to have helped families buying and using it. The FE Team also identified that the habits, beliefs and poor knowledge of mothers are constraints to the improvement in nutritional practices. Mothers tend to breastfeed for a limited period because they go back to work in the field or elsewhere. In Kean Svay, the supervision from the OD has been insufficient and mothers do not keep cards; this can explain at least part of the decline in post partum vitamin A supplementation in that area. In Lovea Em, the vitamin A supplementation is also often given without proper documentation and the government registers are different than those used by the KSCSP; this may also explain the decline seen in the KPC surveys. ______________________ 10 This indicator introduced for the first time in the August 2003 surveys is probably more reliable than the self-reported use of iodized salts of the previous surveys. The question on self-reported use of iodized salts was not maintained in the August 2003 survey and no trend can be established. 16 d. IMCI The two graphs below show the improvement in the case management of diarrhea achieved in the KS OD. The percentage of children 0-23 months with diarrhea in the last two weeks who received ORT increased from 53% to 100% in Leuk Dek and from 29% to 85% in Lovea Em. It also slightly increased from 81% to 92% in Kean Svay during Phase 2 of the project. Figure 7 Percentage of children 0-23 months with diarrhea the last two weeks who received ORT Figure 8 Percentage of children 0-23 months with diarrhea in the last two weeks who received an antidiarrheal medicine or antibiotic 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em The KSCSP staff has been collecting the number of diarrhea cases seen in the ORT corners of the HCs in the three ADs of the Kean Svay OD. These data are reported in the monthly KSCSP reports and compiled in summary tables on an annual basis.11 These data have recently been compiled and trends by months and by AD have been plotted that show a two or three times larger number of reported cases in Lovea Em than in Leuk Dek or Kean Svay throughout the reporting period. There is no analysis of completeness of the reports from the health centers and no interpretation of this finding so far. One objective of the KSCSP was to reduce the percent of children 0-23 months with diarrhea in the last two weeks who received anti-diarrhea medicines or antibiotics. Figure 8 above shows that the persistent high rate of 54% found at the end of the Phase 1 of KSCSP in Kean Svay has increased to 63% in August 2003. These indicators have ______________________ 11 See Monthly reports for FY01, FY02, and FY03. 17 otherwise decreased from 58% to 41% in Leuk Dek and from 73% to 56% in Lovea Em; both levels remaining far from the target of 10% adopted in the DIP. The sharp drop followed by an equivalent increase in this indicator in Kean Svay in October 1996 and in Leuk Dek in January 2002 is unexplained and may be due to the poor definition of the type of antidiarrhea medicine is used. Figure 9 shows that the percentage of children 0-23 months with cough and difficulty breathing in the last two weeks who were brought to a public health facility reached 94% or more in the three project areas in the Kean Svay OD. The lower value of this indicator observed at baseline in Leuk Dek is not explained. On average, the percentage of mothers of children 0-23 months who got advice or got treatment from a public health facility (PHF)/provider for diarrhea, ARI and DHF was slightly higher in Leuk Dek (88%) than in Kean Svay (80%) and Lovea Em (72%). In Leuk Dek, however, the CBDDS shows that only 38% of children under 5 who were sick and referred to a public health facility by a VHV actually sought care in a PHF. Among them, only 64% were treated, that is, only 24% of those referred were treated. Figure 9 Percentage of children 0-23 months with cough and difficulty breathing the last two weeks who were brought to a trained provider Figure 10 Percentage of caretakers of children 0-23 months who can name at least one danger sign of DHF (cold extremities, bleeding, abdominal pain, weakness) 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em One objective of the KSCSP was to increase the capacity mother of children 0-23 months to recognize signs of pneumonia. The table below shows that the percent of mothers citing fast breathing or chest in-drawing as a sign that calls for immediate referral or treatment increased in Leuk Dek and Lovea Em but that the high levels reached in Kean 18 Svay at the end of Phase 1 decreased. The target adopted in the DIP of 90% of mothers able to recognize 2 or more danger signs is not achieved in any of the implementation area. Leuk Dek Lovea Em Kean Svay BL MTE FE BL MTE FE BL MTE FE Fast breathing 31 54 59 11 - 35 - - 51 Chest in-drawing 9 16 29 5 - 13 601 - 23 1 See Final Evaluation of the KSCSP Phase 1 Similarly, part of the case management of DHF is early diagnosis. Figure 10 above shows that the percentage of caretakers of children 0-23 months who can name at least one danger sign of DHF (cold extremities, bleeding, abdominal pain, and weakness) increased dramatically in Phase 1 in Kean Svay, where it kept increasing in Phase 2, and also by a similar amount in Leuk Dek and Lovea Em. Surveillance is also important in the control of DHF, and the CBDDS has contributed to the early detection and response as well as the afterwards documentation of a Dengue Fever outbreak in FY03. Prior to 2003, mothers of children 0-23 months who had diarrhea in the last two weeks were asked whether they gave more, the same amount or less food and fluids other than breast milk to their child. In 2003, following the IMCI approach, those questions were asked to mothers of children 0-23 months who had any illness in the last weeks. In the two graphs below, it is assumed that these indicators do not depend much on the type of illness and can be compared across surveys. For the food indicator, Figure 11 shows an increase from about 50% to 97% in Leuk Dek and to 86% in Lovea Em during the Phase 2 of KSCSP. It also kept increasing in Kean Svay, after a large increase during Phase 1 similar to that observed in Phase 2 in Leuk Dek and Lovea Em. In Kampong Thom, the indicator, which remained related to diarrhea only, increased moderately during the KSCSP implementation period starting from a relatively high level that presumably had been reached through the other health activities conducted in this area. For the fluid indicator, Figure 12 shows quite high levels at baseline and substantial increases to reach more than 90% in all three ADs of the Kean Svay OD at the end of the project. 19 Figure 11 Percentage of children 0-23 months with diarrhea/sick in the last two weeks who received the same amount or more food Figure 12 Percentage of children 0-23 months with diarrhea/sick in the last two weeks who received the same amount or more fluids other than breast milk 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em Kg Thom PB Kg Thom PS 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em The data from standardized clinical observations conducted by the OD staff during supervision visits, discussed in section 3.c (iv), suggest that the quality of case￾management is lower in the health centers in Kampong Thom than in the Kean Svay OD. The graph below shows the average percentage of selected tasks12 related to the case management of diarrhea, ARI and DHF that were conducted according to standards during a period of several months in 2003 in the four KSCSP implementation areas. There were no data on the trends of these indicators available during the FE. ______________________ 12 Thirteen among the 43 tasks in the current checklist were selected for this analysis. 20 Figure 13 Quality of case management of childhood illness in Kean Svay and Kampong Thom ODs 0% 20% 40% 60% 80% 100% Kean Svay Leuk Dek Lovea Em Kg Thom e. Birth Spacing Figure 14 below shows the impressive increase observed in Leuk Dek in the prevalence of modern contraceptive among mothers of children under 2 who are not pregnant and do not desire any children within two years. The value of this indicator increased from 6% at baseline to 64% at midterm, 14 months later, and then decreased slightly to 51% at the end of the project.13 In Lovea Em, the contraceptive prevalence increased from 11% at baseline to 24% at the end of the project. In Kean Svay, the contraceptive prevalence that had reached 47% at the end of the Phase 1 decreased to 19% at the end of Phase 2. Figure 15 shows that the overall decrease in contraceptive prevalence in Leuk Dek between January 2002 and August 2003 is accompanied by a diversification of the method mix. The decrease in the use of injections (10 percent points) and pills (8 percent points) is partially compensated by an increase in the use of condoms (4 percent points) and tubal ligation (1 percent point). The reason for the overall decrease in the above indicator of contraceptive prevalence is otherwise unclear. The contraceptive prevalence among all women with a child under two actually decreased even further (a 33% instead of 20% decrease) since the proportion of mothers who were pregnant or who desire another child increased from 13% to 27% between midterm and the end-of-project KPC surveys; the reason for this increase in the context birth spacing promotion is unclear. ______________________ 13 p = .004. 21 Figure 14 Percentage of mothers of children 0-23 months, not pregnant and who do not desire a child within two years, who are using a modern method of contraception Figure 15 Method-specific contraceptive prevalence among mothers of children 0- 23 months, not pregnant and who do not desire a child within two years in Leuk Dek 0% 20% 40% 60% 80% 100% Jan-96 Jan-98 Jan-00 Jan-02 Jan-04 Leuk Dek Kean Svay Lovea Em 0% 10% 20% 30% 40% Inj. Pill Cond. IUD TL Jan-02 Aug-03 The FE Team attributed the success of the birth spacing intervention to the availability of various contraceptive methods, the provision of services at the community level (counseling and distribution of pills and condoms), the provision of services free of charges for poor women, the training of WVC and HC staff, the application of the COPE approach, the use of a special register and information system,14 and the good cooperation between HC and WV, OD, and the community. Among constraints is the persistence of rumors in the community, the existence of side effects with some contraceptive methods, the reduced activities of KSCSP Phase 2 at the community level resulting in the need for clients to go to the HCs. In Kean Svay AD, the FE Team identified that the number of new clients decreased as compared to that achieved during Phase 1 and that the follow up of clients was insufficient. ______________________ 14 Introduced in 2001 with support of UNFPA. 22 3. Results: Cross-cutting approaches a. Community mobilization The KSCSP approach to community mobilization was primarily to create, train and strengthen VHVs and HCMCs; regularly meet with VHVs and HCMC members; and strengthen TBAs, traditional healers, and private drug sellers. According to the design of the KSCSP, this community mobilization was primarily done in Leuk Dek where 50 VHVs were trained nearly every month, regular meetings were held with 42 HCMCs members from the four HCs, and 65 Village Development Committee (VDC) members from 13 villages were trained in planning for development. In August through November 2003, the KSCSP contracted Dr Chea Samnang, Director of Rural Health Care Department, Ministry of Rural Development, to conduct an assessment of the performance of the VHVs in the project area. 15 This study provides valuable expert insight on their selection, recruitment, training, incentives and various roles, and on their relationships with the HCs, HCMCs and VDVs. Besides confirming the critical role of VHVs in achieving of the results of KSCSP, the study emphasized the importance of ensuring that the HC and OD staff continues supporting VHVs at the end of the project. Overall, the FE Team found that the community mobilization approach of the KSCSP, building the capacity of individual community members and establishing functional community-based structures to support them and make the link with HCs, was very effective. They found that at the end of the project the three community mobilization objectives listed in Table 3 were met. Among lessons learned for future activities are the following: · The selection of VHVs should be done according to the national guidelines 16 and use criteria such as their willingness to engage in such work, be at least 25 years old, know how to read and write, be respected by the community, and be not too busy with other matters. · Discussion and problem solving approaches, using PHD and OD staff as facilitators, seem to be the best way to train VHVs. · Regular post-training follow up and feedback are important to ensure proper motivation and guidance. During the FE group interviews with mothers, village and community leaders, VHVs and HC and OD staff, it clearly appeared that there is a demand for the continuation KSCSP activities at the community level, particularly the training of VHVs, TBAs and traditional ______________________ 15 See Assessment of Village Health Volunteer performance, KSCSP, August to November 2003. Report prepared by Dr. Chea Samnang, Department of Rural Health Care. Ministry of Rural Development. KSCSP, 2003. 16 The Government (MOH and MDR) just finalized and issued a new guideline for Village Health Support Groups. 23 healers; the support of the activities of VHVs; and the support and strengthening of the HCMC. During the field visits interviews, the FE Team identified the following sustainability plans and intentions: · The HCs will continue training and strengthening of VHVs and HCMC through regular monthly meetings, including provision of snacks and transport allowances. In addition, the HCs will not charge the VHVs and their family members for services and treatment. · The ODs will provide technical support to the HCMC and encourage them to have regular meetings. · The HCMC and VHV will maintain their activities with the HC even though they receive less support from WVC. These sustainability plans appear realistic. HC chiefs interviewed during field visits or participating in the FE Team discussions all recognized that they could cover the costs of the VHV meetings using their resources from their cost-recovery schemes. As the KSCSP staff had more regular meetings with VHVs than the OD and HC staff,17 however, the OD and HC will have to change their monitoring schedule to meet with VHVs more frequently. b. Communication for Behavior Change Overall, the KSCSP approaches to Behavior Change Communication (BCC) included health education in groups and on one to one basis; demonstrations; health education campaigns; the use of leaflets, posters, banners, cassette players; and health education regarding the illness of the child when the mother or caretaker brings a sick child to the HC or the VHV. These approaches were applied in the context of the community mobilization discussed in the previous section. In 2003, the BEHAVE framework was used to “retro-fit” the BCC plan developed with the external BCC consultant in December 2001. 18 The KSCSP staff developed strategies for the two specific behaviors that had been selected with the consultant, early initiation of breastfeeding and condom use. Parts of this framework are complete and well developed while others need further work, particularly the aspects that relate to the newly identified supporting groups (as opposed to the priority groups). The CBDDS includes data on the number of women and men who received health education in Leuk Dek. Figure 16 shows a steady increase in these activities between FY01 to FY03. On average in FY03, each women of reproductive year received about one health education session per year—about twice as much as what men received. ______________________ 17 See consultant report Assessment of Village Health Volunteer performance. 18 See annex of First Annual Review report for BCC consultant report and related comments in the MTE report. 24 However, it is likely that pregnant women and mother of young children received more frequent inputs. Figure 16 Number of men and women who received health education from VHVs in Leuk Dek, FY01 to FY03 0 5000 10000 15000 20000 FY01 FY02 FY03 men women The KSCSP has conducted regular assessments of the knowledge of VHVs on control of diarrheal diseases (CDD), dengue hemorrhagic Fever (DHF), acute respiratory infections (ARI), tetanus, EPI, the 6 immunization preventable childhood diseases, iodine deficiency disorders (IDD), vitamin A, and birth spacing.19 The project staff gave a three￾level score (Good/Fair/Weak) for each topic on the basis of the answers provided to the related questions. Between 10 and 20 VHVs were interviewed each month on various topics. Figure 17 shows that on average, the percentage of VHVs who scored Good increased from 55% to 90% within six months of project implementation, and then remained stable at that level until the end of the project. The individual graphs of the score for each topic are presented in Appendix I . Figure 18 shows the similar pattern obtained with the data on knowledge about CDD, DHF and ARI from monthly exit interviews of 50 to 100 mothers attending health centers, except that a plateau is reached more slowly and at a lower level, particularly for ARI. These findings are consistent with the program design whereby VHVs with higher literacy levels than average mothers in the community receive regular training from the KSCSP staff and transmit their knowledge to mothers. ______________________ 19 See KSCSP monthly reports. 25 Figure 17 Average knowledge score on various topics for VHVs in Leuk Dek Figure 18 Knowledge score on CDD, DHF and ARI among mothers attending health centers in Leuk Dek 0% 20% 40% 60% 80% 100% Oct-01 Apr-02 Oct-02 Apr-03 Oct-03 Note: Score for individual knowledge topics are provided in Appendix I 0% 20% 40% 60% 80% 100% Oct-01 Apr-02 Oct-02 Apr-03 Oct-03 CDD DHF ARI The various KPC results presented in section B.2 are evidence of the changes in health behavior and care seeking achieved by the KSCSP, and that the related project objectives were met. The increase in early initiation of breastfeeding (see Figure 3) and in condom use (see Figure 15) reflects the more intensive efforts placed on bringing change in those two behaviors as a result of the development of detailed behavior change strategies. The FE Team found that the factors that contributed to positive health behavior change in the communitywere: - Training of the VHVs, through whom changes in behavior actually took place - Continued intensive health education activities - Good relationship between HC and community - Monthly meetings of HCMC and VHV to review the behavior change activities - Use of leaflet and poster as a method of behavior change - Actual demonstrations and focus on the diseases at the time they occur - Provision of real examples to VHVs during training and monthly meetings - Selection of VHVs based on criteria such as qualification, knowledge and respect by the community For the project staff, the main challenges have been the difficult geographical accessibility and the frequent floods, the fact that it is difficult to meet parents for health education because they work in the field, and the low literacy of the population. To 26 overcome these challenges, the project staff also stayed in the villages to meet mothers and fathers, they continued work in areas not flooded when necessary, and they used pictorial/pictures for health education and demonstration. The FE Team found that to sustain the positive health behavior changes after the end of the project, health education in the community should continue. The HC staff will therefore use the VHV reports and train and monitor them regularly, and they will remind mothers about on health messages and practices during their visit to the HC. To sustain the use of iodized salt, they will ensure that it remains available in the community and ask support from the ADP with this respect. . These sustainability plans and intentions have been regularly discussed during meetings of the KSCSP, ADP and OD staff and are realistic. c. Capacity Building Approach (i) Strengthening WV US and WV Cambodia The KSCSP has been an opportunity for WVUS and WVC to learn about several aspects of child survival programming, among which the FE Team identified the following. · Structure and effectiveness of various models of implementation of child survival interventions. · Importance of fostering strong collaboration between community, VHV, HC, OD, PHD, MOH, and other Child Survival partners in Cambodia. · Importance of building capacity of HC staff to ensure sustainability · Importance of a good reporting system of project activities · Value of the COPE methodology and tools in developing relationships between HC staff and clients. · Identification of tools and methods most useful to integrate child survival interventions into ADPs (community registers, indicators, KPC surveys, etc). · Specific expertise in 30-cluster KPC surveys, LQAS, COPE, PLA, BEHAVE, IMCI and other tools and methods commonly used in child survival programs. · General expertise in design, implementation and evaluation of child survival projects (DIP preparation in Cambodia, DIP review in USA, MTE and FE) · Improved system and expertise in grant and financial management through training and application of USAID procedures. In November 2001, WVC held a series of workshops to develop a set of health indicators for WVC ADPs. These workshops were facilitated by the National Health Program staff and involved most of ADP managers and health coordinators, operation health coordinators and some operation managers (about 50 participants). In February 2002, WVC issued a paper specifying Core, Recommended and Optional indicators for each 27 ADP that included primarily indicators used in the KSCSP and other child survival programs.20 During the field visit interviews, the ADP managers recognized that the KSCSP helped: · Building good relationships between the ADP and the PHD, OD, and HCs. · Improving the ADP staff knowledge on quantitative surveys (KPC and LQAS) to quantify the achievements of health programs · Integrating health components into the ADP design documents. · Recruiting one health coordinator for the Leuk Dek ADP (probably one of the CSP staff) Several tools developed and implemented during the KSCSP have been used in other child survival projects by World Vision US and APRO. (ii) Strengthening local partner organizations The KSCSP has been systematically planning and implementing project activities with the OD and HC staff. This involved the DIP, MTE and FE; quarterly and monthly planning meetings; regular training and supervision opportunities. The FE Team found that this approach has been crucial in building the capacity of the local partners to implement child survival interventions. During the FE field visits interviews, the OD and HC staff consistently stated that their technical knowledge and practices related to IMCI, EPI, birth spacing, and case management of childhood diseases have increased as a result of the training provided by the project. The management training also has increased the managerial skills of the OD,21 HC and VDC staff. The COPE training and implementation helped HC’s staff to conduct self-assessments, solve problems and improve communication with the community. The KSCSP used an Organizational Capacity Assessment (OCA) tool with the OD teams of Kean Svay (in February 2002, August 2002 and November 2003) and in Kampong Thom (in August 2002 and November 2003). This participatory tool consists in a list of 53 indicators covering the areas of governance, management practices, human resources, financial resources, service delivery and sustainability. For each indicator, the users of the OCA give themselves a score between 1 and 6 depending on the perceived need for attention to the corresponding organizational features.22 ______________________ 20 See Health Status Indicators for World Vision Cambodia’s Area Development Programs. National Health Program, WVC, February 2002. 21 In 2003, the KSCSP supported 3 OD staff members to attend the 6-month Health Services Management course offered by the National Institute of Public Health/Cambodia . This OD staff then provided 3-day courses to HC staff. 22 1: Needs urgent attention and improvement, 2: Needs attention, 3: Needs improvement on fairly wide scale, but not major or urgent, 4: Needs improvement in limited aspects, but not major or urgent, 5: Room for some improvement, 6: No need for immediate improvement. 28 In Kean Svay OD, the average score increased from 4.6 (range 2-6) to 5.7 (range 5-6) between February 2002 and November 2003. The most important positive changes since February 2002 were in areas of employment and personnel practices, resource planning and allocation, human resource development, financial reporting, support from local institutions, and partnership in the development process. In Kg Thom OD, the average score increased from 4.8 (range 2-6) to 5.6 (range 2-6) between August 2002 and November 2003. Important positive changes included staff understanding of the OD goals, recognition of the communities as partners, regular surveys of needs and use of findings for planning, all areas of planning, application of new skills acquired through training, definition of priorities in collaboration with partners and monitoring and analysis of baseline data. Lowest scores were found for administrative procedures and operation manuals, financial inventory and control and procurement procedures. The OCA seems to have successfully helped the OD to identify weaknesses and improve its organizational capacity while documenting the process. (iii) Strengthening Health Facilities The OD and HC staff and the HCMC members recognized that the KSCSP was effective in improving management and services in HCs and that this resulted in the increased use of the HC services. Among the main improvements are the availability of services on a 24 hour basis, the regular outreach activities, the improved quality of treatment, the good behavior of HC staff towards clients; and the increased cost recovery of the HCs. In October 2002, the KSCSP began the introduction and implementation of the COPE methodology in the HCs of the Kean Svay OD. Starting with an orientation workshop followed by a series of follow up activities with the trainees, the KSCSP and OD staff gradually trained and involved a total of 131 staff in 14 of the 17 HCs of the Kean Svay OD, that is, an average of 9 staff per health centers and 95% of the total staff of these HCs. In November, a total of 283 problems had been identified (21 per health centers; range 5 to 42) among which 227 (77%) had been resolved by the HC staff and 66 (23%) had been referred to the OD or the KSCSP. Leuk Dek AD identified the largest number of problems because it started applying COPE earlier. The distribution of the type of problems identified by the health centers in each AD shows that Services and Activities is the most frequent and that this distribution is similar in the three ADs except for Human Resources problems, more frequently identified in Leuk Dek. AD # of Type of problems Health Centers # of Problems Structures & Building Material & Equipment Human Resources Services Activities Leuk Dek 4/4 148 6% 27% 27% 40% Kean Svay 5/6 56 13% 30% 4% 54% Lovea Em 5/7 79 6% 24% 6% 63% Total 14/17 283 7% 27% 17% 49% To evaluate the HC’s performance, the KSCSP used a series of tools like the observation checklists, exit interviews, and record reviews during supervision visits, and the COPE follow up assessments more recently introduced. These tools can be used to evaluate a wide range of issues such as staff performance and knowledge, good responsibility of HC 29 staff, ability to solve problem on time, HC management, and drug keeping. They are effective because feedback can be given immediately or through feedback reports and meetings. Data from the clinical observations checklists are presented and discussed elsewhere (see section B.2.d). The supervision tools used by the OD and project staff also include observation of managerial aspects of the health centers. In Kampong Thom, using a Good/Fair/Weak classification of the health centers depending on the number of satisfactory observation items, 70% of HCs were found Good for their register and documentation, 70% for their drug store and other materials, 43% for their cold chain system, 30% for their ORT corner and 13% for their cost recovery system. The supervision report for Kean Svay was not available during the FE and no comparison could be made. Following the introduction of COPE in health centers in Kean Svay OD, a quality of care assessment tool was used to score HC using a list of 40 items classified as general information, hygiene, drugs and contraceptives, IEC, client-provider interaction and counseling, service delivery, and infection preventions.23 A value between 0 (non existent) to 5 (excellent) is given to each item. Data from 14 HC collected in 4 successive quarterly assessments between January and October 2003 were available during the FE and showed a distribution of the score across all possible values and a total percentage score between 65% and 90%, both observations suggest a fairly sensitive set of indicators. The scores in the four HCs in Leuk Dek in which COPE was first introduced increased by about 10% between January and October 2003. The FE Team used findings from the field visits and their own experience to formulate the following lessons learned with respect to health facility strengthening. · The COPE approach is very effective and findings from health facility assessment conducted by the supervisory team should be considered for action and services improvement. The KS OD felt that this tool is very important and that the ADP should continue supporting its use on an ongoing basis. · Quality of care was improved through the use of HCMC reports as guidance to correcting the problems and issues that they raised. · HC staff with multiple skills can be rotated · There should be staff available in HC on a 24 hour basis. · HC should develop work plans to manage their day to day activities, To sustain these achievements once the program close, the OD and HC staff will continue implementing COPE and supervision activities, training the new HC staff, and supporting the activities of the HCMC and VHVs. They plan to ask for support from the ADP for these activities. These plans are realistic given the capacity of the OD and HC staff to do so and the good relationship that they have with the ADP. ______________________ 23 See Quality Improvement by COPE report 30 (iv) Strengthening health worker performance The FE Team found that the approach for strengthening health worker performance was effective because technical skills were updated during training and supervisory visits, the knowledge gained during training was implemented, the observation checklists helped HC staff identifying weak points and improving performance, and all the HC staff benefited from the increased cost-recovery. The training reports, the supervision reports and data, and the COPE report and follow-up data are evidence that the health worker performance improved during the KSCSP. The FE Team identified that to best strengthen health worker performance, it is important to: · Conduct regular monthly meetings · Ensure regular updates on technical issues · Use COPE approach to help change HC’s attitudes · Use of data collected from every HC · Adopt immediate problem solving techniques when issues arise · Follow-up on training or any actions taken with HC staff To sustain the improved health worker performance after the project ends, the OD will continue supervision and follow-up action including regular meetings with HC staff. They will also continue the training activities with support from the ADP. These plans are realistic. The KSCSP introduced several tools to assess health workers’ performance such as clinical observations, exit interviews and record reviews. Results from the latest observations were analyzed during the FE to identify any differences in the quality of care in the various project areas (See section B.2.d). Only comparison across project implementation areas were made at that time and adding data collected earlier may make trends analyses possible. The regular exit interview reported in the KSCSP monthly reports (see section B.3.b) are in fact used to measure the knowledge of mothers attending HCs, not that much the performance of the health workers who provided the services. The HC management assessment and the COPE follow up tools discussed in section B.3.c(iii) provide indirect HWP assessments. (v) Training Appendix E shows the progression of the HC staff training activities from February 2001 to November 2004 in Kean Svay OD and from March 2002 to June 2003 in Kampong Thom OD.24 The KSCSP provided regular training opportunities to all the HC staff in the Kean Svay OD, which represents a target group of about 100 health workers. Training in Kampong Thom was provided to a smaller number of HC staff and less frequently. The range of topics of this training, sometimes provided several times through refresher training, was broad and relevant to the project: ARI/CDD/DHF (5 ______________________ 24 See Training Reports 2002 and 2003. 31 days), Birth Spacing (clinical-3 days, and management-2 days), EPI (3 days), Breastfeeding (3 days), Vitamin A (1 day), IDD (1 day), HIS (1 day), Basic Management skills (5 days). The training materials for HC staff were those developed and used by the MOH. Appendix E shows the progression of these VHV training activities from June 2001 to November 2004 in Leuk Dek and from September 2002 to June 2003 in Kampong Thom. In Leuk Dek, the KSCSP also trained TBAs, drug sellers and traditional healers between September 2001 and March 2002, with refresher training in April 2003. The training provided to VHVs also covers a wide range of topics such as EPI (including Hepatitis B), Birth Spacing, ANC, ARI, CDD, DHF, IDD, Vitamin A, BF, HIS, STD , HIV/AIDS and Nutrition. These topics were provided several times as refresher. The training materials were developed by WVC. The OD, HC, and KSCSP staff found that the training approach of the KSCSP was effective. All the training topics stated in the DIP were addressed by the KSCSP. The results of the pre- and post-training tests, the KPC surveys, the clinical practice observations and the findings from the FE field visits all indicate that the training implemented resulted in improved knowledge, skills and performance of the participants. Through interviews of the OD and HC staff and group discussions, the FE Team established that the strengths of the KSCSP training program was that regular discussions were held between the OD, HC, and KSCSP staff to select the training topics, trainers and trainees; the OD and HC staff participated in the delivery of training; the training was based on the national guidelines; and the training topics were relevant and suitable to the HC staff and the VHVs. The OD and HC staff plans to conduct refresher training after the end of the project. The HC staff may be able to continue training VHVs with their own resources but the OD will need to find other sources of funding, including WVC or the ADPs, for continuing training the HC staff. d. Sustainability Strategy The sustainability objectives articulated in the DIP (see Table 3) were met except for the involvement of the private sector providers. Advocacy was conducted at the community level through COPE, at the provincial and the OD levels through regular meetings, and indirectly through quarterly meetings with other NGOs at the national level. The KSCSP phase out planning process started 2 years ago and is documented in a series of minutes of the regular meetings between the OD, the ADP and the KSCSP during which joint POAs and issues of transfer of assets are discussed. The OCA results (see section B.3.c(ii)) show that the Kean Svay and the Kampong Thom ODs have developed systems for short- and long-term continuity. WVC will continue providing technical assistance through the ADP Health Coordinator, Operation Health Coordinator, and the Health and HIV/AIDS Program Coordinator in Phnom Penh. 32 The KSCSP approach to build financial sustainability was effective. The OD introduced cost recovery mechanisms into the HC services delivery to sustain VHVs and regular HC activities. The OCA results show that KS OD rated itself 5 on the scale of 1 to 6 for this indicator. The ADPs will take up the KSCSP’s role of providing at least some support in the area of training and community mobilization. During the field visits, the FE Team was able to observe the high demand for continued project’s activities created at the community, HC and OD levels. The VHVs and other community groups are fully involved in and influence the delivery of these activities. 33 C. Program Management a. Planning The FE Team found that the KSCSP planning process was clearly perceived to be inclusive by the KSCSP, OD and HC staff, who had quarterly and monthly planning meetings on the implementation of the DIP. The KSCSP DIP was the result of a collaborative and participatory process and therefore had broad ownership. It was found practical by the KSCSP, OD and HC staff. No gap was identified. Sections written by people knowing the local situation were more useful than those written by outside experts who did not understand the Cambodia context. Some tables of the DIP were translated into Khmer. b. Project staff training The training and change in knowledge, skills and competencies of the project partner's staff are discussed in section B.3.c(v). The KSCSP staff also benefited from valuable training opportunities on various topics such as KPC and LQAS surveys, COPE, BEHAVE, IMCI and Monitoring and Evaluation. c. Supervision of project staff Based on the field visits and group discussion, the FE Team found that: · The supervisory system of the KSCSP was good but could be further improved. · Supervisory plans are in place at the OD level, although not documented. · HCs maintain Golden Books to record visits from supervisory staff among other visitors. · The Kean Svay OD modified the supervisory checklists and is equipped to train HC staff. · The Kean Svay OD is able to articulate what an adequate supervisory system should be, which is an indication that they understand its importance. · The supervision system helped improving health workers performance and quality of services by detecting and correcting problems early. · There was not enough time to do adequate supervision and the geographic area to be covered was too big. These findings demonstrate that the KSCSP was successful in institutionalizing the supervision system although more support is still required to maintain it on the same scale, particularly to increase the frequency of the supervisory visits. d. Human resources and staff management The Leuk Dek ADP, the KS OD and HC and the HAHP have met regularly to discuss personnel policies. Essential personnel policies are in place within WVC and ADPs to continue the KSCSP activities. The ADP in LD increased its budget to include health staff and support the VHVs. 34 At the beginning of the KSCSP, there have been problems and misunderstandings between the KSCSP and ADP teams but these differences were gradually sorted out and the various teams now work well together. Team building experiences and many joint meetings were organized which greatly helped the process. Since last year, the WVC/HAHP has a stress management team or “Care Group” who meets every 2 or 3 months and conduct outdoor team building workshop. The KSCSP staff found that there has been a good management and teamwork in the project. They regularly received expressions of appreciation from other staff in WVC and the KSCSP has often been used as demonstration site. There has not been much staff turnover during the project implementation period. The occasional changes in project staff had little impact on project implementation because responsibilities were appropriately transferred to other staff members. WVC encouraged all KSCSP project staff to re-apply for other positions. At the time of the FE, one staff member was recruited in the HIV/AIDS program (STAR5), 3 in the Mainstreaming AIDS/ADP Program (MAAP), one in the WVC Monitoring and Evaluation Department, and one is considered as health coordinator for the ADP in LD. e. Financial management WV is a large and mature organization with good financial policies and procedures. These organizational guidelines were a help to the project. Budget management was more difficult when natural disasters such as flooding prevented implementation of planned activities. The financial coordination with WVUS was good except for slow communication in the last quarter of the project. The ADPs will carry on only a few activities of the KSCSP, and there will be sufficient funding for this. The project developed its own plans for financial sustainability of activities through the ADPs, and did not need technical assistance with this respect. f. Logistics The FE Team found the logistic support from the KSCSP had an important impact on the HAHP, OD, HC and ADP activities: vehicles helped project staff go faster and save time, especially Leuk Dek where the geographical access is difficult; bicycle for VHVs were very useful; in Leuk Dek, one set of video player, TV and generator for each HC was very useful for health education; tri-motorcycle ambulances in HCs were used for health education and to refer patients from village to HCs. The main challenges were the high maintenance cost of the vehicles for the KSCSP and of the tri-cycle ambulances and the generators for the HCs. At the OD level, it is difficult to say whether the logistic system was adequate with respect to the national guideline. The government priorities keep changing every year and the needs are vast. The government finds it difficult to support the system. At the time of the FE, the OD had not yet received funds for FY03. 35 At the HC level, there is a lack of clean water, electricity, radio communication and ambulance to refer patients to Phnom Penh. The existing equipment is not durable. There is no fuel for the generator that is therefore used for emergencies only. Clinical forms are in short supply. There is a need for educational posters, and medical, surgical and laboratory equipment. Most of the staff is using personal vehicles for outreach activities. At the ADP level, the logistical support and distribution system was adequate. As the distribution of the equipment and logistic supply was done directly to the OD, the ADP has not had the opportunity to be involved in discussion about the future use and maintenance of this equipment. g. Information Management The KSCSP staff collected a wealth of data to monitor progress towards the objectives and these data have been used at different levels: VHVs, HCMC, HC, OD, KSCSP and HAHP. In 2003, the data for several of indicators collected routinely have been compiled in summary tables and graphs, thereby meeting a MTE recommendation. This background work made it possible to use some of this data during the FE and for the present report. Most of these analyses at the aggregate level were done by the Dr. Ou Chandara Mony, Chief, Technical of Kean Svay OD, who was recruited on a part-time basis on the KSCSP when the M&E Officer was transferred to the WVC M&E Department. Dr. Mony will return to the OD at the end of the project. The project initiated and implemented the CBDDS system through the VHVs and with the government health staff but did not work on the HC/OD health information system except indirectly through training of the staff and emphasis on the use of data. The FE Team found that the PHD, OD, HC, HAHP and project staff is well aware of the project achievements and of the data and evidence that they collect and use to documents them. The KSCSP staff trained ADP staff in the implementation of KPC surveys and in the use of community registers. Some of the project data and related experience has been shared with the ADPs across WVC and with other Child Survival partners in Cambodia. Below are brief descriptions and some results and comments of four components of the KSCSP information system that also include references to other sections of the report where specific data are discussed. 36 (i) Monthly Activity Reports The KSCSP used a system of 2-page monthly activity reports prepared by the project manager. 25 One section of these reports was a standardized account on planned activities in Leuk Dek and the other section listed special activities, meetings or other events that occurred in the month. A series of indicators from these monthly reports were compiled on an annual basis. The data for most of these indicators came from the VHV reports on their activities and on birth, diseases and death in the community. These reports are part of the CBDDS system that is further discussed below. These data have helped the KSCSP and HC staff improving the performance of the VHVs and ensuring progress towards several objectives. The monthly reports also include the knowledge assessment conducted with the VHVs and with mothers attending health centers that were presented in section B.3.b. Actual assessments of the performance of VHVs based on observations were not done. Mothers attending health centers may be more knowledgeable than other mothers in the community that do not, and they may or may not have received health education about one or the other topic during their visits, but regular sampling in health centers is convenient monitoring system. These two assessments were conducted in Leuk Dek only and there is no data so far to compare knowledge of the VHV and mothers in the other project implementation areas. (ii) Supervision and COPE reports The KSCSP and OD staff prepared supervision reports that include data on health workers performance and health center management. Starting in 2003 with the introduction of the COPE approach in Leuk Dek, the KSCSP and OD staff also prepared COPE follow-up reports. The supervision and COPE follow-up tools (results presented in sections B.2.d and B.3.c(iii)) are useful and were used by the supervision teams to improve the health worker performance and the quality of care in health centers. Gaps in performance found during these assessments were regularly addressed by conducting meeting with OD and HC staff, offering refresher training, and following-up on the adopted changes. The OD staff actually started modifying these tools to respond to new needs. (iii) Intermittent surveys and assessments The KSCSP has also collected data on intermittent basis including numerous KPC and LQAS surveys in the various project implementation areas, and the OCA of the OD in KS and KT. These data were mainly used by the HAHP and OD staff during key planning events such as the DIP workshop and the MTE and FE. ______________________ 25 See Monthly and Quarterly Project Reports, Year 2001, 2003, and 2003. WVC-KSCSP. 37 (iv) CBDDS The KSCSP has implemented the Community-Based Death and Disease Surveillance (CBDDS) system to supplement the formal health information system with morbidity and mortality data on children under five and to monitor trends and evaluate the effectiveness of child health and survival interventions. The CBDDS implemented in the Kean Svay AD during Phase 1 of the KSCSP was replicated in the Leuk Dek AD. The CBDDS includes information on deliveries by outcome, place and attendant; deaths under five, including detail on age, place, and Verbal Autopsy; maternal deaths; cases of infectious diseases. Adjustment in the data collection forms for VHV and in the Verbal Autopsy methodology were made in 2002. A questionnaire for Social Autopsy was also prepared and tested in 2002 but has not been implemented yet. The CBDDS is operated in a participatory and sustainable way through the local health staff. The VHVs collect the data and submit monthly reports that include information on their activities. The HC and OD staff then provides systematic feedback to VHVs and timely response when needed, with support from the project. Preliminary analyses26 of the available CBDDS data suggest a quite complete and stable reporting, as assessed by the percent of expected reports actually received, during the 28- month period of implementation from July 2001 to October 2003. These preliminary analyses are based on the periods of 07/00 to 10/01, 11/01 to 10/02 and 11/02 to 11/03 referred here as FY01, FY02 and FY03 for convenience. A total of 1936 births and 149 deaths under five were reported during this period of implementation. Between FY02 and FY03, the number of births decreased by 17%, of which a maximum of 5% might be due to missing monthly reports in FY03; no other reason than a decrease in fertility consistent with the documented increase in contraceptive use seems to explain this decrease in the number of reported births. In FY03, 51% of the reported deliveries occurred at home with a TBA (a 16% decrease from FY02), 39% occurred at home with a health worker, and 10% occurred in a Health Center, Hospital or Clinic. These figures are consistent with those found in the August 2003 KPC survey of mothers with a child under 2 who reported that 87% of deliveries occurred at home and 47% with a “trained health provider.” The preliminary analyses of the CBDDS data also show that neonatal mortality decreased by 33% each year between FY01 and FY03 and that equivalent decreases in post neonatal mortality resulted in quite stable infant mortality. These data are presented in Figure 19. While the decrease in neonatal mortality could be related to the intense child survival activities in the area, the increase in post neonatal mortality is difficult to interpret. A shift in the age definition criteria is not excluded. If anything, the rate of reporting ______________________ 26 Personal communication, Dr Oun Sophal. 38 probably improved during the period of implementation, and this would tend to increase mortality estimates for all age groups. Rates for FY01 are based on only four months. Figure 19 Trends in mortality under five as reported by the CBDDS in Leuk Dek 0 20 40 60 80 100 Number of deaths per 1,000 live births Child (12-59 months) 14.7 19.0 12.7 Postneonatal (1-11 months) 19.6 35.9 42.0 Neonatal (0 month) 39.2 26.4 17.8 FY01 FY02 FY03 Note: Mortality rates calculated by dividing the number of deaths reported in each age category and fiscal year by the number of births for that year (204, 946 and 786 for FY01, FY02 and FY03, respectively). Among 142 deaths under five reported in the CBDDS, 95 (67%) remained with unknown diagnostic while the results of the Verbal Autopsies conducted by the Health Centers and KSCSP staff show that 28 (20%) were attributed to pneumonia, 5 (3.5%)to neonatal infection, 5 to prematurity, and 5 to meningitis.27 The CBDDS includes reporting of cases of chronic cough, measles, hemorrhagic fever and severe diarrhea among children under five, children 6 to 14, and adults 15 years and older. Data on measles and hemorrhagic fever are discussed elsewhere (section B.2.b and B.2.d, respectively). Collecting data on chronic cough among adults (1 case in FY03) might be a good contribution to the early detection and control of tuberculosis but the link between the CBDDS and such program it is not clear at this point. The cases of severe diarrhea among children under five (3 cases in FY01) might also be too rare in a context of very high use of ORT in the community and reporting them on a monthly basis ______________________ 27 See Main Activities of Monitoring in Leuk Dek District, Kean Svay, Kandal PHD, in year 2001-2003. WVC/KSCSP. 39 might not be sensitive enough to constitute a efficient early warning system for cholera or dysentery outbreak. The CBDDS includes data on the activities conducted by the VHVs which gives opportunities to monitor their performance: · Distribution of postpartum vitamin A (see section B.2.c), ORS packets and condoms. · Number of referrals from VHVs to public health facilities (see section B.2.d.) · Number of women and men who received health education in Leuk Dek (see section B.3.b.) The following lessons learned were developed for the Lesson Learned workshop presentation on CBDDS: · CBDDS can provide reliable data to measure impact of reproductive and child health services, detect disease outbreaks, and monitor VHV activities · CBDDS can increase participation of VHVs and other community members but also needs strong community support itself · CBDDS data can be analyzed at the HC level to give immediate feedback and follow up action · The Pregnancy Tracking Form (community register) has a central role to play in the CBDDS · Verbal Autopsies need to be kept simple. · CBDDS needs at least 2 to 3 years to be operational · Sustainability of the system depends on the sense of ownership of its implementers and users, a minimum of resources for the form, and a continued support to the VHVs. h. Technical and Administrative Support The FE Team acknowledged that the KSCSP provided regular and appropriate technical assistance to the government participants though discussion during regular meetings, joint planning and implementation of all activities, and training and supervision opportunities. The HAHP staff also provided managerial and technical assistance through regular visits, meetings and informal discussions. Almost all the technical assistance needed for the project was available. Three external consultations were successfully conducted on BCC, CBDDS and VHV performance assessment, and provided useful directions to the project in their respective areas. One area where technical assistance was needed but not available was the definition of specific objectives, data needs, and analysis plan to estimate the cost-effectiveness of the different models of implementation of child survival (see MTE recommendation). World Vision US/HQ provided both managerial and technical support through the Program Officer, Finance Officer, Contracts Officer and Technical staff assigned to the project. The Project Officer provided a point of contact for all communication between the project and the HQ. In addition, narrative reporting was coordinated through the Program Officer. The Finance Officer monitored project expenses at the HQ level and 40 prepared financial reporting in collaboration with the field project staff. Notice and interpretation of legal obligations provided for in the grant was facilitated by the Contracts Officer. Legal arrangements for consultants required by the project were also facilitated by the Contracts Officer. The WVUS Technical Team provided backstopping as needed by the project. The Regional Health Advisor was instrumental in the design, monitoring and evaluation of the project. He played a key role in providing continuity between the project phases, and ensuring that project activities were designed and conducted according to the best practices known in the region. i. Management Lessons Learned The FE Team identified the following overall management lessons learned. · A well-defined structure of management helps running health systems more effectively. · Systematic monitoring can help identifying and solving problems early and improving quality of care. · There should be an adequate plan for sustainability. · Communication chains must be kept clear. · Good working relationships and cooperation between officials ensures better quality of services. · Investing in staff capacity building in a planned and strategic manner helps improving performance, quality of services and sustainability · It is important to strengthen supervisory skills so that meaningful feedback can be given to the teams. · It is important to involve the community, especially VHVs, HCMC and local authorities, to increase program ownership. · Record keeping is an important element of good management. · Many interventions and achievements of the KSCSP can be replicated in other provinces (example of COPE that can be used in Kandal PHD and in Kampong Thom). 41 D. Conclusions and Recommendations 1. Child health and survival The KSCSP has been successful in achieving its child health and survival objectives in the direct impact area, the Leuk Dek AD, in three years of implementation. In Kean Svay, the coverage of many interventions has increased from the levels achieved in Phase 1 of the project, but it has decreased for others—a concern for the sustainability of the project. In Lovea Em, the coverage of most of the KSCSP interventions has increased although in general less than in Leuk Dek. In Kampong Thom, valuable results have been achieved, but the lack of comparable data makes it difficult to compare with the results achieved in KS OD. Table 6 presents a summary of the achievement of the specific interventions of the KSCSP in Kean Svay AD during the two phases of the project and in the three other implementation areas during Phase 2. The structure of the table corresponds to that of Table 5 where the five implementation models are characterized based on the type and relative intensity of project support. In this qualitative assessment, the achievement of one intervention is positive (+) if the related objective is achieved28 or if the results are deemed equivalent.29 ______________________ 28 End-of-project objectives are specified in the DIP for Leuk Dek only. 29 Examples of equivalent results are the same absolute increase from a lower baseline coverage or a smaller increase from an already higher baseline coverage. 42 Table 6 KSCSP achievements by implementation model and intervention Implementation model Kean Svay OD Intervention Kean Svay AD Phase 1 (1) Phase 2 (2) Leuk Dek AD (3) Lovea Em AD (4) Kampong Thom OD (3 ADs1 ) (5) Immunization 1. Complete immunization of infants + - + + - 2. Tetanus Toxoid in pregnant women + - + + N/A Nutrition 3. Exclusive breastfeeding + + + + N/A 4. Vitamin A in children 6-23 months + - + - N/A 5. Vitamin A in post partum mothers + - + + N/A 6. Use of iodized salts + - + - + IMCI 7. Use public health facility when ARI + + + + N/A 8. ORT when diarrhea + + + + N/A 9. Same or more food when sick + + + + + 10. Same or more fluids when sick + + + + + 11. Knowledge of danger sign of DHF + + + + N/A Birth spacing 12. Contraceptive prevalence + - + + N/A Note: + : positive achievement (objectives achieved or equivalent result); - poor achievement; N/A: no available data. 1 Prasath Sambo, Prasath Balang, and Sandan. Conclusions of this preliminary comparison of the various KSCSP implementation models are limited by the lack of complete study design and allocation of specific resources at the beginning of the project. However, the following conclusions can be drawn: - All the objectives were met in Phase 1 in Kean Svay and in Phase 2 in Leuk Dek, the two models of relatively intensive support. This finding suggests that providing support to health centers for outreach activities (Leuk Dek, Phase 2) achieved equal or better results than a more expensive mobile team approach (Kean Svay, Phase 1). - The results achieved in Kean Svay in Phase 1 were not sustained in Phase 2 when the intensive support to this area decreased. - More intensive support to outreach and community mobilization in Leuk Dek produced better results for some but not all interventions than in Lovea Em. - Direct support to the OD in Lovea Em achieved better results than indirect support to the ADPs in Kampong Thom, two implementation areas with otherwise limited project support. 43 The KSCSP collected enough valuable data to warrant further investigation of these issues, as discussed in section D.3 below. Intervention-specific recommendations: · WVC should investigate and document the success of the KSCSP regarding selected interventions like birth spacing, postpartum vitamin A, and exclusive breastfeeding. The findings should be disseminated to various audiences in Cambodia and elsewhere. · WVC should investigate the reasons for and the nature of the persistent high use of anti-diarrhea medication in the project areas. More specific indicators for this behavior are needed to further address this concern. · WVC should investigate the reasons for the decline in the coverage of vitamin A supplementation among children 6-23 months observed in Lovea Em. 2. Capacity building Following on its success in the Kean Svay AD, the KSCSP was able to further build the capacity of the Kean Svay OD to implement child survival activities in other areas. During Phase 2 of KSCSP, all local partners (OD, HC, VHVs and local authority/commune leaders) were systematically involved and participated actively. The capacity of the OD and HC staff improved in terms of knowledge, clinical and managerial practices, behaviors towards to the clients and support systems. At the community level, VHVs and community leaders gained ownership of the KSCSP￾supported activities and will continue their implementation. ADPs in Leuk Dek and in Kampong Thom have integrated some child health and survival activities, approaches and tools into their plans. 3. Data and documentation The amount and quality of the monitoring data collected during the KSCSP constitutes a valuable resource for an in-depth evaluation and documentation effort at the end of the project. Much more can be done for dissemination within ADPs, other health and child survival projects within WV, and within international health and child survival community: WVC, WVAPRO and WVUS should complete the analysis of the cost-effectiveness of the various models of implementation of child survival proposed in the DIP and initiated during the FE. Among task to accomplish with this respect are: · A full analysis of the effectiveness of the project in the various implementation areas using coverage data from the KPC surveys conducted for this purpose. The use of one database including all the KPC surveys data would best serve these analyses. Any other appropriate data that might be available within the KSCSP or elsewhere should be identified and used. Some summary indicators of all interventions should be developed to allow comparing the overall effectiveness across areas. 44 · A full analysis of the cost of the various inputs from the KSCSP and possibly other sources. The type of support identified in this report constitutes a first identification of these inputs that need to be further specified and quantified. · Identification and documentation of the key other factors that might have influenced the effectiveness of the KSCSP in each area. Internal, project-related factors might be specified using data such as mother and VHV knowledge (assessment are made in LD and could be done in KS, LE, and KT), health worker performance (observation checklist from supervision), and project management and leadership. External factors to be considered include community mobilization; human factors and leadership at the community level and in government health services; literacy; accessibility; size of the population; other ongoing projects). An end-of-project KPC survey in the control area (baseline done in November 2000 in Kasach Kandal) may help assess trends in the coverage of interventions resulting from other factors than the KSCSP. · Involvement of stakeholders and former KSCSP staff in the definition of the question to answer, identification and collection of data, analyses, and dissemination of findings in Cambodia. · Dissemination of relevant findings outside Cambodia. WVC, WVAPRO and WVUS should review the status of the Community-Based Disease and Death Surveillance Systemin Kean Svay before turning it over at the end of the project. This review should be conducted within the perspectives of other CBDDS projects that WV is supporting worldwide (Bangladesh, Indonesia, Philippines, and India). The review of the CBDDS of the KSCSP should include: · A full assessment of the quality of the existing data including the completeness of reporting of births and deaths and the quality of the attribution of date or age at death over time. · A full analysis of trends in fertility and mortality and of the impact of the KSCSP taking into account the limitation of the data and any other supportive evidence on health programs, migration, and socioeconomic factors. · A full assessment of the operations of the CBDDS (community participation, use of data, ownership, workload, quality control) and recommendation on revisions to the need and capacity of the Leuk Dek ADP, OD and HCs after the end of the project. · Realistic plans for continued technical support for at least two more years after the end of the project. The availability of additional quality data will increase the reliability of the analyses and interpretation of the existing data, and give the opportunity to better assess the value of the CBDDS in Cambodia. If the quality of the data and potential feedback to the community and health centers is warranted, Verbal and Social Autopsies of all deaths should be conducted during a predetermined period of time, say two years. The KSCSP introduced and used several new monitoring and evaluation tools and methods. Among tools that have appeared useful are the clinical observation and the HC management checklists, the knowledge assessment questionnaires for VHVs and mothers, and the Organizational Capacity Assessment. WVC, WVAPRO and WVUS should conduct participatory assessment of these tools using the available data and the experience of their users to draw conclusions and recommendations about their use, and 45 disseminate these findings within World Vision and other organizations working in health and child survival. 46 E. Results highlights The Ministry of Health and World Vision in Cambodia have implemented a seven-year Child Survival Project in the Kean Svay and the Kampong Thom Operational Districts (OD) from 1997 to 2003. Phase 1 of this project was successfully implemented in Kean Svay Administrative District (AD) from 1997 to 2000, and Phase 2 was also implemented in the two other ADs of the KS OD and in three ADs of the Kampong Thom OD from 2001 to 2003. One objective of the Phase 2 was to identify the most cost-effective models of integration of child survival interventions within the Area Development Programs (ADP), the programming units of integrated development based on a 10-15 year commitment to improve child well being and reduce poverty in defined geographical areas. The technical focus of the two phases of the KSCSP was on immunization, nutrition, IMCI and birth spacing. In each area, the KSCSP provided various types of support at three levels of the health system: the OD, the Health Centers, and the community. In addition, the KSCSP provided this support through the structure of an ADP in Leuk Dek and in the Kampong Thom OD. These types of support characterize five implementation models. To compare the effectiveness of the various models, the KSCSP conducted a series of KPC surveys on the basis of which an overall assessment of the achievement of each model can be made. The table below summarizes the relative support provided and overall achievement of each implementation model. Implementation models Kean Svay OD Kean Svay AD Phase 1 Phase 2 Leuk Dek Lovea Em Kampong Thom OD (3 ADs) Support Operational District +++ +++ +++ +++ + Health Center +++ ++ +++ + + *mobile team + - - - - Community +++ + +++ - - ADP - - + - + Achievement +++ + +++ ++ + In three years of implementation, the KSCSP Phase 2 was successful in achieving its child health and survival objectives in the direct impact area, the Leuk Dek AD. These results were similar to those achieved in Phase 1 in Kean Svay AD without mobile team and direct involvement of project staff in service delivery, that is, in a less expensive way. During Phase 2 in Kean Svay AD, the coverage of several interventions increased from levels achieved in Phase 1 but decreased for others—a concern for the sustainability of the project. In Lovea Em, the coverage of most interventions increased although in general less than in Leuk Dek. In Kampong Thom, valuable results were achieved, but the lack of comparable data makes it difficult to compare with results in KS OD. These preliminary findings show that more in-depth analyses of the available data and experience from the KSCSP are warranted. 47 APPENDICES 48 Appendix A Final Evaluation Team Members I- Team Leader: Dr. Marc Debay, independent consultant. II- Facilitators: 1- Dr. Sri Chander, WVAPRO/Singapore. 2- Dr. Douglas Shaw, HAHA/WVC. 3- Dr. David Grosz, WVUS. 4- Dr. Ly Vanthy, HAHP Manager, WVC. III- Observers: 1- Mrs. Sushma Cornelins, Operation Manager, WV India. 2- Mr. Jaya Kumur, Bali ADP Manager, WV India. 3- Mss. Esther Indriani, Project Officer, WV Indonesia. IV-Kean Svay/Leuk Dek Core Group: No Name Sex Function Location 1 Dr. Prak Phan M Vice-Director Kandal PHD 2 Dr. Chea Eng Mao M PHD-MCH Chief Kandal PHD 3 Mr. Phan Samnang M PHD-EPI Chief Kandal PHD 4 Mr.Tuy Saroeun M Kean Svay OD Chief Kean Svay OD 5 Mr. Kim Sophon M Deputy District Governor Leuk Dek District 6 Mr. Mourng Samon M Deputy District Governor Kean Svay District 7 Mr. Yi Sophoannarith M Vice Chief of KS OD/Lovea Em Lvea Em district 8 Mr. Bin Samreth M Vice Chief of KS OD/Leuk Dek Leuk Dek district 9 Mr. Pen Sokhoeun M EPI Chief of OD Kg. Thom Kg. Thom OD 10 Dr. Ou Chandaramuny M Chief of Technical Unit KS OD Kean Svay OD 11 Mr.Yun Sa-Eng M Chief of Kg. Phnom HC Leuk Dek district 12 Mr. Mak Sakhoeun M Chief of Prek Tunloap HC Leuk Dek district 13 Mr. Teang Sarath M Chief of Prek Dach HC Leuk Dek district 14 Mr. Thang Seng M Chief of Sandar HC Leuk Dek district 15 Mrs. Sao Sareth M Vice Chief of Dey Eth HC Kean Svay district 16 Mr. Norng Sovanna M Chief of Banteay Dek HC Kean Svay district 17 Mr. Ros Hoy M Chief of Samrong Thom Hc Kean Svay district 18 Mr. Horn Hongsreng F Koki Thom HC Staff Kean Svay district 19 Mr. Nuv Soeun M EPI Chief of Lvea Em district Lvea Em district 20 Mr. Long Heng M Chief of Toeuk Klaing Lvea Em district 21 Mrs.Hem Samorn F VHSG Kg. Phnom HC Leuk Dek district 22 Mr. Hean Sourn M VHSG Sandar HC Leuk Dek district 23 Mrs.Sok Samnang F CC of Prek Tunloap Commune Leuk Dek district 24 Mr. Oy Heam M VDC of Kpop Ateav Commune Leuk Dek district 25 Mr. Ngourn Pheap M Leuk Dek ADP Staff Leuk Dek/WVC 26 Dr. Pek Kimsan M Health Operation Coordinator Kg. Thom -WVC 27 Dr. Sour Kim An F OIC of KSCSP Manager. HAHP-WVC 28 Dr. Pen Sophea M Training Coordinator KSCSP-WVC 29 Mr. Keo Sereivuth M M&Evaluation Officer M & E unit/WVC 30 Mrs. Kan Chamroeun F Health Officer KSCSP-WVC 31 Mrs. Hourt Sokda F Health Officer KSCSP-WVC 32 Mrs. Huy Dany F Health Officer KSCSP-WVC 49 V-Kompong Thom Core Group: No. Name Sex Function Location 1 Dr. Meas Sokha M Deputy Director Kg. Thom PHD 2 Mr. Pen Sokhoeun M EPI Chief of OD Kg. Thom Kg. Thom OD 3 Mr. Tol Kimreth M Chief of Chheuteal HC Sandan Distric 4 Mr. Nou Puthy M Chief of Mean Chey HC Sandan District 5 Mr. Tep Sakin M Chief of Taing Krasao HC Prasath Sambo District 6 Mr. Seng Sophal M Chief of Chhouk HC Prasath Sambo district 7 Mr. Ros Phan M Chief of Sambo HC Prasath Sambo district 8 Mr. Lim Srun M Chief of Salavisay HC Prasath Balang district 9 Yi Yeap M Commune Development Council Prasath Balang district 10 Sourn Chhorn F VHV of Mean Chey HC Sandan district 11 Sam Thean F VHV of Salavisay HC Prasath Balang district 12 Sim Seak M Commune Devel. Council Prasath Sambo district 13 Kim Chea M Commune Devel. Worker Sambo ADP-WVC 14 Dr. Yim Pivatho M Health Coordinator Balang ADP-WVC 15 Dr. Pek Kimsan M Health Operation Coordinator WVC-KPT 16 Dr. Pen Sophea M Training Coordinator KSCSP-WVC 17 Mrs. Huy Dany F Health Officer KSCSP-WVC 50 Appendix B Final Evaluation Methodology The KSCSP is part of the USAID/Child Survival and Health Grant Program (CSHGP) that supports US-based PVOs in the implementation of child and maternal health and nutrition, family planning, HIV/AIDS and infectious disease activities. The CSHGP requires grantees to conduct external Midterm Evaluations (MTE) and Final Evaluations (FE) as part of their project and issues guidelines on how to conduct them. 30 The main characteristics of the MTE and FE are presented below. Midterm Evaluations Final Evaluations Assess progress towards objectives Identify problems in design or interventions Develop recommendations for remaining of project Assess and documenting results Identify effective approaches Develop lessons learned for future projects The FE of the KSCSP was conducted according to the following three principles of the USAID guidelines. 1. The KSCSP FE was highly participatory, with a total of 57 participants representing various local partners (including representatives from the PHD, OD, HC, VHV and communities in Kean Svay and Kampong Thom) and World Vision (US Headquarters, Asia Pacific Region Office, National Cambodia Office, and KSCSP). 2. During all the group discussions, participants made specific references to project planning documents such as the DIP and the MTE report. 3. The FE process was designed to ensure that participants identify and use all available project data to answer the FE questions. The KSCSP FE was conducted for the following audiences: 1. Staff of the KSCSP and its local partners. 2. Staff from World Vision national, regional and US headquarters offices. 3. Staff of the USAID mission in Cambodia and headquarters in the USA (CSHGP) Although designed for a full participation of the entire team of 57 participants (hereafter the FE Team), the FE was facilitated by a smaller group of 7 members (hereafter the FE Facilitators) comprising 4 WV staff members directly involved in the KSCSP, 3 observers from WV India and Indonesia and the FE Team Leader. As about two third of the FE Team had already participated in previous evaluations of the KSCSP, the entire FE Team was invited to define the FE questions and identify and analyze the relevant data to answer them. The various steps of this FE process emphasizing the use of project data and evidence are summarized below and in the diagram that follows. ______________________ 30 See USAID/GH/HIDN/NUT Child Survival and Health Grants Program. Guidelines for Final Evaluation, July 2003. 51 1. Define the FE Questions using the USAID guidelines, the MTE recommendations and any other source identified by the FE Team members. Group Work (technical, cross-cutting, and management) followed by plenary presentation and discussion. Output: List of FE questions 2. Identify existing data sources and specific questions, tasks or methods to answer the FE Questions defined in #1. Group Work followed by plenary presentation and discussion. Output: Completed table below FE Question Data Source Method … 3. Sort the data sources and proposed methods identified in Step # 2 within three categories: Field Visits, Secondary Analyses and Group Discussion. Specify questions or task to be done for each FE Question. Output: Completed table below Data source FE Question Method Field Visits … Secondary analysis … Group Discussion … 4. Finalize tools, sampling, logistic, and schedule for each methods for field visits and conduct them. Output 1: Completed questionnaires, instructions or guidelines for type each interview or task; Output 2: Consolidated reports on findings from each interview or task; Output 3: Consolidated report on findings from each type of interview or task (sample)31 5. Specify and conduct secondary analyses. Output : Consolidated reports on findings from each analyses. 6. Specify topics for and conduct FE Group Discussions. Output : Consolidated reports on results of discussion of each specific question 7. Specify answers to all the FE questions using all data available. Group work (same as in step # 1 and 2). Output : Reports with answer to each FE questions 8. Discuss and develop consensus on FE conclusions. FE Team. Output : Notes on changes to above reports The diagram below summarizes these steps. ______________________ 31 This 40-page report called “Field Visits Data by FE Questions” is available at the KSCSP office. 52 KSCSP Final Evaluation Flowchart MTE recommendations USAID FE Guidelines FE Questions Secondary data analysis /Document Review FE Groups Discussion Primary data analysis Methods and Tools for Field Visits Data collection Yes Data exist? Specify source Answer available? Specify Answers to FE Questions Data can be gathered in Field Visits? No Yes No Yes No Discuss and develop consensus on FE conclusions Other 53 The FE Team identified as existing data sources as relevant to the FE Questions the various KPC and LQAS surveys, the CBDDS, the OCAs, the KSCSP Activity Reports (monthly reports, training reports, etc), the supervision reports, the OD, HC reports, special studies (VHV performance assessment; CBDDS, etc). The FE Team also found that given the timeframe of the FE, field visits could include interviews with the PHD, OD, and HC staff, and with HMHC, VHV, TBAs, and mothers and fathers. The interview guidelines and other tools for the field visits were developed with the following principles in mind: Relate to clearly defined FE questions; Relate to clearly defined target group/interviewees; Prepare one Interview guide per target group/interviewees; Use same questions in different Interview Guides if needed; Use mainly open-ended questions, but not necessarily only; Keep questions clear and simple; Do not include several questions or issues in one question; Do not suggest answer in question; Prepare sequence of questions carefully; Ensure appropriate environment/dynamic to conduct the interviews. The following limitations and constraints were faced during the FE: · There were not always enough translators conversant in KSCSP and technical matters and able to translate texts and discussions both ways. · There has been some confusion with the new approach involving everyone not only in the data collection but in the entire evaluation process, from defining FE questions to formulating answers and conclusions. For instance, it was not foreseen that the groups would choose to answer them all the questions in the USAID guidelines—a quite ambitious task. · Different groups sometimes adopted different approaches at each step of the process, each with pros and cons; this resulted in difficulties when consolidating the various outputs of the groups. · Sampling for the Field Visits was implicitly imposed by logistical constraints or preliminary arrangements and it was not always easy to retro fit this sampling into the specific methodology developed to answer the FE questions. · Much available project data was identified but not analyzed during the group work. Some of these data was analyzed later and the results included in the FE report. The FE of the KSCSP ended with a Lessons Learned workshop at the national level that gathered 56 participants representing various Cambodians and international organizations involved in child health and survival in Cambodia. The purpose of the workshop was to draw lessons learned from the seven years of implementation of the KSCSP with the participants and begin their dissemination to this and to a larger audience. The workshop focused on three presentations or highlights of the KSCSP: (1) Community-Based Death and Disease Surveillance; (2) Partnership and Capacity Building at the Community and the Health System levels; and (3) Comparing models of delivery of health services. Each presentation included the overall approach, the main results, and the lessons learned related to the topic, and was followed by a moderated large group discussions. The agenda and list of participants of the Lessons Learned Workshop are in Appendix G Appendix H , respectively, and the content and minutes of the presentations and discussions are available at the WVC office and included in the FE report as appropriate. 54 Appendix C Final Evaluation Documents Reviewed Kean Svay Child Survival (Follow-On) Project Detailed Implementation Plan, World Vision Cambodia, April 2001 and Reviewer’s Comments. Annexes to Kean Svay Child Survival (Follow-On) Project Detailed Implementation Plan, World Vision Cambodia, April 2001 (includes results of KPC Survey). Kean Svay Child Survival Project Final Evaluation (Phase 1), World Vision Cambodia, September 2000. Kean Svay Child Survival (Follow-On) Project First Annual Review, World Vision Cambodia, October 2001. Population statistics of Kien Svay Operational District in Year 2001-2004. WVC￾KSCSP. Monthly Reports, FY 2001, WVC-KSCSP. Monthly and Quarterly Reports, FY 2002 (Oct. 01 – Aug. 02), WVC-KSCSP. Monthly Reports, Compiled for FY 2003, WVC-KSCSP. Main Activities of Monitoring in Leuk Dek District, Kean Svay, Kandal PHD, in year 2001-2003. WVC/KSCSP. Report of Supervision of Health Centers and Health Center staff. Kampong Thom. Kampong Thom Provincial Health Department and WVC/KSCSP. KSCSP 2003. USAID/GH/HIDN/NUT Child Survival and Health Grants Program. Guidelines for Final Evaluation, July 2003. Community-based Disease and Death Surveillance: System Operation, Verbal and Social Autopsies. Consultancy report, Sophal Oum. May 2002. Health Status Indicators for World Vision Cambodia’s Area Development Programs. National Health Program, WVC, February 2002. Training reports FY 03. Training reports 2002. Knowledge, Practices and Coverage (KPC) Survey Report. January 2002. Assessment of Village Health Volunteer performance, KSCSP, August to November 2003. Report prepared by Dr. Chea Samnang, Department of Rural Health Care. Ministry of Rural Development. KSCSP, 2003. 55 Appendix D Final Evaluation Schedule Time 8 Dec (Monday) 9 Dec (Tuesday) 10 Dec (Wednesday) 11 Dec (Thursday) 12 Dec (Friday) 13 Dec (Saturday) AM FE Team meeting at the KSCSP Office --Expectations --Methodology Presentation of Group Work Facilitators: Finalizing the field visits tools based on the Group Work outputs Field Visit Leuk Dek Field Visit Kg Thom Field Visit Leuk Dek Field Visit Kg Thom Presentation of preliminary CBDDS data analyses (Dr Oum Sophal) PM Group Work: Identification of FE Questions and data sources Group Work: Formulation of Field Visits methods and tools Kg Thom Core Group meeting Field Visits Reports Field Visits Reports Compilation of field visits reports Time 15 Dec (Monday) 16 Dec (Tuesday) 17 Dec (Wednesday) 18 Dec (Thursday) 19 Dec (Friday) 20 Dec (Saturday) AM Group Work: Analyses of Field Visits and answers the FE Questions Group Work, Continued Presentation of Group Work Preparations for Lessons Learned Workshop, continued Lessons Learned Workshop (see agenda) PM Group Work, Continued Presentation of Group Work Meeting with Dr Oum Sophal on CBDDS. Preparations for Lessons Learned Workshop Debriefing meetings with WVC Senior Management and with USAID/Cambodia Lessons Learned Workshop, continued 56 Appendix E Timeline of project implementation Selected FY01 FY02 FY03 FY04 Project Activities O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F KPC surveys x x x x DIP wrkshp/review/approval x x x FAR / MTE / FE x x x BCC/CBDDS/VHV Consult. x x x x CBDDS x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x COPE orientation/follow up x x x x x x x x x x x x x x x Training Community—Leuk Dek VHVs x x x x x x x x x x x x x x x x x x x x x x x x TBAs x x x Drug sellers x x x Traditional healers x x x Community—Kean Svay VHV x x x x x x x x x x x x Health Center—KS OD x x x x x x x x x x x x x x x x IMCI x x x Health Center—KPT AD x x x x 57 Appendix F Update on MTE recommendations This Update on the MTE recommendations was prepared by the WVC and KSCSP staff prior to the FE, and edited afterwards for inclusion in the FE report. MTE Recommendation – September 2002 Final Evaluation update – November 2003 Technical Approaches 1. Immunization 1.1 The KSCSP should review the available data in the ADP household survey to assess their adequacy as baseline to measure the impact of the project. Secondary analyses should be conducted if necessary for this purpose. This information will be critical to compare the effectiveness of the three models of integration of child survival into ADPs. FE KPC survey tools thoroughly reviewed – all relevant CATCH questions included. MoH IMCI Mother Card used as basis to review questions on danger signs and health seeking behavior. Based on experience during these surveys, further changes will be made in relation to questions on danger signs in general and specific for diarrhea and ARI. Health module of HHS for ADPs reviewed with decisions on core, recommended and optional questions with standardized Khmer language questions and responses. However Kompong Thom ADP HHS proceeded prior to this review taking place. 2. Nutrition 2.1 Given the high levels of undernutrition demonstrated by the Leuk Dek KPC survey, the KSCSP should assess the capacity of health centers staff and the VHVs to identify malnourished children (growth monitoring) and provide adequate nutritional education and counseling. Quarterly Growth Monitoring by HC staff supported by VHVs has not been possible as HC staff find this very time consuming. However, opportunistic screening is done. The KSCSP has not had the capacity and resources to focus on the important area of child nutrition, although there have been specific micronutrient interventions and ongoing nutrition health education. Other CS partners in Cambodia, who have ongoing CS projects, are better positioned to address this recommendation and document lessons learned and best practice. 2.2 Given the low levels of timely introduction of complementary food, the KSCSP should assess the related knowledge and practices of mothers of children under 2, and those of the health care providers. The FE KPC surveys included some questions on foods given and frequency of feeds as quantitative data. No complementary qualitative research has been done by this project in the last 12 months. However, as for 2.1 above, this would be a strategic role for continuing CS projects. 58 MTE Recommendation – September 2002 Final Evaluation update – November 2003 2. 3 The KSCSP should measure length in future anthropometric surveys to distinguish between acute and chronic malnutrition. No anthropometric measurements were made during the three FE KPC surveys for logistical reasons. The WV Partnership Transformational Development Indicators (TDI) include height and weight to be measured in all ADPs once every three years as a minimum. When this is done in Leuk Dek (2005) the results can be compared to the baseline data obtained from the KSCSP KPC survey done in January 2002. 2.4 The validity of the indicator of self-reported use of iodized salts should be assessed before using it as a project monitoring and evaluation indicator. All salt was tested in all households included in the sample for all three last KPCs. 3. IMCI 3.1 The KSCSP should support the IMCI orientation and training of the Kean Svay OD staff, and begin adapting the related training of the HC staff and VHVs. This should be done in coordination with the National IMCI unit of the MOH and the OD staff, and by using the 3 WVC staff already trained in IMCI. Approval for this recommendation was obtained from MoH and WHO, and case￾management training plan and budget for OD and HC staff were revised accordingly. IMCI training plans are in process of implementation – this was one specific use of the No-Cost Extension funds. Leuk Dek HC staff received IMCI training in June 2003 with follow-up in October. Due to the unavailability of IMCI trainers and the dates for the FE, IMCI training will commence from 22 December 2003 for HC staff from Kean Svay and Lovea Em 4. Birth Spacing 4.1 The KSCSP should introduce a question in the next KPC survey like “What method are you currently using?” and another question to probe the actual protection status (when was the last injection? Are you taking the pill now? Etc). The question used in the KPC 2000 and 2002 should be maintained to make it possible to compare results from one survey to the other, and help understand what each indicator actually measures. Khmer language questionnaire clearly asks about current use. The additional recommended questions were not included because of concerns that some mothers who may be using contraception without the knowledge and consent of their husbands may feel threatened and not respond or respond incorrectly. Mother’s who receive Depo-Provera are given a card but the FE KPC survey did not ask to see this. In a future ADP HHS, we could test questions to verify contraceptive use by asking to see the OC packet or record of injection. An additional response to record use of the monthly “Chinese” pill was added in the last KPC. 4.2 The CPR should include mothers of children under 2 who are pregnant or who want children within 2 years since most of them would in fact benefit from birth spacing. It is possible to report data for both denominators. The SKIP instructions in the KPC questionnaire were not updated so that use of contraceptive methods was not asked for women who wanted another child in the next two years. 59 MTE Recommendation – September 2002 Final Evaluation update – November 2003 4.3 The KSCSP should explore the opportunities to diversify the contraceptive methods mix available to men and women in the project area. The program has reviewed the information on choices and increased this component of the BS program. The BCC intervention to increase men’s use of condoms for birth spacing as well as birth spacing refresher training has attempted to meet this recommendation. KPC data analysis gives an indication of any changes in diversity of methods used. 4.4 The KSCSP should identify and document the key determinants of the success of this intervention (high demand, mode of service delivery, community support, etc) before the end of the project. Good progress was made to conduct this study with NIPH, but late October they advised that they would not be able to do it. At present we are searching for another consultant but may decide to conduct a one-day workshop with carefully selected participants in January 2004 to meet this recommendation. Cross-Cutting Approaches 1. Community Mobilization 1.1 The KSCSP should continue supporting the HCMC and FBC to ensure they reach the capacity to operate as per the MOH Guidelines after the end of the project. MoH have released final Guidelines (with name changes! Feedback Committee now Village Health Support Group). Four HCMC in Leuk Dek have been meeting monthly since September 2003 with minimal financial support from the project. VHSGs (VHVs) join these HCMC meetings. 1.2 The KSCSP should continue supporting the Mother’s Group and Children’s Groups. Mother’s Groups meet regularly when HC outreach occurs with no financial support provided by the project. Most of the time these are small groups of mothers rather than large groups, as many mothers are busy with other activities. 1.3 The KSCSP should consider creating and supporting Men’s/Father’s Groups and Family Groups. No father’s groups have been formed because it is very difficult to gather fathers for regular meetings due to their absence for work. However, during the last 12 months there has been focus group discussions for men only in all 24 villages to discuss birth spacing, especially condom use. 1.4 WVC should conduct a formal evaluation of the performance of the VHVs before the end of project to draw lessons about the support environment (HC, HCMC, FBC), training received (methodology, contents, effectiveness), and likely impact on the behavior and health status. This evaluation should be conceived as a component of the cost effectiveness study of the various models of integration of child survival into ADPs. Dr Chea Samnang, Ministry of Rural Development, was contracted to do this evaluation. The Final Report was delivered on 8 December 2003 and formed an important part of the Final Evaluation. This study should link to the Perceptions of VHV studies conducted by Barbara Main in 2000 (see summary in First Annual Review). 60 MTE Recommendation – September 2002 Final Evaluation update – November 2003 2. Behavior change communication 2.1 WVC should ensure that the whole process of developing and implementing at least the two selected behavior change strategies is completed and documented by the end of the project. This will serve to build the capacity of WVC and partners to apply this valuable approach in other projects. Ongoing activity for men’s involvement in birth spacing and promotion of colostrum feeding. The BEHAVE framework has been used to “retro-fit” the BCC plan developed with the BCC consultant from Johns Hopkins in December 2001. The two key BCC strategies have been documented and the FE KPC surveys included some indicators to measure progress. 2.2 The KSCSP should carefully assess the need for IEC materials for VHVs and HC, presumably, on the basis of well-defined behavior change strategies. During the last 12 months the project has not developed any new IEC material but has purchased IEC material from the NCHP and used material given by HKI and UNICEF on IMCI, nutrition, Vitamin A, IDD and breast feeding. 2.3 The KSCSP should quantify the behavioral change activities at the community level. See response to 2.1 above. KPC data measures progress towards behavior change. The results of the focus group discussions with men on contraceptive use, especially condoms, provide complementary qualitative information. 3. Capacity Building 3.1 The KSCSP should use one of the Health Officers in Leuk Dek as an ADP Health Coordinator for the remaining of the project to facilitate the transfer of the current health activities, expertise, and responsibilities of the KSCSP. WVC should encourage the ADP and KSCSP staff to plan and conduct activities, and ensure that the ADP staff has the opportunity to benefit from the KSCSP training in health and child survival, as appropriate. One Public Health Nurse has already applied for the position of ADP Health Manager in Kandal-Takeo, but the recruitment process imposed that other candidates are considered too before the final decision is made. Handover Plan for the 3-4 communes where the ADP is working has been developed in partnership with OD and Leuk Dek ADP through regular meetings. Some delays caused by the fact that the Leuk Dek Project Manager moved to another position in WVC and a replacement Manager had to be recruited and orientated to the KSCSP before she could have relevant input into this plan. 3.2 WVC should ensure that at least some continued funding will be available to the Leuk Dek ADP for 1 or 2 years after the end of the project to cover critical activities like training of VHVs and HC staff. No-Cost Extension for 4 months obtained from USAID. Unsuccessful with search to seek additional funding from February 2004, but Leuk Dek ADP will support two of the four health centers. 61 MTE Recommendation – September 2002 Final Evaluation update – November 2003 3.3 WVC and KSCSP should plan for a complete analysis of the data collected during the supervision visits by the end of the project. This requires ensuring quality and consistency of the data collected, and recording any changes in the procedure that may affect the results. The data should be entered in a structured electronic database that allows statistical analyses. The emphasis of such analyses should be on assessing improvement in quality of care in the various models of implementation, and on assessing the value of the current supervision tools and data for improving and monitoring quality of care. A few key IMCI quality of care indicators have already been selected at MTE to replace the “good” “fair” “poor” categories. Raw data is available from the start of Phase 2 and can be entered into this database. In August 2003, the project’s M&E officer was successful in obtaining a position with the new M&E Unit of WVC. Dr Chandaramony, from the Kean Svay OD has been seconded to the project as M&E Officer since then. These two persons have prepared a database for data collected during supervision visits. The experience Dr Chandaramony gains in this position will significantly support sustainability of M&E activities following project closure. 3.4 The KSCSP should consider using the OCA tool with the HCs, HCMCs, FBCs, and VDCs. These assessments should be done now and before the end of the project to assess progress in the capacity of these organizations and learn about the use of these tools. These tools may complement or overlap those recently proposed by the Ministry of Rural Development. The OCA tool was used with the Kean Svay and Kompong Thom ODs as part of the FE and results compared to previous assessments. Current OCA tool is otherwise not suitable for community-based organizations (HCMC, FBC & VDC). The project has not been able to convince the ADP to use the tool proposed by the MRD to assess VDC capacity as this tool apparently requires prior training and there is still some uncertainly about whether this will become a valid and valuable tool endorsed by the MRD. 3.5 The KSCSP and the KS and KPT OD staff should continue regular supervision using the observation, exit interview, and record review tools with the primary objective of providing direct feedback and support to improve the quality of care in health centers. Ongoing activity. See 3.3 above - database set up. Mechanism for regular (quarterly) feedback by OD and ADP established so that this process can continue after the CS project ends. In Kg Thom feedback is provided immediately following the supervision visit. The value of this feedback was assessed during the FE through interviews/ FGD with HC staff and OD. 3.6 WVC should introduce Problem Solving methods and skills among KSCSP and partners staff. PHD (1), OD (3) and KS Project Manager has attended COPE Facilitator training in September 2002 and developed COPE training plan for KS OD Health Centres. COPE has then been operating successfully in all 4 HC in Leuk Dek for over one year, and has been extended to 4 HC in Lovea Em and 5 HC in Kean Svay. Reports was provided during the FE and the value of the process towards quality of care was assessed in interviews/ FGD with HC staff and clients. 3.7 WVC and KSCSP should assist the KS OD in the definition of a few indicators of quality of care and client satisfaction with the health services. The indicators selected following the MTE have been routinely collected and entered into the project database with feedback as described in 3.5 above. 62 MTE Recommendation – September 2002 Final Evaluation update – November 2003 3.8 KSCSP should ensure that the appropria te information and reports about VHV activities reach the KS OD office. A two-track mechanism has been established after the MTE: one report to the OD and copy to the project. 3.9 The KSCSP and KS OD should continue improving the referral system, for instance starting with the establishment of a referral letter. Other CS partners have some experience in this area but not much success. There is a MOH approved letter of referral which is used by HC staff but often not respected by the receiving facility. The moto-ambulances are used for referral but often only as far as the main highway when people transfer to alternative transport to travel to Phnom Penh. Follow-up after referral depends on strong but informal relationships between HC staff, VHVs and the local community. 4. Sustainability 4.1 The KSCSP staff should prepare formal Exit Plans with the OD and each HC partners. See Capacity Building 3.1 above. 4.2 The KSCSP should assist the new HCMC and FC (now VHSG) in improving the cost-recovery mechanisms in the HC. See Community Mobilization 1.1. Training in cost-recovery following MOH guidelines was provided by the Leuk Dek OD during the establishment of the HCHC and VHSG. Two HC have a formal cost-recovery mechanism and the other two will soon move from an informal to a formal mechanism. 4.3 The KSCSP and the OD and HC staff should explore possible use of the new MED program in Leuk Dek to provide alternative sources of funding for health. The links between the CS project and MED program are not strong but the Leuk Dek ADP will have primary responsibility for facilitating integration of MED and health activities. 4.4 The KSCSP should assist the OD and HC staff in creating partnership with private providers and increase their involvement in provision of quality CS services. One training for drug sellers, one for traditional healers and one for TBAs have been conducted during the year. The pre- and post-test results are in the project Training Reports. There has not been regular formal monitoring of the impact of this training. Village “corner-stores” identified as an important source for some medications but their training not in the project plan. One key area to address would be the private practice of public health providers. This issue faced by all CS partners. There are plans to review of available Cambodian literature to help direct action. 63 MTE Recommendation – September 2002 Final Evaluation update – November 2003 Program Management 1. Planning, monitoring and evaluation 1.1 The KSCSP should carefully document the nature, quantity and cost of the inputs provided to the outreach and community levels activities because these are probably an important determinant of the effectiveness of the four KSCSP models of implementation. This depended on the outcome of the workshop proposed in 1.3. 1.2 WVC should begin planning for drawing lessons learned, including the cost-effectiveness study and the final evaluation The key lessons learned of the KSCSP were presented during a one-day Lessons Learned Workshop at the end of the FE. 1.3 WVC should organize a study design workshop with the MOH, USAID, and selected interested partners to define specific objectives, data needs, and analysis plan to estimate the cost-effectiveness of the different models of implementation of child survival. This protocol should be developed soon enough to allow primary data collection before the end of the project if needed. Shortly after the MTE an ex-WHO consultant was approached with a request for initial input. After repeated promises and delays, this person advised he would be unable to help at this initial step. At the same time, the project staff was significantly affected by a cash-theft situation, the M&E officer moved to another position within WVC and the Project Manager took extended leave. Other project priorities, in addition to regular planned activities, were the reviews of the CBDDS, VHVs and Birth Spacing. There have been ongoing discussions with WVUS about a possible intern who could contribute to this assessment but the first steps of identifying what we could realistically do with the data we already have has not occurred. 1.4 WVC should plan the various KPC and other assessments in the various project areas on the basis of the cost effectiveness study protocol. See 1.3 above 1.5 WVC should continue developing its participatory review methodology by defining the specific problems to be solved, analyzing the potential sources of biases in the results, and allocating more time to the preparation and analysis stages The Staff Development Unit of WVC has recently been strengthened and past performance thoroughly evaluated. Methodology to enhance PLA knowledge and skills is an important part of all WVC projects, especially ADPs, and will be addressed through the SD Unit. 64 MTE Recommendation – September 2002 Final Evaluation update – November 2003 2. Human resource development and management 2.1 The KSCSP professional staff should take the lead in the completion of the project as planned, and in particular in the transfer of activities to the MOH and ADP, in the design and implementation of the cost effectiveness study, and the preparation of the final evaluation and lessons learned workshop in 12 months. WVC should provide technical and managerial assistance as needed. The OD, Leuk Dek ADP and KS project staff have led the process of handover/ phase-out of child survival activities. The HIV/AIDS and Health Program of WVC has provided technical and management support as needed. However, in early November 2003, the Project Manager took extended leave to be with her ill mother in NZ. The HAHP has therefore increased technical and management support in preparation for the FE and project closure. 2.2 WVC should discuss with the KSCSP staff the options of continued employment with WV at the end of the project. M&E Officer employed in M&E Unit of WVC. Finance/Admin staff dismissed following cash theft. Leuk Dek ADP Health Coordinator an option for other staff. Two drivers encouraged to apply for positions in a new HIV/AIDS project. A Mentoring CS project with the American Red Cross which would have offered employment to the three public health nurses was unsuccessful. URC in Cambodia have indicated that they want a partnership with WVC for urban RCH-HIV/AIDS which may provide employment for other project staff– a concept paper and budget for an integrated maternal health project in Phnom Penh has been submitted to URC and their response is awaited. 3. Information management 3.1 The KSCSP should ensure that the CBDDS system is fully operational in the Leuk Dek district by the end of the project. Dr Oum Sophal, Deputy Director General of Health and former Director NIPH, has continued to oversee the process in relation to Verbal Autopsy (completed in Khmer) but less progress has been made on the Social Autopsy form (not yet completed). The CBDDS system has been operating in Leuk Dek since June 2001 with improvements made based on Dr Oum Sophal’s recommendations. 65 MTE Recommendation – September 2002 Final Evaluation update – November 2003 3.2 WVC and the KSCSP should undertake a full analysis of the CBDDS data available at the KSCSP office before the end of the project to first calculate coverage and completeness rates at the village level, and then mortality and birth rates, and maybe other rates and trends. This analytical effort should aim at drawing conclusions about the health status and health services of the Kean Svay and Leuk Dek ADs, if possible, but also at conclusions about the feasibility and usefulness of using CBDDS data at the HC or district levels. The potential use of aggregate data at the HC and district levels does not diminish the importance of its use at the HC and community levels. Dr Oum Sophal has agreed to support the Final Evaluation Team Leader in assessing completeness of data and calculating key rates if appropriate. Dr Oum Sophal has been coordinating data entry leading up to the FE. 3.3 WVC and the KSCSP should ensure that the data collected on a routine basis or during special studies is fully used for programmatic purposes or for development of tools or methods. This may require planning more time for data analysis, interpretation, and dissemination. It may also require to further develop the internal capacity to do so, or to procure external assistance when needed. The use of routine data for programmatic purposes has been strengthened since the MTE. This was assessed during the FE. 3.4 WVC and KSCSP should consider the various changes or additions to the KPC questionnaires as suggested above (age group for vitamin A; denominator for CPR; measures of length) Done - see response to a number of earlier recommendations. 66 Appendix G Lessons Learned Workshop Agenda 19 December 2003 Intercontinental Hotel, Room #1 Time Topic Responsible Person 800-0830 Arrival and Registration 830-0900 Welcoming Speeches Welcome and Introduction USAID Child Survival Projects – global and national Opening of the workshop Mr. Talmage Payne, Country Director WVC Mr. Mark White, Office of Public Health/ USAID Dr Tan Vuoch Chheng, Vice-Director, National Maternal and Child Health Centre 0900-915 Introduction to the workshop Dr. Marc Debay, Team Leader of the KSCSP Final Evaluation 0915-1015 Project Overview and Key Results Dr. Douglas Shaw, HIV/AIDS and Health Advisor, WVC 1015-1045 Coffee Break 1045-1200 Lessons Learned 1: Community Based Death and Disease Surveillance Discussion Dr. Ly Vanthy, HIV/AIDS and Health Program Manager, WVC 1200-1330 Lunch 1330-1500 Lessons Learned 2: Partnerships and Capacity Building 1. Community level 2. Health Systems level Discussion Dr Chea Samnang, Director, Department of Rural Health Care, Ministry of Rural Development Dr Sour Kim An, HIV/AIDS and Health Assistant Program Manager, WVC 1500-1530 Coffee Break 1530- 1700 Lessons Learned 3: Comparison of different models of child health services delivery Discussion Dr. Marc Debay, Team Leader of the KSCSP Final Evaluation 1700-1715 Close of the workshop 67 Appendix H Lessons Learned Workshop List of Participants No. Name Sex Organization 1 Mr. Myles Harrison M WVC 2 Ms. Kerry Anne McKenzie F WVC 3 Dr. Douglas Shaw M HAHP, WVC 4 Dr. Ly Vanthy M HAHP, WVC 5 Dr. Sour Kim An M HAHP, WVC 6 Mr. Seng Choun Leng M HAHP, WVC 7 Mr. Mom Kim Yoeun M HAHP, WVC 8 Dr. Srey Mony M STAR 4/5 Project, WVC 9 Dr. Mak Munint M MAAP, WVC 10 Mrs. Huy Dany F KSCSP, WVC 11 Mrs. Hout Sokda F KSCSP, WVC 12 Mrs. Kan Cham Roen F KSCSP, WVC 13 Dr. Pen Sophea M KSCSP, WVC 14 Dr. Yim Yaren M Kg. Thom OP, WVC 15 Dr. Te Chilay M Kg. Chhnang, WVC 16 Dr. Seng Sokun Theary F Kg. Thom, WVC 17 Dr. Pek Kimsan M WVC 18 Mr. Keo Serievuth M WVC 19 Mr. Rommel Caringal M MED, WVC 20 Ms. Lim Chakrya F PDD, WVC 21 Mr. Reinhard Tietze M WVC 22 Mr. David Grosz M WVUS 23 Ms. Esther Indrinai F WV Indonesia 24 Dr. Marc Debay M F.E Team Leader 25 Mr. Heng Bunsieth M CRS 26 Ms. Michelle Lang F HKI 27 Ms. Sivan F World Relief 28 Mr. Geof Bowman M World Relief 29 Dr. Chuck Lerman M ARC 30 Dr. Meas Pheng M ADRA 31 Mr. Sin Samai M ADRA 32 Dr. Sieng Nam M UNICEF 33 Dr. Chhin Lan F UNICEF 34 Dr. Gita Pillai F URC 35 Dr. Sin Somony F Medicam 36 Dr. Severin von Xylander M WHO 68 No. Name Sex Organization 37 Dr. Chea Samnang M Ministry of Rural Development 38 Dr. Ung Vibol M NPH 39 Dr. Hong Rathomony M CDC, MoH 40 Dr. Prak Phan M Kandal PHD 41 Dr. Chea Eang Mao M Kandal PHD 42 Dr. Va Loung Khan M Kg. Thom PHD 43 Dr. Tout Bunnareth M Kg. Thom OD 44 Dr. Ou Chandara Mony M Kean Svay OD 45 Mrs. Hang Setha F Loek Dek ADP, WVC 46 Mr. Yi Sophornnatith M Kean Svay OD 47 Mr. Bin Samreth M Kean Svay OD 48 Mr. Lich Bunthoeuth M Care 49 Ms. Nay Chamnap F PDD, WVC 50 Dr. Tan Vouch Chheng F NMCH 52 Dr. Reginald Xavier M Save the Children Australia 53 Mr. Meas Sambath M Save the Children Australia 54 Ms. Jesse Rattan F Care 55 Mr. Bun Sok M USAID-Cambodia 56 Ms. Frances Tain F WHO 69 Appendix I VHV Knowledge assessment 0% 20% 40% 60% 80% 100% Oct-01 Apr-02 Oct-02 Apr-03 Oct-03 DHF CDD ARI 0% 20% 40% 60% 80% 100% Oct-01 Apr-02 Oct-02 Apr-03 Oct-03 Tet EPI 6 diseases 0% 20% 40% 60% 80% 100% Oct-01 Apr-02 Oct-02 Apr-03 Oct-03 IDD Vit A 0% 20% 40% 60% 80% 100% Oct-01 Apr-02 Oct-02 Apr-03 Oct-03 Birth Spacing