Northeast Cambodia Child Survival Program Chhlong Operation District, Kratie Province, Cambodia MID-TERM EVALUATION PARTNERS FOR DEVELOPMENT United States: 1320 Fenwick Lane, Suite 406, Silver Spring, MD 20910 Tel: 301-608 -0426, Fax: 301-608-0822, E-Mail: pfdinfo@partnersfordevelopment.org Cambodia: No. 24, Street 294, BP 551, Phnom Penh, Kingdom of Cambodia Tel./Fax: 855-23-213-335, E-Mail: PFD.Cambodia@ BigPond.com.kh by: Sheryl Keller, MSN, MPH Som Sophy, MRD-Rural Health October 2002 Phnom Penh, Cambodia ACRONYMS ANC Antenatal Care ARI Acute Respiratory Infection CIDA Canadian International Development Agency CMR Child Mortality Rate COPE Client-Oriented, Provider-Efficient CS Child Survival CSTS Child Survival Technical Services DHF Dengue Hemorrhagic Fever DIP Detailed Implementation Plan EC European Community EOI Emergency Obstetric Interventions EPI Expanded Program of Immunization GI Gastro-intestinal HC Health Center HIV/AIDs Human Immune Deficiency Virus/Acquired Immune Deficiency Syndrome HKI Helen Keller International IEC Information, Education and Communication IMCI Integrated management of Childhood Illnesses IMR Infant Mortality Rate LQAS Lot Quality Assurance Sampling MOH Ministry of Health MOU Memorandum of Understanding NCCDP Northeast Cambodia Community Development Program NCCSP Northeast Cambodia Child Survival Program NCRHP Northeast Cambodia Reproductive Health program NERP Nutritional education Rehabilitation Program NGO Non-governmental Organization OD Operational District ORS Oral Rehydration Solution ORT Oral Rehydration Therapy PFD Partners for Development PHD Provincial Health Department PSI Population Services International RACHA Reproductive and Child Health Alliance RH Reproductive Health TBA Traditional Birth Attendant TOT Training of Trainers TT Tetanus Toxoid UNFPA United Nations Family Planning Association UNICEF United Nations Children’s Fund USAID United States Agency for International Development VAC Vitamin A Capsule VDC Village Development Committees VHV Village Health Volunteer VSO Voluntary Service Overseas WHO World Health Organization TABLE OF CONTENTS Executive Summary……………………………………………………………. i I. Background A. Program Context…………………………………………………………. 1 B. Program Description 1. Initial Design…………………………………………………………… 3 2. Changes and Adaptations to Program Design…………………….. 4 II. Evaluation Methodology 6 III. Findings A. Assessment of Program Design………………………………………. 7 B. Implementation Progress 1. Status of Planned Activities, Use of DIP………………………….. 9 2. Field Observations…………………………………………………… 11 C. Assessment of Program Management 1. Program Structure……………………………………………………. 18 2. Staff Development……………………………………………………. 19 3. Collaboration with Other Organizations………………………….… 20 4. Use of Technical Assistance………………………………………… 22 D. Assessment of Program Impact………………………………………. 22 IV. Discussion and Conclusions A. Overview………………………………………………………………….. 26 B. Village Health Volunteers……………………………………………….. 26 C. Health System Strengthening………………………………………….. 29 D. TBA Training……………………………………………………………… 29 E. Program Management and Staff Development……………………….. 29 F. Nutrition Interventions……………………………………………………. 30 G. Monitoring and Evaluation……………………………………………….. 31 H. Sustainability………………………………………………………………. 32 V. Recommendations…………………………………………………………. A. Overall Recommendations 33 B. VHV Activities 33 C. Health System Strengthening 35 D. Program Management 36 E. Monitoring and Evaluation 36 F. Other: 37 Annexes: Scope of Work………………………………………………………………… 38 Documents Reviewed………………………………………………………… 42 List of Persons Met…………………………………………………………… 43 Analysis of Mortality Data……………………………………………………. 45 i Executive Summary A partner for Development (PFD), an international NGO specializing in grassroots community-level interventions in remote and/or underserved areas, has worked in the isolated, impoverished northeastern provinces of Cambodia since 19921 . PFD was awarded a 42-month USAID Child Survival Program Grant from October 2000 to March 2004. The goal of the Northeast Cambodia Child Survival Program (NCCSP) is to improve the health of children under five in Chhlong OD of Kratie Province through control of diarrheal disease, immunization, improved nutrition and promotion of breastfeeding. The program’s primary strategy is to increase caretakers’ case managment abilities and promote improved health behaviors through health education conducted by a network of Village Health Volunteers (VHVs). Linkage with the health system, and a potentially sustainable system for supervision of VHV activities, is addressed through the use of government Health Center (HC) staff to both train and supervise VHVs. The program has also trained HC staff and private drug sellers in diarrheal disease management, and is pilot testing the “Hearth” model of nutritional rehabilitation and education. Complementing the NCCSP is an UNFPA/Ministry Of Health (MOH)-funded Reproductive Health Program, and a USAID-funded Malaria Program. In September 2002, the approximate mid-point of the Grant, PFD recruited two independent consultants to conduct an external evaluation of the Program. The Team reviewed Program documentation, met with Provincial and Operational District (OD) health officials, and visited Chhlong OD from 16 – 27 September 2002 to view activities firsthand. While in the field, the Team visited every HC in the OD, interviewed the HC staff responsible for supervision of VHVs, visited a village where the Hearth model nutrition program was being piloted, and visited 11 other randomly selected villages. In each village, 2 VHVs, the village chief or other member of the Village Development Committee (VDC), and two female villagers with children of the target age were interviewed. In each of the 11 villages visited, visual checks were made of drug sellers to see if they had ORS in stock. PFD had conducted a repeat household survey of key indicators in January 2002 and the raw data was made available to the Evaluation Team. In addition, statistics on VHV activities were available from the Program reporting system, as were statistics on births and deaths reported by the VHVs for their catchment areas. Verbal autopsy forms had been completed for all maternal and under-five deaths reported in 2001. The Evaluation Team Leader, with assistance from PFD staff clinicians, analyzed the verbal autopsy reports and assigned a probable cause of death (see Annex IV). These data, together with the more qualitative information gathered from direct observation and extensive interviews with the randomly selected VHVs and villagers, provided the Team with a solid base for assessment of the Program. The Team found that the Northeast Cambodia Child Survival Program has successfully implemented almost all planned activities, with a few remaining ones in an advanced stage of planning. The Program will clearly succeed in implementing all activities prior to 1 Prior to 1996, this was as a part of the US branch of AICF (International Action Against Hunger), from which PFD subsequently separated. PFD was incorporated as a separate NGO in 1996 and continued management of the former AICF Cambodia program. ii the end of the grant period except for three minor ones which have been deleted due to poor performance in piloting or low priority In addition, the Program will have implemented a significant number of additional interventions directly related to child survival and responsive to documented causes of child mortality and weaknesses of the health system. The primary strategy of empowering and motivating village women to adapt improved health behaviors through peer education from Village Health Volunteers (VHVs) has proven sound and is working well. VHVs are known and active in their communities and both qualitative assessments by the Team and quantitative data from a repeat household survey confirm that they have achieved dramatic success in increasing knowledge and changing behaviors. The ORT treatment rate has more than doubled since VHV activities began. Immunization coverage documented by card has increased from 14% to 63% -- considerably higher than the national average. Since the nutrition and breast-feeding interventions were not introduced until the first half of 2002, the mid￾term survey was limited to diarrhea and immunization. However, the Team found clear indications in the field that VHVs are conducting education on nutrition and breast￾feeding with at least as much intensity as the other intervention areas, and that mothers are retaining and acting on these messages. Village women and community leaders value the Program, as do Provincial and District health managers and the staff of health facilities. All HCs reported a noticeable increase in attendance for immunization since the Program started, and most also realized an increase in general utilization of HCs’ antenatal and curative services attributable to VHV referrals and publicizing of services. Key factors in the success of the NCCSP VHVs are: · Participatory introduction of the Program and VHV selection · Manageable workloads (1 VHV per 50 households) · High quality training and appropriate IEC materials · Close supervision and follow-up The supervision of VHVs by HC staff is working very well. VHV Supervisors are providing tangible assistance as well as less tangible but important morale boosts to the VHVs, and enjoy the supervision. It has broadened their awareness of the communities and given them a point of entry wherein they are assured of a welcome, and an intermediary who can help sort out issues that arise between facility staff and villagers. It has also made their work easier to do and increased their own performance in terms of HC utilization and immunization coverage. The time spent on VHV supervision and associated paperwork is small and does not seem to adversely affect staff’s ability to carry out their usual HC tasks. Only 2 staff per HC, spending a couple of afternoons per month, is required. While most VHVs are well versed in their current material and active, about 10% are not, despite Supervisor efforts, and no productive purpose is served by keeping them in the Program. Most VHVs have an adequate grasp of the topics they have been taught but a minority needs refresher training. iii Analysis of verbal autopsy data and interviews with VHVs, HC staff and villagers indicates both a need and potential for increasing the range of VHV interventions to address other significant causes of under5 mortality, e.g., acute respiratory infections (ARI), peri-natal causes, dengue/malaria, dysentery, and typhoid. While the IEC materials provided by PFD are excellent and being well used, the record￾keeping system for the VHVs could be improved upon. There is considerable duplication of information between the monthly report and the register, and the information is often inconsistent. In addition to recording events and activities in two different places (the register and the monthly report form) most VHVs first record it on another scrap of paper and then transfer the information to the forms at the end of the month. Although the VHV Program has been very successful, there is a clear consensus among VHVs, community leaders, VHV Supervisors and PFD national staff that something needs to be done to build in a small incentive for VHVs over the long run if the currently high performance is to continue. Preliminary results of the Hearth model look quite encouraging, but it is a very labor￾intensive undertaking. It will therefore be very important to have a solid base of data on improvements in nutritional status, which the Program is already poised to collect, and also data on mortality impact, which can be obtained through the VHV surveillance system. Improvements in health status require work at both the demand and supply side. Cambodia is unique in that, due to decades of civil war and the destruction of social institutions and human resources during the Khmer Rouge genocide, its public health system is less than a decade old. The initial Program design assumed that another NGO would be building capacity in the HCs, Referral Hospital and OD Management Team, and in general probably underestimated how weak the system was. The Program has made appropriate adjustments to provide support to HCs and this can be expected to improve the situation by the end of the grant period, but the needs of the Referral Hospital and OD Management Team remain largely unmet. Particularly urgent is the need for competent, reliably staffed, and equipped 24-hour obstetric facilities and an inpatient pediatric ward able to provide oxygen, intravenous antibiotic therapy, laboratory diagnostics, and management of DHF. While PFD is in general doing an excellent job of implementing a large program under difficult conditions, implementation could be further enhanced through a restructuring of Program Officer responsibilities so that one staff has primary responsibility for overseeing all health-related activities – VHV and health system strengthening, RH activities, and the malaria program – in a specific geographical area. This would save travel time and allow for a comprehensive view of village health activities. The quality, commitment and high morale of PFD’s national staff have been critical to the Program’s success. PFD has made a number of investments in developing the capacity of its local personnel, who make up the majority of the staff and are the direct implementers. The Program has done a good job of obtaining and using baseline data, and conducting repeat surveys to measure changes. A survey on nutrition and breastfeeding indicators is planned soon. PFD has made skillful use of in-country expertise through iv collaborations with other organizations, and has returned the favor by sharing materials and lessons learned. PFD has actively pursued partnerships and collaborations with government, other NGOs, and International Organizations and made skillful use of it in implementation of the Program. PFD has also actively sought other donor funding to support complementary or expanded Program activities. In addition, formal external assistance has been utilized in development of pilot tests and both qualitative and quantitative research The NCCSP is better placed for sustainability than most programs since its primary activity is the imparting of new information, skills, and behaviors to communities. When successful – as these appear to have been – such changes are intrinsically self￾sustaining as changed community attitudes and practices get handed down to subsequent generations. Since the VHV approach has proven so effective, however, it would be highly desirable for it to continue beyond the current end date of the grant. It is possible for this to happen without PFD, since the VHVs are already linked to, and under the supervision of, the government health system. However, it will take time before the nascent Cambodian health system has the managerial skills and financial resources to be able to sustain the VHV program. The best contribution PFD can make to program sustainability is to continue its efforts to strengthen the health system, and to create incentive mechanisms for VHVs that are ultimately self-sustaining through normal market forces (e.g. social marketing). The Evaluation Team strongly recommends that the Program continue -- keeping the basic structure of VHV selection by communities, one VHV per 50 households and supervision of VHVs by HC staff -- and expand, resources permitting, to the other OD of Kratie Province so that the entire Province is covered. The Team also recommends that USAID fund an extension of the Program after the scheduled end date of the grant in March 2004 in order to allow activities which are having a demonstrable impact on child health to continue, and help ensure permanent returns on its investment by supporting the Program until the health system is sufficiently developed to take it over. In addition, a number of specific recommendations are made for expansion and/or strengthening of the program, including: ß A phased expansion of VHV health education/referral to include ARI, pregnancy and delivery, malaria/dengue, dysentery/typhoid, skin infections, iodine deficiency and child safety. ß VHV social marketing of health commodities in order to provide a small financial incentive with long term potential for sustainability, while also increasing access to, and use of, desirable products. ß Revision of the VHV record-keeping system (registers and monthly reports) to be more accurate and user-friendly, and periodic village censuses to ensure that all eligible families are covered by the Program. ß Continued surveillance of maternal and under5 deaths, with revision of the verbal autopsy forms to capture more detail and to be completed by the same staff who will conduct the analysis and assignment of cause of death; careful monitoring of both anthropometrical and mortality data in the Hearth pilot villages. v ß Integration of program management so that specific PFD Program Officers monitor and supervise all activities in a geographical area: RH, VHV education, and malaria. ß A shift in focus by the PFD Water and Sanitation Program from water supply to environmental sanitation in Chhlong OD, in response to documented causes of mortality. ß Community health education using techniques such as drama or puppet shows on topics that require a broad community response to address, e.g. environmental sanitation issues. ß Replacement of the small number of VHVs who are not active, and targeted refresher training for those who need it. ß TBA training in interventions that do not require availability of emergency obstetric care, with particular attention to immediate care of the newborn to establish a patent airway and prevent hypothermia, hypoglycemia. ß If and when emergency obstetric care is available, specific referral and partnership mechanisms linking TBAs with the hospital, including arrangements, which allow the TBA to maintain a (non-medical) role during the hospital delivery. ß PFD national staff training in supervisory techniques and management skills, and gradual devolution of management responsibilities to them. ß Expansion of health system strengthening to include the Referral Hospital unless another agency is able to do so. 1 I. Background A. Program Context Partners for Development (PFD) is an International NGO specializing in grassroots community￾level interventions based on the principles of community organization. Technical areas of expertise include water and sanitation, maternal/child and reproductive health, and village agriculture/food security. All interventions are preceded by, and built upon, participatory rural appraisals and formation of Village Development Committees (VDCs). PFD specifically targets remote and/or underserved areas, and has worked in the isolated, impoverished northeastern provinces of Cambodia since 1992.1 Chhlong Operational District (OD)2 is located in Kratie Province of northeastern Cambodia. It has a total population of approximately3 126,000 people dwelling in 103 villages ranging in size from less than 100 to over 1,000 households. It is transected by the Mekong River, with the majority of its inhabitant’s subsistence farmers. Over half of all households are below the poverty line. 49% of adult women are illiterate, and most of those who can read, do so at only a primary level. 15% of the population are ethnic minorities, unusual for Cambodia, which is largely homogenous. Ethnic minorities in the area include the Cham (indigenous Muslims of Malay ancestry) and the Stieng (an indigenous hill tribe peoples). Two-thirds of the population is concentrated in villages along the banks of the Mekong, with the remainder in scattered, isolated inland villages. Roads are extremely bad, often impassable in the rainy season, and flooding is common. Greater than usual flooding has occurred for the past three years. Riverine villages can be accessed by boat during the rainy season, but inland villages are often accessible only by lengthy walks through deep mud or by oxcart. Within the general breakdown of riverine versus inland, there are further regional differences in Chhlong OD. Among riverine communities, those on the eastern banks of the Mekong are the most socioeconomically advanced, with a number of thriving markets. Villages on the western banks are less developed, with fewer markets and less commerce. A significant minority of riverine villages are ethnic Cham, and are unusually organized and cohesive by Cambodian standards. Inland, with the exception of one sizable town (the capital of Snuol administrative district), villages are scattered and remote, but differ considerably in livelihood and lifestyle. Some are largely self-contained rice farming communities wherein people remain in the village much of the time, camping out in their rice fields only during the peak agricultural seasons of planting and harvest, and thereby having considerable opportunities to meet. Other villages are engaged in year-round cash crop cultivation or work on rubber plantations, with the result that residents are rarely at home and may stay overnight for extended periods in their fields, resulting in infrequent contact and interaction among villagers. Some inland villages contain significant numbers of ethnic Stieng, who have markedly lower levels of literacy and education, and do not always speak Khmer, isolating them from the larger society and limiting their access to markets and other services. 1 Prior to 1996, this was as a part of the US branch of AICF (International Action Against Hunger), from which PFD subsequently separated. PFD was incorporated as a separate NGO in 1996 and continued management of the former AICF Cambodia program. 2 An Operational District is an administrative division designated by the Ministry of Health, covering a population of approximately 100,000 persons. It is the primary locus of health care services and headed by a District Management Team. 3 All population figures are based on the 1998 census adjusted for a growth rate of 2.49% per year since 1998. 2 The PFD Northeast Cambodia Community Development Program (NCCDP), funded by USAID/Cambodia since 1993, has conducted participatory rural appraisals, facilitated the development of Village Development Committees (VDCs), and assisted communities in installation of tube wells and latrines throughout the Provinces of Kratie and Steung Treng. As a result, the majority of villages in Chhlong OD have functioning VDCs and a source of potable water. Latrines are less common, especially in the inland communities. In addition to the infectious diseases common throughout Cambodia (respiratory infections, diarrhea, typhoid, immunizable diseases etc), much of Chhlong OD is highly endemic for malaria. Dengue, previously confined to urban areas., has recently spread into the rural areas of Cambodia and is a new problem in Chhlong OD. Schistosomiasis is also present and there have been cholera outbreaks in the past. Iodine deficiency is very common. Infant and child mortality are higher than the already-high national average, with an Infant Mortality Rate (IMR) of 97/1,000 and Child Mortality Rate (CMR) of 68/1,000 compared to 80/1,000 and 53/1,000 respectively for the country overall (1998 Census)4 Almost all deliveries occur at home, and are usually performed by Traditional Birth Attendants (TBAs). Malnutrition is widespread and undoubtedly contributes to mortality from other causes. A 1998 survey found that 25% of children 6-59 months were wasted (<-2.00whz) and 40% were stunted (<-2.00haz). Vitamin A deficiency is widespread, with night-blindness reported in 3.3% of children aged 24-59 months, and 4.0% of pregnant women. Due to the combination of inadequate iron intake (especially in the early weaning ages), parasites, and frequent malarial infections, anemia is extremely prevalent among young children in the NCCSP area: 81% have hemoglobin’s below the WHO cut-off of 11.0 g/dl. It is also extremely widespread among pregnant women (86%)5 . In keeping with the Ministry of Health (MOH) guidelines, the first level of health care in Chhlong OD are the Health Centers (HC) which provide preventive health services (immunization, antenatal care, family planning) and simple outpatient curative care. Immunization is conducted on an outreach basis for villages beyond walking distance while other services are facility￾based. HCs are open only in the mornings, Monday through Friday. There are 10 designated HCs in Chhlong. One of these, Snuol, covers a population size more than double the MOH standard (20,790 as opposed to a standard of 10,000 per HC). In addition, the catchment area of Snuol is exceptionally large in geographical terms, with some villages as much as 30km away from the HC. PFD has assisted the OD in constructing two “Health Posts” (smaller versions of HCs with fewer staff and less comprehensive services) in Snuol to increase accessibility. Even with these, much of the population are unable to access services due to the length of time and cost of travel. Further aggravating this problem is the fact that the communities furthest away tend to be those engaged in cash crop cultivation or employment in the rubber plantations, and thus unusually constrained in the time available to travel for health services. Of the 10 HCs, 9 meet MOH standards in terms of minimal number of staff and have been issued a full drug kit. One, the remote inland HC of Damrei Pong, has an inadequate number and quality (level of qualification) of staff, preventing it from receiving the full drug kit and 4 Mortality rates for the program area are the average for the province as a whole; OD-specific rates are unavailable. The 1998 census, due to the model used for indirect estimation, underestimated infant and overestimated child mortality rates as subsequently demonstrated in two national demographic and health surveys. However, since the latter sources do not provide data specific for Kratie province, census rates are used here to enable comparison between the program area and the country as a whole. 5 UNICEF-WFP Baseline Survey of Community Action for Social Development Program survey 1998. Data reflect Kratie and Steung Treng Provinces combined. 3 severely impeding its ability to provide services on a consistent basis. Of the 9 HCs which meet MOH minimal staffing standards, six are functioning at or above the national norm, i.e. are reliably open, providing services on a regular basis, and are being utilized by villagers for curative care, antenatal services and family planning. Three others have significant problems resulting in underutilization. Together with Damrei Pong this makes 4 out 10, or 40%, of HCs either not functional or poorly performing. The Referral Hospital is likewise weak with very low levels of utilization and virtually no deliveries talking place, despite the fact that it is the sole point of referral for obstetric emergencies in the OD. In 1998, PFD won a two-year entry-level Child Survival Program Grant from the USAID Private and Voluntary Cooperation (PVC) Bureau in Washington DC. for development of a child survival program in Chhlong OD. This grant ran from October 1, 1998 to September 30, 2000, during which time baseline quantitative and qualitative research was conducted, a Detailed Implementation Plan (DIP) designed, and partnership arrangements were finalized with the government and other NGOs. Based on the DIP, PFD was then competitively awarded a 42- month USAID Child Survival Program Grant from October 2000 to March 2004. B. Program Description 1. Initial Design The goal of the Northeast Cambodia Child Survival Program (NCCSP) is to improve the health of children under five in Chhlong OD through control of diarrheal disease, immunization, improved nutrition and promotion of breastfeeding. Specific objectives include: 1. Increase the proportion of diarrheal episodes which are appropriately managed at household level (ORT, recognition and referreal for danger signs, continued feeding during, and increased feeding after, diarrheal episodes); 2. Increase the preoportion of caretakers practicing hygiene behaviors that prevent diarrhea (hand-washing, dafe drinking water); 3. Increase the proportion of children who are fully immunized; 4. Increase the proportion of women of reproductive age immunized at least twice for tetanus; 5. Reduce the proportion of children under 3 years of age with severe or moderate malnutrition; 6. Reduce the prevalence of micronutrient deficiencies (especially Vitamin A and iron) in children 6-59 months old; 7. Increase the proportion of mothers exclusively breastfeeding infants up to 5 months of age; 8. Increase the proportion of mothers who initiate breastfeeding within one hour of delivery; 9. Increase the proportion of mothers providing appropriate weaning foods at age 6 months; 10. Increase the proportion of diarrhea cases in children under 5 which are correctly managed by health center staff or trained drug sellers; 11. Increase the capacity of health center staff to provide EPI services 4 12. Increase the capacity of TBAs to promote appropriate breastfeeding; and, 13. Increase the capacity of HC staff to promote breastfeeding, provde Vitamin A supplementation, identify malnutrition (growth monitoring) and provide appropriate nutrition counselling. The Program’s primary strategy is to increase caretakers’ case managment abilities and promote improved health behaviors through community health education conducted by a network of Village Health Volunteers (VHVs). VHVs are selected by communities in a partcipatory manner and responsible for approximately 50 households apiece. In addition to its primary task of educating mothers on improved care taking and preventive behaviors, the Program also planned to promote linkages between the community and the government health system and to increase the capacities of both public and private providers. Linkage with the health system, and a potentially sustainable system for supervision of VHV activities, was addressed through the use of HC staff to both train and supervise VHVs. The program planned to train HC staff and private drug sellers in diarrheal disease management, and to provide training and support (transport, cold chain equipment) for HC immunization activities. In addition to promoting improved breastfeeding and weaning practices through VHV health education, the program planned to pilot test three different approaches to improved nutrition: (1) education alone (6 HC catchment areas); (2) education plus growth monitoring (2 HC catchment areas); and (3) education plus implementation of the “Hearth” model of nutrition intervention, whereby a positive deviance study is conducted at village level followed by a Nutrition Education and Rehabilitation Project (NERP) in which the community is assisted in preparation and feeding of nutritious supplemental meals to malnourished children using locally available, affordable food sources. NERPs are accompanied by growth monitoring and intensive education of mothers, and were planned for 2 HC catchment areas. The Northeast Cambodia Child Survival Program (NCCSP) is primarily funded by USAID, with matching funds from a variety of donors, including the Canadian International Development Agency (CIDA), the Japanese Embassy, and the European Community (EC). The NCCSP is staffed by 1 Program Co-Ordinator (international), 1 VSO nurse (international, volunteer), 1 Team Leader (national), 9 Program Officers (national) and 2 administrative staff. 7 of the Program Officers are currently assigned to VHV community health education while the other two and the VSO nurse are assigned to strengthening of the health system. In addition, a national physician Child Survival Officer devotes 50% of his time to the Program, focusing on Health System Strengthening. The remaining 50% of his time is spent on the PFD Malaria Program, which is funded by USAID. Complementing the NCCSP is a UNFPA/MOH-funded Reproductive Health Program in which village-based agents sell contraceptive pills and condoms and provide IEC on birth-spacing and HIV prevention in 80 of the 103 villages of the OD (those within walking distance of a Health Center are excluded). In some cases these agents are also VHVs; in others, both PFD social marketing agents and VHVs are in the same villages. Three national staff are assigned fulltime to the RH program. 2. Changes and Adaptations to Program Design The overall goal, strategies, and primary activities of the program have remained unchanged, with only a few revisions, additions and deletions to planned activities as outlined in the DIP. 5 Revisions: In order to render the nutrition pilot more manageable in size and complexity, and to avoid the ethical dilemma inherent in identifying severe malnutrition without providing an adequate intervention, the “education plus growth monitoring” group was dropped, leaving a group receiving only nutrition education, and a group receiving the “Hearth” model – education, growth monitoring and NERPs. The Hearth group will be smaller than originally envisioned (2 HC catchment areas, which would equal approximately 20 villages) as it is extremely labor intensive. The actual number of villages which can be covered by Hearth before the end of the grant period remains to be seen, but looks to be in the range of 10 villages total, still sufficient for purposes of comparison. The establishment of physical “hand-washing corners” in households, originally planned as part of the VHV hygiene intervention, was revised after it encountered considerable community resistance and practical problems (lack of space in homes, many of which are one room shacks). Hand-washing remains an intervention but measured in terms of presence of soap and water within the home and reported/observed behavior. The cooking demonstrations originally planned by VHVs to promote appropriate weaning foods has been modified to have VHVs demonstrate the foods and explain their preparation, owing to the logistical difficulties of actual food preparation, and because the means of preparation (adding ingredients to rice porridge) is already well known to Cambodian mothers. Formal mother’s support groups for nutrition and breastfeeding have not been formed, since the concept of “support group” is alien to the culture and the VHV catchment areas (typically containing about 10 women with a child under age 2) already serve the same purpose in a more informal manner, gathering women together for group education and discussion. Additions: The level of involvement with HCs has been expanded beyond the originally envisioned training on diarrhea, nutrition and immunization. At the time of program design it was expected that another NGO would be working with the health system to strengthen overall management and service delivery in the HCs and Referral Hospital. When this did not materialize, and weaknesses in the health system proved a significant barrier to improving community health status, the program added a Health System Strengthening component, focusing on Health Centers. This initiative includes training and follow-up in Integrated Management of Childhood Disease (IMCI), training of HC staff in use of the “Client-Oriented Provider-Friendly” (COPE) tool to identify and solve service delivery problems, and technical assistance in management and service delivery for the four weakest HCs. Severe flooding in 2001 and 2002 created an emergency need for safe drinking water as usual sources were temporarily contaminated or submerged. The VHV network was used to distribute water purification tablets and ORS packets to households in flood-affected villages. Deletions: The following minor activities were dropped from the original design: 6 · a possible soap-making initiative was ruled out due to labor intensiveness and the availability of soap in the province. The program still plans to promote soap distribution, possibly by social marketing, with commodities obtained commercially or from an NGO. · a possible soy product-making initiative was discontinued after pilot testing was unsuccessful. · referral of malnourished children to HCs was dropped because HCs currently do not provide any type of nutritional rehabilitation. Neither does any other part of the Cambodian health system. Malnourished children who are ill are, of course, still referred for curative care. II. Evaluation Methodology In September 2002, the approximate mid-point of the Grant, PFD recruited two independent consultants to conduct an external evaluation of the Program. The Evaluation Team consisted of Sheryl Keller, MSN, MPH (Team Leader), and Ms. Sam Sophy, a specialist in rural development and primary health care. Ms. Keller is a public health professional with extensive international experience, including over 13 years experience in Cambodia. Ms. Sophy is a Cambodian national on staff with the Ministry of Rural Development’s Department of Rural Health. The Scope of Work for the evaluation appears in Annex I. The Team reviewed the DIP, baseline survey, annual report and other relevant Program documentation (see Annex II) and visited Chhlong OD from 16 – 27 September 2002 to view activities firsthand. While in the field, the Team visited every HC in the OD, interviewed the HC staff responsible for supervision of VHVs, visited a village where the Hearth model nutrition education/rehabilitation program was being piloted, and visited 11 other villages selected as follows: 1 village per HC catchment area for the 9 HCs which cover 7-11 villages each, and two villages from the one HC (Snuol) which covers a larger catchment area of 20 villages. Within the HC catchment areas, a village (or, in the case of Snuol, 2 villages) was selected randomly with probability proportional to size. In each village, 2 VHVs, the village chief or other member of the Village Development Committee (VDC), and two female villagers with children of the target age were interviewed. In villages with more than 2 VHVs total, VHVs were selected by random draw on the day of the visit, so that neither PFD, the HC nor the VHVs knew in advance whom would be interviewed. VDC members and village mothers were selected through convenience sampling but excluding any living in the immediate vicinity of the VHVs interviewed. Of the 22 randomly VHVs selected for interview, 20 were successfully located and interviewed; two others (each from a different village) where away at the time of interview. Of a total of 20 HC staff who supervise VHVs, 19 were interviewed and one was away for training. Village chiefs were successfully located for interview in 9 villages; in the remaining two, a VDC member and the wife of the village chief were interviewed as the village chief was unavailable. In each of the 11 villages visited, visual checks were made of drug sellers to see if they had ORS in stock. Those who did were asked about the levels of sales and receipt of any training. PFD professional staff were interviewed, and meetings were held with the Operational District Health Team in Chhlong and with the Provincial Health Department in Kratie to elicit government feedback on the Program. A full list of persons met appears in Annex III. PFD had conducted a repeat household survey of key indicators in January 2002 and the raw data was made available to the Team. In addition, statistics on VHV activities were available 7 from the program reporting system, as were statistics on births and deaths reported by the VHVs for their catchment areas. Verbal autopsy forms had been completed for all maternal and under-five deaths reported in 2001. The Evaluation Team Leader, with assistance from PFD staff clinicians, analyzed the verbal autopsy reports and assigned a probable cause of death (see Annex IV). These data, together with the more qualitative information gathered from direct observation and extensive interviews with the randomly selected VHVs and villagers, provided the team with a solid base for assessment of the Program III. Findings The Evaluation Team’s findings will be presented in four parts: 1) Assessment of Program design; 2) Implementation Progress, successes and constraints 3) Program Management: structure, staff development, collaboration with other organizations and use of technical assistance; and 4) Assessment of Program impact. A. Assessment of Program Design The Program has proven possible to implement largely as planned, with only minor changes. VHVs have been selected in every village, and have catchment areas of about 50 households each. Most VHVs are active and respected by their communities, and supervision of VHVs by HC staff is taking place as planned. The fundamental design of the Program is sound, has proven feasible to implement, and – as will be described subsequently – is having the desired impact. The Evaluation Team identified three issues with respect to Program design, as follows: 1. Time required to establish the Program 2. Expectations of the health system 3. Focus of VHV Interventions Timeline The original timeline for activities was over-ambitious given the need to select VHVs in 103 different villages in a participatory manner (thereby ensuring their credibility with mothers in the village), train PFD staff and HC staff in adult training methodology as well as course content, conduct courses for the VHVs in the four intervention areas, and train HC staff in supervision of the VHVs. All of this was originally projected to be completed by the end of 2001, whereas in reality it took until June of 2002. The main flaw in the original timetable was an under-estimation of the time required to assist villagers in selection of a VHV in 103 different villages scattered across a large geographical area, with very poor roads and frequent floods making access extremely time-consuming, and the time it would then take to train 400 VHVs in batches of 20 each. In addition, the amount of information that could be included in a single course proved to be less than anticipated. It was initially planned to combine both immunization and diarrheal control in a single module, but this proved impractical, especially since the first module also needed to include an explanation of the VHV role and record-keeping system. 8 The development of training and IEC materials suitable for rural Cambodia proved more time￾consuming than anticipated, as a total of five manuals and related IEC materials had to be identified or newly developed: VHV training manual for Immunization, VHV training manual for diarrhea, VHV training manual for nutrition / breastfeeding, VHV Supervision training manual, and TOT Training Manual for VHV Supervisors. An additional problem was that the possibility of some VHVs dropping out after the Program actually began, with the consequent need to recruit and train replacements, was not anticipated. 30 of the originally recruited VHVs dropped out shortly after receiving the first training, apparently because of unrealistic expectations regarding incentives and workload. Although over-ambitious expectations of the time required to fully establish the Program has resulted in many activities being about 6 months behind schedule, this is not likely to detract from achievement of Program objectives. As will be discussed shortly, survey data and observations in the field demonstrate that significant progress is being made towards achievement of all key objectives and mid-term targets have been reached or surpassed in most cases. Health System The originally planned level of involvement with the health system reflected both an overestimation of the capacities of the system and the expectation – reasonable at the time of Program design – that another NGO would be working with HCs, the Referral Hospital and the District Health Management Team. Unfortunately, that NGO was forced to withdraw from Chhlong due to an interruption in funding, and the capacities of the health system have proven to be much weaker than anticipated, creating a significant constraint. While the Program’s design in terms of working with communities and effecting positive changes in health status through demand-side interventions was sound, it may have underestimated the importance of addressing both demand and supply sides of the equation, especially in a country where the health system is still in very early stages of initial development. The Program has made necessary and appropriate adaptations to address weaknesses at Health Center level, including assignment of staff to work intensively with the weaker health centers, support for basic training of HC staff (the MOH “Foundation of Health Center Training course)”, support for the establishment of Village HC Feedback Committees, and, more recently, the planned introduction of Integrated Management of Childhood Illnesses (IMCI) in all HCs through support for IMCI training and intensive on-the-job follow-up. The latter is being undertaken in close coordination with the MOH IMCI working group and is part of a national pilot with the potential for informing nationwide policy. Still largely absent, and much needed, is support for the District Referral Hospital and strengthening of the capacities of Operational District Health Managers. There is some possibility of another NGO undertaking these starting early 2003, but funding is not yet certain. Interventions The intervention focus areas could be improved upon given causes of mortality, as shown in table 1 below. However, this detailed information, derived from the VHV registers and verbal autopsies, was not available at the time the interventions were selected. PFD was aware that Acute Respiratory Infection (ARI) was a major cause of death in children, but decided not to include it due to lack of a health system sufficiently strong to effectively handle referrals at the time the Program was initiated (see above). 9 Table 1: Under5 Deaths Chhlong OD May- December 2001 Probable Cause of Death Number Percentage ARI 43 25% Peri-natal 20 12% Meningitis/encephalitis 19 11% Malaria 17 10% Neonatal Tetanus 15 9% Typhoid/other infectious 10 6% GI Other 8 5% Accidents 7 4% Diarrhea 6 4% Dysentery 5 3% DHF 4 2% Malnutrition/Starvation* 3 2% Sepsis 3 2% Unknown 11 6% TOTAL 171 100% * Refers to malnutrition as the sole cause of death. Malnutrition as a contributing factor to death from infectious disease is not reflected and, based on known nutritional status in this population, can be assumed to be considerable. A detailed report, included methodology and criteria for assigning cause of death, appears in Annex IV. As can be seen from the above table, diarrhea is not a major cause of death among children under five in the Program area, whereas ARI, other infectious childhood diseases (including malaria), and peri-natal causes are quite significant. The relatively low contribution of diarrhea may reflect both the clean water interventions PFD had already conducted in the area (tube wells, household water filtration systems, and related health education) and the fact that Cambodian mothers traditionally continue or increase breastfeeding, and provide traditional teas, to children during diarrhea. While diarrhea has not proven to be a leading cause of mortality in the area, the diarrhea/hygiene interventions are still helpful. Improved hygiene will help reduce the incidence of dysentery and typhoid, and better feeding practices with diarrhea will reduce malnutrition which in turn is an important contributing factor to child deaths from infectious disease. There is sufficient time left in the Program to add on interventions that address what are now known to be more important causes of mortality, and the VHV infrastructure to absorb and implement such new interventions is solidly in place. B. Implementation Progress 1. Status of Planned Activities, Use of DIP The DIP is being actively used by Program staff to plan activities and assess progress. As noted in Section IIIA above, modifications to the DIP have been minimal and appropriate. The following table summarizes the status of all activities outlined in the DIP except for three minor activities noted in section IIIA above has having been deleted. 10 Table 2: Status of Planned Activities Activity Status 400 VHVs from 103 villages selected with village participation. Replacements trained for 30 VHVs who dropped out after the Program began Completed 20 staff from 10 HCs selected and trained as VHV Supervisors Completed Appropriate IEC materials identified or developed. (Visually￾literate immunization and diarrhea flip-charts were developed by PFD; a suitable nutrition/breastfeeding flipchart was obtained from Helen Keller International (HKI).) Completed Training of Trainers (TOT) in adult learning techniques, immunization, diarrhea, nutrition and breast-feeding provided to PFD staff and HC VHV Supervisors Completed VHVs trained in immunization, diarrhea, nutrition and breastfeeding Completed VHV registers and monthly report forms developed and training provided Completed VHV-led health education to village mothers on immunization, diarrheal control, nutrition and breastfeeding Ongoing in all villages VHVs mobilize mothers and children for immunization sessions Ongoing in all villages VHVs assist in community mobilization for VAC distribution campaigns Being done every six months in partnership with HKI Supervision by VHVs by HC staff Ongoing in all villages Drug sellers identified and ORS supply channels established Completed Drug sellers were trained to promote sale of ORS Completed (156 sellers) Hearth model piloted Underway in two villages, expansion to others planned HC capacities assessed Completed – HC Resource Survey and assessment of consultation practices HC staff trained in diarrhea management Completed in all HCs via the “Foundation for HC Training” course which includes diarrhea management HC training on cold chain maintenance and record-keeping Completed Establishment of Child Survival Coordinating Committee Completed and functional HCs provided with transport to conduct EPI outreach and VHV supervision Completed all HCs - motorcycle and reimbursement of fuel Follow-up survey to assess impact of VHV health education Done for immunization and diarrhea; planned end of this year for nutrition and breastfeeding TBA training on breastfeeding promotion Planned for 2003 ORS corners established in HCs Planned starting 4th quarter 2002 HC staff trained and encouraged to provide ANC iron supplementation Planned for 2003 Iron distribution by VHVs pilot tested Planned for 2003 11 As can be seen from the above, the overwhelming majority of planned activities have taken place and the Program is well poised to complete all before the grant end date. In addition to the planned activities, several others not originally in the DIP have been conducted or are planned, most of them related to strengthening of the health system. These include: · EPI outreach coverage assessment and identification of underserved areas (done) · Interventions to address causes of low EPI outreach in identified villages (done – per diem and transport support for villages far from the HC) · Introduction of the COPE self-improvement tool in HCs (done) · Formation of Village HC Feedback Committees (the official linkage between HCs and communities) (done by PFD in 3 HCs by PFD, the remaining 7 by another agency.) · Intensive technical assistance, facilitative supervision, and on-the-job training, in the 4 lowest-performing HCs (commenced and ongoing) · Introduction of IMCI in all 10 HCs as part of a national pilot (in preparation phase. PFD is actively participating in the national IMCI workforce and training of OD and HC staff will commence this year) · .Analysis of birth and mortality data from VHV registers (done for 2001). · Verbal autopsies for under5 and maternal deaths with analysis of probable cause (done for 2001, see Annex IV for full report). · Training of HC midwives in family planning, safe delivery, handling of obstetrical emergencies, post-partum care (planned for 2003). · Training of TBAs in safe delivery, recognition and referral of complications, and care of the newborn. · Social Marketing of safe birth kits (pilot). · In-service training on computer use for OD staff (done). 2. Field Observations VHV Activities PFD has recruited and trained a total of 402 VHVs, including 30 replacements for VHVs who completed the first training session but proved either unwilling or unable to actually function. At less than 10%, for a volunteer Program, that drop-out rate is not excessive. In all 11 randomly selected villages visited by the Team, community leaders6 were aware of the VHV Program and could identify the specific VHVs for their village by name. 10 out of 11 could describe the activities of the VHVs in detail; the eleventh knew only that it was some type of health education. Of the 22 mothers approached in the 11 villages (convenience sampling, but out of immediate range of a VHV’s house), all were aware of the VHV Program and could give the name of the VHV responsible for their household. All reported having been approached by the VHV and 20 of the 22 had received health education from her, while 2 stated they had been asked to attend but not gone due to being “too busy”. Above all, VHVs were associated in 6 These consisted of the village chief in 9 villages, a VDC member in one village, and the wife of the village chief in one village. 12 mother’s minds with the immunization program, as the person who let them know when immunization would take place and reminded them to bring their children. Mothers also recalled that the VHVs had flipcharts about child health. All but two could remember the subject matter, and most could describe the content in detail from memory. Although all of the 20 VHVs interviewed by the team were found to be active, the HC staff who supervise VHVs indicated that overall about 10% of the total are not functional (ranging from 0 – 20% per HC catchment area) due to low motivation or lack of time. This is not an unusually high percentage for a volunteer program in its second year, but worth addressing through replacement of the minority who are not able to function in their designated capacity. All community leaders interviewed expressed the opinion that the Program was valuable and important. Suggestions for the future included expansion and/or intensification of VHV activities (introduction of other intervention areas, particularly ARI, malaria/dengue) and provision of some type of incentive to VHVs to ensure continued motivation. Mothers described the VHVs as either “useful” or “very important”, with the main reason being that it helped ensure immunization for the children and increased villagers’ knowledge about health. Most mothers expressed a desire to have VHVs teach other topics, particularly ARI, dengue, malaria, and general management of a child with fever. Of the 20 VHVs interviewed without advance notice, (2 each from 9 villages and one in each of two others), 100% had IEC materials with them and all but one had her register of families on hand to show. In addition to possessing flip charts and brochures, a number of VHVs had health education posters prominently displayed on the walls of their homes. Based on their own descriptions and information gathered from community leaders and mothers in the village, all 20 were active as VHVs although the level of activity varied considerably. All VHVs were carrying out mobilization for immunization, and this activity was seen as the easiest to conduct since it was quite clear who to target and when a particular household no longer needed to be targeted. Health education presented more of a challenge since it was not linked to receipt of a service and there is no clear-cut beginning and end to the education of a mother. It appeared that all VHVs had initially conducted education on each of the topics (immunization, diarrhea/hygiene, and nutrition/breastfeeding) to their entire catchment area, or as much of it as they could reach, immediately after receiving the training. Since then, each has settled into her own routine which varies considerably by individual VHVs. At the lowest end of the spectrum, a few were conducting group education only on the days when the HC Supervisors come (once a month) and reaching only those who readily come forward to attend. At the high end of the spectrum were VHVs who systematically visited every household with a pregnant women or child under five monthly, and made use of special opportunities such as the birth of a new baby or an episode of diarrhea to review specific topics with the mother. VHVs varied in the approaches used and had largely developed them on their own, some being more creative in doing so than others. There is scope for learning from each other, but no single strategy applies to all since there are important differences in localities that affect how and when VHVs can reach mothers. In villages on the river, individual home visits are possible except during peak rice farming times (planting and harvesting), while some of the inland villages are engaged year round in cash crop cultivation and rarely at home, making it necessary to set up special appointments for group education sessions. VHVs in the more socioeconomically developed villages along the river expressed difficulty in repeatedly conducting education on the same messages and a need for a wider repertoire in 13 order to keep the mothers interested in attending. However, in remote inland villages where levels of education are minimal and the population have little contact with the outside world, VHVs reported that it took multiple sessions on the same topic for information to sink in. These perceptions of the VHVs were consistent with Team findings in discussions with mothers. In the riverine villages, particularly those on the more developed eastern side of the Mekong, mothers could readily rattle off the health education messages they had learned in great detail and took obvious pride in doing so. In inland villages, mothers had more difficulty and often could describe only the general topic without specifics; some needed probing to identify topics even though they had attended health education sessions with the VHV. VHVs also varied considerably in their grasp of the material and skill in teaching it. All could use the flip charts, which are well designed, visually-literate and appropriate to the rural conditions and culture. All VHVs in the riverine communities and some of those inland displayed a clear grasp of the material and expressed a feeling of confidence. However, about half of the VHVs in the inland areas said they had either forgotten or were unclear on some of the information. Mothers’ ability to describe what they had learned from VHVs tended to be consistent with the VHVs level of knowledge and confidence (see above). While there is likely some direct cause and effect between VHV skills and success in educating mothers, both are also the result of a common set of causes affecting these communities: extreme isolation and poverty, very low levels of socioeconomic development, lack of mass media, extremely low levels not only of education and literacy but also of basic ability to articulate ideas, and widespread anemia, iodine and other micronutrient deficiencies, which have a direct impact on the ability to conceptualize, convey information and remember it. While the IEC materials provided by PFD are excellent and being well-used, the record-keeping system for the VHVs could be improved upon. It consists of a register to identify target households and track pregnancies, births, deaths, and immunization status, plus a monthly report form on VHV activities (number of education sessions, referrals to the HC etc). There is considerable duplication of information between the monthly report and the register, and the information is often inconsistent. Some of the VHVs do not understand how to record (or perhaps how to calculate) immunization coverage. Most are recording pregnancies and births, but recording of deaths seems to have tapered off with time to the point that few were found on the register for this year (unlike 2001 where reported deaths were close to the number expected). VHVs were aware of the deaths in their catchment area of 50 households, and could readily provide that information but had in most instances not written it down. A likely reason for the better recording of pregnancies and births than of deaths is that the former information is used by the VHV herself while the latter is not, and the importance of recording death information has not been stressed by Supervisors. In addition to recording events and activities in two different places (the register and the monthly report form) most VHVs first record it on another scrap of paper and then transfer the information to the forms at the end of the month. The activity reports are collected monthly by the Supervisors and seem to be more complete than the registers, which are meant to serve as a tool for organizing VHV activities and provide annual data which can guide Program intervention. These purposes are not well understood by many VHVs. VHV Supervision 100% of the VHVs interviewed reported receiving supervision from HC staff, knew their supervisor by name, and described the supervision as being helpful and supportive, and occurring regularly once a month. Supervisors, for their part, were able to list by name the VHVs in any given village without advance notice, were clearly familiar with each individual VHV, and 14 knew which ones were most active and knowledgeable. The vast majority of VHV Supervisors seemed sincerely interested in and supportive of the VHV’s work. The good rapport between VHVs and their Supervisors from the HC is reflected in the extent to which each group advocated for the other in talks with the Team. HC staff made a point of raising the incentive issue and pleading on behalf of the VHVs, while VHVs made a point of stressing the good work being done by their Supervisors, and some were observed to speak in defense of the HCs with regard to problems or criticisms raised by villagers. Most VHVs saw publicizing HC services and encouraging HC attendance as part of their job and expressed a good understanding of HC issues. For example, they could explain that the reason HCs often did not supply the medications desired by villagers was because they were following national protocols, and that overuse or misuse of antibiotics is harmful. All Supervisors recalled receiving training from PFD, both a 5 day training course specifically in VHV supervision and TOT for immunization, diarrheal disease and nutrition/breast-feeding. The training was described as adequate and all 19 Supervisors interviewed felt confident in their role and ability to oversee VHV activities. They also seemed to be actively enjoying it, and to have a closer knowledge of village health needs and perceptions than is usually found among HC staff in Cambodia. Indeed, the HC staff may be gaining as much as the VHVs are from the relationship as it gives them an entry point into the villages and an advocate to help them negotiate the class and cultural barriers that otherwise often impede provider-client relations in Cambodia. All HCs reported a noticeable increase in attendance for immunization since the Program started, and several expressed that in addition to getting better coverage, the work of immunization outreach was now much easier to do, since HC staff no longer had to try to persuade busy villagers to stop and bring the children but rather found the people already assembled and waiting for them. This not only saved time, but removed a potential source of friction between HC staff and villagers, and avoided the previously common problem of HC staff traveling long distances to villages only to come away having immunized few or no children. 8 out of the 10 HCs also reported an increase in general attendance at the HC (both curative and antenatal) attributable to VHV referrals and publicizing of services, and one mentioned that people are now more accepting of ORS whereas previously they had not wanted it and insisted on antibiotics. The better HCs are realizing the benefits of VHV referrals considerably more than the weaker ones, where erratic working hours or perceived poor quality of care discourage attendance. The Supervisors have a checklist provided by PFD to aid them in assessing VHV effectiveness and to provide feedback on training sessions. Most described it as easy to use and “helpful”, but none could show the Team a completed checklist so it is unclear if they actually fill it out during supervision or have simply taken it as a general guideline. In any case, the supervision being done seems effective. Health System Strengthening As previously noted, original plans to train HC staff on immunization, nutrition and diarrhea have been expanded to more systematically address weaknesses in the health system since other NGO assistance failed to materialize. The 4 weakest HCs (Snoul, Ksim, Tamao and Damrei Pong) are receiving intensive on-the-job training and facilitative supervision, the latter in collaboration with OD managers. PFD staff spend approximately 6 days per month working side by side with the consultation staff of these HCs to improve their diagnostic and case management abilities. A simplified IMCI algorithm has been introduced and training in full IMCI 15 is planned for October for all HCs, with on-the-job follow through by PFD staff afterwards. IEC materials for HC use on ORS have been developed and the establishment of “ORS corners” is planned. PFD staff are also working with these HCs on general management issues, including the use of cost recovery funds and staff regulations/attendance. Not yet begun, but planned, is formal clinical training for HC midwives and on-the-job follow up in the areas of safe delivery, management of obstetrical complications, ante/post-natal care and family planning. Training of TBAs in safe delivery, recognition and referral of complications, breastfeeding promotion and immediate care of the newborn is also planned. These interventions have the potential for considerable impact given the high contribution of peri-natal causes to infant mortality in Chhlong and significant number of maternal deaths (see Annex IV). All 10 HCs in the OD have been assisted in the formation of HC Village Feedback Committees, provided with the “Foundation of Health Center Training” (an MOH course which includes diarrhea and immunization), trained in use of the COPE self-assessment tool, and provided with transportation for EPI outreach (one motorcycle per HC plus reimbursement of fuel). Since HCs were found not to be providing regular immunization outreach services to remote villages, PFD is providing per diem for EPI in cases where the distance and travel requirements legitimately exceeds the single flat rate available from the MOH. PFD is in the process of procuring kerosene refrigerators for all HCs, which currently have to travel to the OD to collect vaccine. Training in “MPA Module 10” (an MOH course for HC staff on nutrition counseling and growth monitoring) will be supported as soon as revisions to that module are completed. No assistance is provided at present to the Referral Hospital and the current level of PFD staffing would not permit it, but the need is great as it is the only source of referral for HCs. It is hoped that another NGO will obtain funding and start assistance to both the Referral Hospital and the OD Health Management Team, which is also quite weak. PFD is providing some assistance to the OD, primarily capacity building in EPI management and supervision, and use of the COPE quality-assurance tool, but the level of input is far less than would be necessary to bring OD capacities up to the level necessary for effective management of the OD health system. Hearth Model The Hearth model has been introduced in 2 villages, both of which are in their second round of NERPs. One of these two was visited by the Team on the third day of the second NERP round. In that village, 1/3 of the children had been identified initially as moderately or severely malnourished (weight for age) and enrolled in the first NERP which by all accounts went very well. Mothers, VHVs, PFD staff and even commune officials were enthusiastic about the initiative ,and mothers appear to have internalized the lessons learned, including messages contrary to pre-existing traditional beliefs. Staff reported that “when we first started preparing the porridge and added trakouen (an indigenous leafy green vegetable) and fish, the mothers’ eyes got wide and frightened because they had always heard it is dangerous for small children. But after they saw the children eat it with no problem they relaxed, and by the second day they helped to make it. Now they feed it to their children regularly”. Several mothers and VHVs emphasized that the villagers were convinced of the validity of the educational messages because they could see a difference in the children with their own eyes. Mothers were very eager to weigh their children (one had even gone into the marketplace to use a vegetable scale in between the NERP rounds) and discussed the amount of weight gained. Although a few mothers were resistant to bringing the children to NERP during round one, they joined after seeing the others attend and by the second round there was no significant resistance 16 encountered; indeed, the staff had problems with mothers begging for admission for children above the cut-off weight for age. The growth monitoring conducted prior to the start of the second round found no new cases of malnutrition in the village, and 6 out if 11 children previously identified were now above the cut-off point for moderate/severe malnutrition. It remains to be seen how long NERPs need to be conducted in a village before levels of malnutrition drop to an acceptable point and whether that is sustained after NERPs cease. The model plans to institutionalize growth monitoring within the villages so that mothers are able to assess their children’s nutritional status on their own, armed with the nutritional knowledge provided during the NERPs. The answer to both questions is likely to vary greatly by village. The village cited above was quite small in size with fully a third of the children under 3 reached by the first NERP, allowing for a very rapid saturation with educational messages. Larger villages would likely take a much longer time, as may inland villages where levels of education and speed of comprehension of new information are markedly lower. The great strength of the Hearth model is that it is built around healthful existing indigenous practices, identified through a positive deviance inquiry, utilizes food resources available in the village, and provides intensive education tailored specifically to local practices and locally available foods. However, these same factors make it extremely labor intensive, since every single village has to be separately assessed in terms of beliefs, behaviors and food resources. Training of drug sellers The NCCSP trained 156 drug sellers in the OD on use of ORS and assisted in establishing a supply source of them. In the 11 villages visited by the team, ORS was found to be available in all drug stores and drug stalls. However, it was unavailable in cases where there was either no drug store or only a vendor of multiple products with a few medicines among them. In general, these were the inland villages. Drug stores reported that the ORS packets sold well, and the packets seen looked new, suggesting rapid turnaround of stock. The possibility of having VHVs sell ORS has been explored by PFD but met with difficulty due to lack of government policy on the selling of government supplied ORS outside of health facilities. There is, however, an explicit government policy that permits this with contraceptives on a cost￾sharing basis, so a precedent exists. Alternatively, non-government sources of ORS could be sold. Iron Distribution Pilot The NCCSP DIP calls for a pilot test of three approaches to iron supplementation for women of reproductive age and children, in collaboration with the MOH, WHO and HKI. This activity has not yet begun due to the need to first develop national guidelines for iron supplementation and to locate an appropriate formulation. It is scheduled to start in 2003. It is unclear whether the iron formulation, being identified by HKI, will also include folic acid, an important consideration given the contribution of malaria to anemia in this population. Constraints and Unmet Needs Most VHVs raised the issue of incentive either directly or indirectly, making it clear that they understood they were volunteers and did not expect a salary, but did need, in the words of one VHV, “need some little incentive to feel able to carry on”. This issue was also uniformly raised by HC Supervisors and PFD national staff, and there is a general consensus that the currently 17 satisfactory level of performance will not be sustainable over the long haul unless some tangible incentive is introduced. The amount of incentive needed does not appear to be high – for example, many VHVs expressed envy of the Reproductive Health social marketing agents, whose average profit is only about US $1-2 dollars per month. The possibility of having VHVs also serve as social marketing agents for various health commodities was raised with each Supervisor, PFD staff, some VHVs, the OD and the PHD and received a favorable response except for one Supervisor who was doubtful that it would be viable in her specific catchment area due to the large number of markets and goods already readily available (riverine area). Aside from incentives, all VHVs and Supervisors expressed a desire for further VHV training. While many mentioned “refresher training”, only a minority of VHVs – most of them in remote inland villages -- actually seem to need a detailed review of the existing material, while the others were seeking a chance to come together with other VHVs and PFD, share experiences and generally get a morale boost. All VHVs and their Supervisors were interested in expanding the range of VHV interventions, with the most frequently mentioned requests being ARI, dengue, malaria, safe motherhood, typhoid, general fever management, and scabies/other skin infections. It is noteworthy that these closely conform to the causes of death identified in the verbal autopsy data (see Annex IV) and that village chiefs, mothers in the village, and HC staff cited the exact same topics as being health priorities. VHVs in areas where the HCs are weak expressed problems in referring people and indicated that they are “blamed” by villagers if they refer them to the HC and it turns out to be unstaffed or otherwise unacceptable. PFD has recently begun working with these Health Centers through on-the-job training, collaborative supervision with the OD Managers, and coaching. Formal training in IMCI and intensive follow-up is planned. Based on the experience in those HCs already functioning at an acceptable level, the combination of HC strengthening and VHV referrals can be expected to produce improved utilization in these HCs once the former is achieved. While the VHVs have greatly facilitated immunization outreach, there are still supply-side problems in some villages due to failure of the HC staff to conduct outreach as scheduled. This problem was cited in the more remote inland villages, some of which are as much as 30km away from the HC and complained of no outreach at all for months at a time, or failure of the HC staff to show up as promised, thereby damaging the VHV’s credibility and making it harder to mobilize people in the future. One reason for poor outreach coverage in remote areas is that the MOH provides a single flat rate per diem for outreach (funded by UNICEF) irregardless of distance or actual travel costs. As a result, there is a clear economic incentive to conduct outreach in villages that are inexpensive to reach and a disincentive – indeed, a financial loss in some cases – for conducting it where travel is difficult and costly. Recognizing this problem, PFD has recently begun supporting per diem for immunization team travel to remote areas where the government per diem is inadequate. In addition, PFD is in the process of acquiring cold chain equipment (funded by a grant from the Japan Fund) for each HC, which will eliminate the time-consuming process of getting vaccines from the OD on the day of immunization. As previously noted, there are problems with the VHV record-keeping system and a need to improve and streamline it so that there is no duplication and information is entered only once, at the time it is obtained, rather than on one paper and subsequently transferred to another with the room for error this creates. In addition, VHVs need more training and Supervisor reinforcement in the use of the register as a tool for targeting households and planning activities, and a special effort is needed to capture information on 2002 deaths before it is forgotten. 18 HCs provide only outpatient care and cannot handle obstetric emergencies; in fact, they have little capacity to perform even normal deliveries due to lack of 24 hour midwifery staff and inpatient beds. There is a very pressing need to strengthen both the technical capacity of Referral Hospital staff and overall hospital management, with particular attention to obstetrics and inpatient pediatric care. Pneumonia, dengue hemorrhagic fever (DHF), malaria, meningitis/encephalitis, and typhoid together account for over half of all under5 mortality in the Program area, and are not easily managed on an outpatient basis. Community level interventions can reduce the prevalence of some, but not all, of these problems and there is simply no getting around the need to have a first line of referral with inpatient capacity. Villagers and village leaders interviewed by the Team where quite vocal in complaining about the hospital, particularly with reference to the recent outbreak of DHF. As noted in table 2 (page 10), training of TBAs has not yet begun. The planned content has expanded beyond the initial focus on breast-feeding to also include safe motherhood and immediate care of the newborn. Maternal and child mortality data (see Annex IV) for the Program area testify to the urgent need for this. Peri-natal factors are the 2 nd leading cause of death in Chhlong OD, and many of the verbal autopsy forms indicated that the baby was cyanotic from birth. Cambodian TBAs do not customarily clear the airway of newborn infants, and even trained midwives are often observed to neglect this simple but life-saving measure. To date, Program health education has been confined to VHV sessions with mothers, although the DIP includes plans for broader community health education using such approaches as drama and puppet shows. VHV education appears to work quite well in reaching mothers, but there are several areas in which a broader audience needs to be reached in order to effect the necessary behavioral change. Most of these fall into the general category of environmental sanitation, e.g: · the relationship between stagnant water, uncovered water jars, and DHF (requiring community action to promote drainage around houses and in the markets); · the role of flies in disease transmission (typhoid, dysentery, diarrhea) and measures to both decrease fly proliferation and protect food from contamination; and, · the value of sanitary latrines and inexpensive means of constructing them. Iodine deficiency is highly endemic in the area. Although the VHVs have been taught to promote iodized salt, it is not universally available and costs more than regular salt, with the result that this message is probably having little effect. Supply-side interventions to assure the availability of affordable iodized salt are needed. C. Assessment of Program Management 1. Program Structure The Program began with a flat organizational structure: one expatriate Program Co-ordinator assisted by initially six Cambodian Child Survival Program Officers, subsequently increased to eight, plus a VSO nurse volunteer who is assisted by two Cambodian staff and works on strengthening of the health system. Recently, one of the Program Officers has been promoted to Team Leader, creating a more tiered structure and manageable span of control for the 19 Program Co-Ordinator. The recently appointed Team Leader has exceptional technical, programmatic and leadership skills and a high level of commitment. His retention and continued professional development will be extremely important, as the presence of a core of highly skilled national staff is critical not only to the success of the NCCSP but to PFD as an organization. The UNFPA/MOH-funded RH Program operates parallel to the Child Survival Program, with a designated Team Leader and two staff. This arrangement, while making it easy to separate out staff costs by donor, does not facilitate an integrated approach to maternal and child health nor make most efficient use of staff time, since two separate cadres of staff undertake time￾consuming travel to the same villages and oversee interventions that both target women of reproductive age. The Program Co-Ordinator is based fulltime in Chhlong, supervised and supported by a Phnom￾Penh based expatriate Health Program Manager who spends 50% of her time back-stopping the Child Survival Program. Above the Health Program Manager is the Country Representative, who also helps support the Program, particularly on administrative issues and in dealings with external agencies. This staffing configuration provides adequate technical and administrative oversight for the Program. The Phnom Penh-based Health Program Manager may need a fulltime assistant as PFD has recently received a new health grant from USAID/Cambodia and is expanding into other parts of the country. Likewise, the Chhlong-based Child Survival Team Leader may need an assistant in the future as he is directly managing a large number of staff (7). 2. Staff Development PFD has made a number of investments in developing the capacity of its local personnel , who make up the majority of the staff and are the direct implementers. PFD National Staff received training on IEC, focus groups, Vitamin A, computer skills, quantitative survey methodology, adult learning techniques, the COPE HC improvement tool, and monitoring and evaluation. PFD staff also received the same training as HC VHV supervisors on immunization, diarrhea, nutrition and breast-feeding. In addition to formal training, on-the-job mentoring is provided by the Program Co-Ordinator and VSO nurse. The majority of local staff described the management style as facilitative and supportive, and morale is quite high. One factor which has greatly helped in that regard is an unusual level of camaraderie and affection among the staff, who share communal housing, and actively socialize together. (“Child Survival staff love each other” was the way one local employee put it.) This esprit de corps has facilitated the rapid transfer of skills and attitudes to newer staff. All staff interviewed, including several with the Program six months or less and coming from a non-community based background, clearly articulated an understanding of the principles of community organization and community participation. When asked how working for PFD compared to working with other NGOs (for those staff with prior NGO health experience), responses included: · “it works from the community level up, not top down” · “the purpose and activities are very clear, very focused” · “before this, I never knew what was going in the communities”. 20 Staff expressed the feeling that working collaboratively with communities takes more time but is more satisfying than conventional approaches: “they (PFD) get the idea from the community rather than imposing it”. Staff with the Program during the entry level phase had the opportunity to learn and conduct participatory rural appraisals, but newcomers have not received training on community organization. Rather, they appear to have picked up the idea and philosophy from the others, a method which works well now due to the currently high level of staff cohesion and socialization, but which might not if the interpersonal dynamics of the team change. Most staff felt that they were actively involved indecision-making (“in PFD, staff can think for themselves and plan and solve problems”). A small minority of staff, who appeared shyer in temperament, indicated that they felt their ideas were not actively solicited by management. Workloads are described as manageable, another factor contributing to good morale (“some days we are extremely busy but other days are slower, so it is all right”). Although staff have received training in a number of areas, formally and informally, all expressed the need for further training specifically in the areas of management skills and supervision, appropriate since much of their time is spent overseeing the supervisory work of others. 3. Collaboration with Other Organizations True to its name, PFD has actively pursued partnerships and collaborations with government, other NGOs, and International Organizations and made skillful use of same in implementation of the Program. PFD has also actively sought other donor funding to leverage USAID resources with funding for complementary or expanded Program activities. Government PFD has official Memorandums of Understanding (MOU) with both Chhlong Operational District (OD) and Kratie Provincial Health Department (PHD) and is in daily contact with the OD and more than weekly contact with the PHD. A Child Survival Coordinating Committee consisting of PFD staff, key OD staff and representatives from the PHD has been established to help oversee the Program and also discuss and resolve health system issues impacting on child health. Both the OD and the PHD expressed satisfaction with the Program and the working relationship in discussions with the Team, and the PHD has requested expansion of the activities to the other OD of the Province. PFD is an active participant in the MOH IMCI Working Group, and has also worked closely with the MOH National Malaria Center and National Center for Maternal and Child Health on technical issues. PFD played a major role in development of the new national malaria strategy. International Organizations PFD works closely with the World Health Organization (WHO) in regard to the IMCI pilot and shares information and seeks technical input from the WHO Malaria Advisor. PFD works closely with UNFPA on reproductive health issues, and received EC funding through them for the RH Program in Chhlong. PFD has extensively discussed with UNICEF issues related to the national immunization Program and iodized salt distribution. 21 NGOs PFD has a formal Memorandum of Understanding (MOU) with Helen Keller International (HKI) and the two agencies have partnered in a number of activities. PFD and HKI collaboratively assisted the OD with twice-yearly VAC distribution campaigns. HKI has assisted PFD in establishing Village Model Gardens in many communities to complement PFD’s nutrition education efforts. PFD uses an HKI-developed flipchart for VHV nutrition and breastfeeding education. HKI trainers assisted PFD in training of PFD and OD staff on Vitamin A.. PFD has obtained advice and assistance from the Reproductive and Child Health Association (RACHA) in liaison with, and training of, private drug sellers. The questionnaire used by PFD for the Health Center Resource Survey was developed through modification of a survey instrument provided by RACHA. PFD and OD staff received TOT on the use of the COPE quality assurance tool for HCs from RACHA and CARE, and both organizations have provided guidance on its introduction. Staff from the local NGO Khmer Students Association were utilized to provide computer training for PFD staff. PSI has partnered with Population Services International (PSI) in submission of a proposal for malaria social marketing to the Global Fund, with the approval of the MOH and the Global Fund Country Coordinating Mechanism. PFD is an active member of Medicam, the official forum for NGOs active in the health sector in Cambodia. Donors PFD has a long-standing relationship with the USAID/Cambodia Mission and has received grants from them for health activities since 1996. PFD has taken an active role in assisting USAID’s collaborative development of a new HIV/MCH strategy and monitoring and evaluation system. PFD has successfully secured matching funding for the Child Survival Program from the Japanese Embassy (cold chain and other HC equipment), the EC and CIDA. PFD has also obtained volunteer staff from the British Voluntary Service Overseas organization for two consecutive terms, providing a nurse to assist with strengthening of the health system. PFD has obtained funding for a community based vector control Program from the Australian Agency for International Development ( AUSAID), and funding from the EC for malaria activities (including the cost of construction of two health posts in the malaria endemic area of Snuol, and support for Foundation Training for Health Centers). EC funding has also been obtained indirectly through UNFPA for reproductive health activities (HIV/AIDs education and social marketing of contraceptives). Recently, PFD has received funding from the World Bank for a study of health practices and access to services by ethnic minorities in Kratie Province, information which will both be used by 22 the World Bank in development of its strategy for ethnic minorities, and be useful to PFD in refining approaches strategies within the Child Survival Program area. 4. Use of Technical Assistance As noted above, PFD has made skillful use of in-country expertise through collaborations with other organizations (and has returned the favor by sharing materials and lessons learned). In addition, formal external assistance has been utilized for the following: · development of the Hearth Model and training of staff in its implementation · staff training in the LQAS methodology · monitoring and evaluation and development of a sustainability plan · a study of health beliefs, practices and barriers to health care among ethnic minorities in Kratie · the mid-term evaluation PFD has also availed itself of the TA available from USAID/Washington’s Child Survival Technical Services (CSTS) with regard to survey methodology and IMCI. Future TA will be needed for refinement of the verbal autopsy form and training of PFD staff in conducting verbal autopsies and analyzing the information, and for help in piloting the introduction of social marketing by VHVs. The latter might be obtainable through partnership with PSI; if not, an external consultant may be necessary as the logistical aspects of a new social marketing initiative are considerable. D. Assessment of Program Impact The foregoing observations are qualitative, based on information gathered from VHVs, Supervisors, villagers, community leaders, and PFD staff, and direct observations in the field. Augmenting this is quantitative data from a repeat survey done by PFD in January of 2002 to assess changes in baseline indicators as documented in the baseline survey done during the two year entry level grant. Since the nutrition and breast-feeding interventions were not introduced until the first half of 2002, the mid-term survey is limited to diarrhea and immunization. However, the Team found clear indications in the field that VHVs are conducting education on nutrition and breast-feeding with at least as much intensity as the other intervention areas, and that mothers are retaining and acting on these messages. The progress demonstrated in immunization and diarrhea in the January 2002 survey (at which point VHVs had been active in immunization for 8, and diarrhea for 4 months) is therefore likely to be indicative of that which will be achieved in nutrition and breast-feeding. Table 3 below compares data on key indicators from the baseline survey to that of the January 2002 survey. The baseline survey was a 30 cluster sample. The January 2002 survey was a Lot Quality Assurance Sample (LQAS) in which an equal number of randomly selected households was interviewed for each HC catchment (VHV Supervision) area, This yields average estimates per HC catchment area, which is not the same as the average for the population, since HC catchment areas vary greatly in population size. The baseline survey provides estimates for the population as a whole. In order to render these two data sets comparable, the raw data from the LQAS was weighted proportionate to the population of each 23 “lot” (HC catchment area). Although such weighting increases the sampling error, all indicators underwent change of sufficient magnitude to be statistically significant. It should be noted that the diarrhea indicators at baseline were collected on children ages 0-24 months while the repeat survey took only children aged 12-24 months. The Team was unable to obtain the raw data from the 1999 baseline to disaggregate the 12-24 month-olds. It is impossible to say whether doing so would increase or decrease the change which has occurred in maternal knowledge and behavior. However, the magnitude of change is so great that there would still clearly be a sizable improvement. Table 3: Comparison of Baseline and Mid-term Household Survey data DIARRHEAL DISEASE INDICATORS 1999 Jan. 2002 CHANGE 1. Breastfed child more than usual during diarrhea 21% 51% 145% 2. Provided more fluids than usual during child's diarrhea 34% 78% 129% 3. Provided more or same amount of food during child's diarrhea 47% 76% 62% 4. ORT Use during diarrhea (ORS, sugar/salt solution, home fluids) 15% 41% 171% 5. Mother can describe correct management of diarrhea (fluids, feeding, referral) 42% 80% 90% 6. Mother has heard of ORS 3% 94% 3369% 8. Mother knows danger signs in diarrhea (when to take child to HC/hospital) 45% 81% 81% 9. Mother knows to feed child more when recovering from diarrhea 17% 72% 326% IMMUNIZATION INDICATORS 1999 Jan. 2002 CHANGE 1. Child Has Immunization Card 32% 87% 170% 2. Child Fully Immunized (Among Children With a Card) 44% 73% 65% 3. Child Fully Immunized (Out of All Children Total, Immunized by Card) 14% 63% 347% 4. Mother knows age at which child should receive measles vaccine 10% 39% 301% 5. Mother knows purpose of TT vaccine 25% 52% 109% 6. Mother has maternal health card 21% 76% 255% As can be seen from the above, an enormous increase in key indicators related to both immunization and diarrheal disease has occurred since the institution of VHV activities. The magnitude of change is all the more impressive taking into account that the diarrhea intervention had been in place only 4 months and the immunization intervention, only 8. It is also extremely impressive compared to nationwide data. At baseline, the Program area had significantly lower immunization coverage than the national level of 32% (for immunization with card); it is now twice as high. Likewise, knowledge of ORS at baseline was only a fraction of the national rural average (3% versus 48% for nationwide rural), but is now nearly double it. Figures 1 and 2 display the change which has occurred in the Program area between 1999 and January 2002 pictorially. 24 Figure 1: Diarrheal Disease Indicators 21% 51% 34% 78% 47% 76% 15% 41% 42% 80% 3% 94% 45% 81% 17% 72% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1999 Jan. 2002 Survey Dates Percentage BF Fluids Feeding ORT Use Knowledge Knows ORS Danger Signs Feed After 25 Figure 2: IMMUNIZATION INDICATORS 32% 87% 44% 73% 14% 63% 10% 39% 25% 52% 21% 76% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1999 Jan. 2002 Survey Dates Percentage Has Card % with card immunized % Fully Immunized Child Knows measles Knows TT Has Mat Hlth Card 26 The preceding table and graphs are confined to those indicators for which baseline data was available for comparison. A number of other indicators were also measured in the January 2002 survey, for which baseline data is unavailable but Program targets were set. The key ones are shown in table 4 below. Since no baseline was available, the data is presented as measured by LQAS, that is to say, as the average value per HC catchment area. The experience with weighting of the indicators for which baseline data was available indicates that the percentage for the population as a whole is usually slightly higher than the average per lot, since the HC catchment areas with the lowest populations also tend to be the lower-performing. Table 4: Other 2002 Indicators (No Baseline Available) Indicator 2002 Average Value per “Lot” Program Target Sanitary disposal of child’s feces (self-reported) 70% 70% Hand-washing before eating or cooking, after defecation (self-reported) 95% 100% Soap Use in hand-washing 90% 90% Place in household set aside for hand-washing 55% 60% Presence in the home of water, soap/ash, and clean cloth 65% 50% Receipt of VAC in past 6 months (child 6-59 mos) 90% 90% Adequate TT protection during pregnancy7 65% 70% IV. Discussion and Conclusions A. Overview The Northeast Cambodia Child Survival Program has successfully implemented almost all planned activities, with a few remaining ones in an advanced stage of planning. The Program will clearly succeed in implementing all activities prior to the end of the grant period except for three minor ones which have been deleted due to poor performance in piloting or low priority . In addition, the Program will have implemented a significant number of additional interventions directly related to child survival and responsive to the documented causes of child mortality and weaknesses of the health system. B. Village Health Volunteers The primary strategy of empowering and motivating village women to adapt improved health behaviors through peer education from Village Health Volunteers (VHVs) has proved sound and is working well. VHVs are known and active in their communities and both qualitative 7 Defined as receipt of 2 or more TT during pregnancy if not previously immunized, or at least one TT during pregnancy if previously immunized. 27 assessments by the Team and quantitative data from a repeat household survey confirm that they are successful in both increasing knowledge and changing behaviors. Village women and community leaders value the Program, as do the staff of health facilities, who have gained easier and more productive access to the community and better service utilization as a result of VHV activities. Key factors in the success of the NCCSP VHVs are: · Participatory introduction of the Program and VHV selection: although very time￾consuming and labor intensive, PFD’s strategy of working together with each individual village to introduce the concept, build support for it, and select VHVs acceptable to the community has paid off significantly. The communities understand the Program and feel a sense of ownership over it. The right people (i.e. persons respected and credible in the community) are serving in the VHV capacity and community expectations of VHVs are realistic. · Manageable workloads: many health volunteer Programs fail due to unrealistic expectations of the amount of time and effort volunteers can supply. The NCCSP has one VHV per 50 households; since VHVs work only with mothers of children under five or pregnant women, this means an actual workload of about 30-35 families, all living in the VHV’s immediate vicinity. This makes it possible for VHVs to fill their role without undue expenditure of time or adverse effect on their household and agricultural tasks. · High quality training and appropriate IEC materials: the training provided to trainers and VHVs utilized appropriate, participatory adult learning methodologies. Pre and post tests were done and additional training provided to low scorers. The IEC materials provided to the VHVs are user-friendly pictorial flipcharts showing typical rural Cambodian life and familiar objects, with no written text on the side facing the mother so that no literacy is required. · Close supervision: VHVs are supervised monthly by staff from the HCs who were specifically trained in adult training techniques, the technical content areas, and supervision of VHVs. PFD staff in turn supervise the supervisors, and make occasional spot checks or problem-solving visits to VHVs. Although the VHV Program has been very successful, this should not be grounds for complacency. Experience worldwide is that it is easier to get a volunteer health program going well than it is to sustain it. There is a clear consensus among VHVs, community leaders, VHV Supervisors and PFD national staff that something needs to be done to build in a small incentive for VHVs if the currently high performance is to continue over the long run. While periodic meetings, new trainings and the like work in the short-term, they do not provide volunteers with a clear sense of what they can expect to receive for their efforts and are also totally dependent on continued NGO investment. A better alternative worth exploring would be having VHVs serve as Social Marketing agents of health commodities. In addition to providing a small commission, this would strengthen their education messages by ensuring the availability of items necessary to the behaviors being taught (e.g: soap, iodized salt, ORS). While the setting up of a social marketing initiative will be labor intensive for PFD, requiring considerable trial￾and-error to identify commodities with sufficient sale potential, locate suppliers and establish distribution channels, this could ultimately become a self-sustaining activity. 28 Somewhat related to the incentive issue is the proliferation of different cadres of volunteers in the Program villages, often one per vertical intervention. PFD’s Reproductive Health Program, which includes social marketing of contraceptives and HIV education, runs in parallel to the VHV Program, usually with different village-based staff. The Malaria Program created a separate cadre of village “malaria volunteers” for a one-time bed net distribution (a decision made by the MOH, despite PFD recommendations to use the existing VHV network); this has been particularly contentious since the “malaria volunteers” each received a bicycle, a fact VHVs often raised in discussions with the Team. While political realities and the need to work collaboratively with government may not always make it possible, every effort should be made to utilize the well-established VHV network to distribute information and commodities rather than create separate one-time cadres of volunteers for special purposes. It would also be highly desirable to integrate the NCCSP with PFD’s UNFPA/MOH-funded Reproductive Health (RH) Program since it is targeting the same villages and often the same women. Furthermore, RH volunteers have to opportunity to earn a small income from contraceptive sales commissions, which VHVs currently lack. It would not be possible for every VHV to sell contraceptives since their catchment areas are too small, but at least a few could. This is discussed further under Recommendations. The supervision of VHVs by HC staff is working very well, a somewhat surprising finding given that VHVs are community-based and community-selected while HC staff are part of a government system which often encounters class barriers in its interactions with the general population. VHV Supervisors are providing tangible assistance as well as less tangible but important morale boosts to the VHVs, and enjoy the supervision. It has broadened their awareness of the communities and given them a point of entry wherein they are assured of a welcome, and an intermediary who can help sort out issues that arise between facility staff and villagers. It has also made their work easier to do and increased their own performance in terms of HC utilization and immunization coverage. The time required for VHV supervision and associated paperwork is small and does not seem to adversely affect staff’s ability to carry out their usual HC tasks. Only 2 staff per HC, spending a couple of afternoons per month, are required. There is a need to expand the range of health education topics covered by VHVs to include others of great health significance. The approach used for the initial modules – gradual introduction of new material with time in between trainings to practice what was taught – worked well and should be retained. In addition, since new material is always more attractive than old, each new training should include a review of previous topics and discussion of strategies for presenting the information on all topics to the communities. While most VHVs are well versed in their current material and active, there is a need to address the problem of those who are not. It appears from discussions with Supervisors that about 10% overall are not active despite Supervisor efforts, and no productive purpose is served by keeping them in the Program. Removal and replacement of non-functioning VHVs should be done prior to the introduction of new trainings and incentive measures. VHVs who are functioning but lacking full understanding of the existing health messages need to be identified and provided with more in depth refresher training. 29 C. Health System Strengthening Improvements in health status require work at both the demand and supply side. The initial Program design assumed that another NGO would be building capacity in the HCs, Referral Hospital and OD Management Team, and in general probably underestimated how weak the system was. The Program has made appropriate adjustments to provide support to HCs and this can be expected to improve the situation by the end of the grant period, but the needs of the RH and OD Management Team remain largely unmet. As previously mentioned, this situation must change if major causes of child mortality are to be addressed. Particularly urgent is the need for competent, reliably staffed and equipped 24-hour obstetric facilities and an inpatient pediatric ward able to provide oxygen, intravenous antibiotic therapy, laboratory diagnostics, and management of DHF. D. TBA Training As has been previously noted, plans for TBA training have expanded to include more than breastfeeding. This is thoroughly justified given the child and maternal mortality findings for the Program area (see Annex IV). However, training on danger signs and referral will have little relevance until such a time as the health system offers a viable referral point (see above). One solution would be a two-phased training, the first commencing in the immediate future and focused on interventions that are not dependent on the availability of emergency obstetric interventions (EOI) : breastfeeding promotion, referral for ANC/TT, aseptic technique and use of safe birth kits, abstention from harmful practices, and care of the newborn. The latter should receive heavy emphasis, and TBAs should be supplied with bulb syringes and made to role play the clearing of an infant’s airway and wrapping of the newborn. A second phase, to be done after basic EOI is available, would involve danger signs in pregnancy and delivery and referral. To be successful this needs to be accompanied by the creation of linkages between hospital staff and TBAs, preferably in the form of partnership arrangements whereby TBAs who refer women are allowed to remain during the labor and delivery and continue to provide psychological and spiritual support. Aside from financial considerations (which are very real, since TBAs both cost less and accept credit or payment in kind), the psychological and spiritual support provided by TBAs is a key factor in their popularity. Cambodian women believe TBAs to possess special powers to reduce labor pain and protect the mother, and TBAs are much more versed in nurturing and comfort measures than trained midwifes. Such partnership arrangements have been piloted elsewhere in Cambodia with considerable success, most notably in the Sotnikhom “New Deal” project where health facilities also share the delivery fee with TBAs. E. Program Management and Staff Development While PFD is in general doing an excellent job of implementing a large Program under difficult conditions, implementation could be further enhanced through a restructuring of staff responsibilities so that each Program Officer has primary responsibility for overseeing all activities – VHV and health system strengthening, RH activities, and the malaria Program – in a specific geographical area. This would save considerable travel time, and will allow for a comprehensive view of village health activities. Expansion of VHV interventions and introduction of social marketing will be labor intensive, making such economies important. Additional measures which could be taken would be replacing the current level of PFD supervision with spot audits of HC VHV Supervisor performance in those HC catchment areas with strong VHVs and strong Supervisors, freeing up more staff time for the new initiatives. 30 The quality, commitment and high morale of PFD’s national staff have been critical to the Program’s success. While PFD’s organizational culture and supportive management style make a positive contribution to staff morale, it alone cannot create the unusual level of esprit de corps now present and there is no guarantee it will not change with time due to factors beyond the organization’s control. It would therefore be wise to provide staff with more formal training in community organization and Program goals and objectives rather than relying on peer acculturation. Also, formal training in management skills and supervision would strengthen staff‘s professional development and ability to gradually assume more of the responsibilities currently handled by the expatriate Program Co-Ordinator and VSO nurse, freeing up their time to address expansion of the Program in scope and, possibly, geographical location since the success clearly warrants expansion to the rest of the Province, and Provincial authorities are receptive to it. While most staff report being able to give input readily, and management is perceived as facilitative and receptive, individuals vary in their willingness to volunteer opinions and a more regular system of staff meetings with a concentrate effort to ensure that everyone has a say would be useful. As previously noted, PFD has recently promoted one of the national Program Officers to the position of Team Leader. Over time, there is scope for additional delegation of management responsibilities to this individual, with the expatriate manager taking on more of an advisory and less of a direct administrative role. This transition will need to be made gradually to avoid overwhelming staff and ensure the needed on-the-job training to develop the requisite skills and experience. F. Nutrition Interventions Preliminary results of the Hearth model look very encouraging, but it is a very labor-intensive undertaking. It will therefore be very important to have a solid base of data with which to evaluate its impact, since government and donors are wary of interventions with time￾consuming and complex “software” requirements. This means not only data on improvements in nutritional status, which the Program is already poised to collect, but data on mortality impact. The VHV network provides a means of recording vital events for the entire population, and this should be done with particular care in Hearth villages. Most urgently, mortality data for the year preceding the start of the intervention is needed to serve as baseline. This is already available from VHV registers and monthly reports but needs to be pulled out for the specific villages and re-verified to ensure no deaths were missed, preferably by a house-to-house rapid survey. Likewise, mortality data since the intervention began should be updated and verified annually. Since this will be a surveillance of the entire village, rather than a sample, no sampling error is involved and direct estimates can be obtained. The under5 mortality rate for Cambodia is sufficiently high that two or three villages alone will generate enough deaths to be able to measure change from year to year. The extent of malnutrition in the areas where Hearth is being done is so high that a measurable mortality reduction will occur within a few years after improvement on nutritional status, if that improvement is significant and sustained. Clear proof of mortality reduction is probably what it would require for the broader health community – government, donors and other NGOs – to scale up the intervention given its complexity and labor intensity. 31 G. Monitoring and Evaluation See above with respect to the Hearth pilot. The record-keeping system of the VHVs needs to be streamlined and new training provided in the use of simplified forms. The recording of deaths needs to be revived through regular reminders by Supervisors during their monthly visits, and a concentrated effort should be made to capture all 2002 deaths (many of which have not yet been recorded) and conduct verbal autopsies as was done for 2001. The verbal autopsy procedure, however, needs revision as that previously used collected too little information and was collected by persons other than those who later conducted the analysis and assignment of probable cause of death. At the onset of the Program, VHVs were given a general description of their catchment area and told to list all eligible women (mothers of children under 5). There was no formal household census done to ensure that all houses in the village were covered. A comparison of the total number of children under five reported by VHVs with the number expected assuming that the fertility and mortality levels documented in the 1998 census have remained static suggests that this initial listing was close to complete, missing no more than 10% of eligible women. Even so, with time more omissions are likely to seep in as new families or family members arrive in the area. A periodic village census, preferably combined with some other activity that requires censusing (e.g. mosquito bed net distribution or voter registration) needs to be undertaken every few years to update VHV registers and ensure complete coverage. The Program has done a good job of obtaining and using baseline data, and conducting repeat surveys to measure changes. A survey on nutrition and breastfeeding indicators is planned soon. To enable comparison with the baseline data without weighting, it would be better to modify the sampling design so that it is a random sample stratified by HC catchment area with number of households in each proportional to the catchment area’s population but with the smallest catchment area including sufficient households to be statistically representative at HC catchment level. This will produce a self-weighting estimate of indicators for the population as a whole plus “lot” specific estimates allowing comparison between supervision areas and identification of low performers. In piloting iron distribution by VHVs, the original design in the DIP of three different approaches (each of which would require monitoring of changes in hemoglobin level) will be unnecessarily labor intensive and a simplified design should be considered. The original plan is to have one group receiving free daily supplementation, one receiving free weekly supplementation, and one purchasing supplements themselves with VHV promotion. A single dosage plan would be adequate, and daily dosage is both easier to remember and known to be more effective. Testing of free distribution should be undertaken only if there is a reasonable likelihood that government and donors would be able to support the costs of free mass distribution on a wide scale (very improbable). Unless there is reason to think that free mass supplementation is a possibility, piloting of some type of social marketing approach (possibly with subsidization) should be considered instead. There is no point in piloting something that has no realistic potential for scale-up, and the implementation and monitoring/evaluation aspects of one approach are considerable enough without the exponential increase created by multiple variants. 32 H. Sustainability The NCCSP is better placed for sustainability than most Programs since its primary activity is the imparting of new information, skills and behaviors to communities. When successful – as these appear to have been – such changes are intrinsically self-sustaining as changed community attitudes and practices get handed down to subsequent generations. The DIP contains a list of sustainability objectives which are not contingent on continued support for the VHVs, and to date the indicators listed have been met. Since the VHV approach has proven so successful, however, it would be highly desirable for it to continue beyond the current end date of the grant. It is in theory possible for this to happen without PFD, since the VHVs are already linked to, and under the supervision of, the government health system. However, the government health system would require both greater managerial capacities and greater financial resources than it currently possesses to be able to sustain the VHV Program. Cambodia is unique in that its public health system is less than a decade old. Prior to the warfare which engulfed the country from 1970 – 1993, there had never been a functioning public health system in the rural areas, and what human resources and facilities did exist were completely obliterated during the Khmer Rouge genocide. The Ministry of Health had virtually no trained managers, and controlled no budget, prior to 1993. The country completely missed out on the developments in primary health care which swept through the rest of the world in the 1970’s-1990’s. It is therefore hardly surprising that Chhlong OD is far from capable of carrying out its current functions adequately, let alone undertaking new ones, and that both the level of government budget allocations to the health sector and the effectiveness of budget disbursement to the periphery are inadequate for basic service delivery. This should not be cause for pessimism, however, as the Cambodian health system is extremely new and steadily improving. Enormous progress has been made in less than a decade in creating a functional Ministry and the basic framework of a national health system from a near-zero base. Reform efforts are underway with regard to both capacity building and increased resources/ financial accountability. Looking back at the progress that the government health system has made in less than a decade, there is every reason to expect that it will eventually reach a point wherein these problems have been satisfactorily addressed and the system is capable of assuming management of initiatives such as the VHV Program. This will, however, take time – realistically, not less than 5, and possibly as much as 10, years. Continued USAID support beyond the current end date of the grant will not only allow activities which are having a demonstrable impact on child health to continue, but help ensure permanent returns on the investment by supporting the Program until the health system is sufficiently developed to take it over. The best contribution PFD can make to Program sustainability is to continue its efforts to strengthen the health system, and to create incentive mechanisms for VHVs that are ultimately self-sustaining through normal market forces (i.e. social marketing). The sustainability of health system strengthening interventions is in a similar situation. Improvements in staff technical skills, attitudes and procedures are inherently sustainable once internalized, but only if the system as a whole provides adequate resources in the form of a living wage and adequate operating budget, and effectively monitors and supervises the sector. None of these conditions yet exist, but there is progress towards them. 33 V. Recommendations A. Overall Recommendations 1. The Program should continue and expand, resources permitting, to the other OD of Kratie Province so that the entire Province is covered. 2. The basic structure of VHV selection by communities, one VHV per 50 households and supervision of VHVs by HC staff should continue. 3. USAID should fund an extension of the Program after the scheduled end date of the grant in March 2004 in order to allow activities which are having a demonstrable impact on child health to continue, and help ensure permanent returns on its investment by supporting the Program until the health system is sufficiently developed to take it over. B. VHV Activities 1. Non-performing VHVs (approximately 10% of the total) should be identified and replaced, and training provided to the replacements. 2. Special remedial refresher training should be provided to the minority of VHVs who do not have a firm grasp of their material. This will need to be preceded by a screening process, since only some VHVs need this training, and among those who do, it will not always be in the same intervention areas. One approach would be to test VHVs at a time when they have come together for another purpose (e.g. for training in a new intervention) in order to identify the number of trainees and subjects needed. 3. Periodic village censuses (every 1-2 years) should be conducted and VHV registers updated accordingly to ensure that all eligible families are covered by the Program. 4. The VHV register should be revised to contain only mother/caretakers name, child’s name, pregnancies, births, deaths, immunization status and contacts in that year. The latter should consist of a simple tick mark made every time the VHV imparts health education to the mother, regardless of whether in a group or individually. 5. The VHV monthly report form should be revised to simply list activity by type (individual education, group education, referral) with tick marks, and VHVs trained to make a tick mark in the appropriate box and on the register “contact” column immediately after each activity. 6. VHVs should be trained in additional interventions. These should be phased in one at a time, with 1-2 months in between new trainings to allow practical experience before going on to new material. Suggested new interventions, in order of priority, are: 34 Intervention Comments ARI Need to also train drug sellers. Emphasize: distinguishing between simple colds and LRTI; danger signs and referral; importance of a complete course of antibiotics Pregnancy and Delivery Might possibly be combined with TBA training, using HC midwives as trainers. Emphasize: nutrition during pregnancy and lactation, importance of ANC and TT, danger signs during pregnancy, use of safe birth kits, danger signs during delivery and post-partum, immediate initiation of breastfeeding, and newborn care Dengue and Malaria Emphasize cause and prevention: bed nets, “baby nets” for infants and children in the day time; mosquito coils, covering/treatment of water jars, drainage to remove stagnant water, importance of following full course of MOH-approved malaria treatment, signs if malaria and DHF and need for prompt medical care. Dysentery and Typhoid Emphasize cause (poor hygiene, flies, improper food handling and preparation) and prevention: environmental sanitation, hand-washing, use of plastic or net covers to protect food from flies, exclusive breastfeeding for infants under 5 months Skin Infections Emphasize cause and treatment of scabies; danger of wound infection and appropriate home management of simple cuts; signs of wound infection requiring medical care; home management of simple sashes (keep dry/gentian violet/HC if not improved) Iodine Deficiency Disease Teach only in accompaniment with a subsidized social marketing campaign. Although IDD has been mentioned in VHV training on nutrition, the complexity of the problem and need to convince people of the value of iodized salt warrant a separate training and flip chart. Child Safety Emphasize prevention of drowning, falls, vehicular accidents, and poisonings. Target older siblings (especially girls) as well as mothers 7. VHVs who are interested should be assisted to become social marketing agents of health commodities in order to provide a small financial incentive with long term potential for sustainability while also increasing access to, and use of, desirable products. Since this will be labor intensive to establish and there is no way of knowing in advance which commodities will sell, this should be piloted first, a few commodities at a time, in a random sampling of villages (both riverine and inland). Commodities should be clearly linked to health education messages and VHVs reminded that the health education is to be provided to everyone in the catchment area, regardless of whether they wish to purchase commodities. Possible commodities are: 35 Commodity Associated Health Education Comments Iodized salt IDD, nutrition Price will need to be subsidized ORS Diarrhea Discussions with MOH needed to develop policy for sale of MOH packets; otherwise, locate non￾MOH supply source soap Diarrhea, dysentery/typhoid nailbrushes Diarrhea, dysentery/typhoid food covers Diarrhea, dysentery/typhoid Ready-made plastic, or VHVs could be taught to make out of used bed nets and bamboo impregnated mosquito nets Malaria Bed and hammock nets “baby” nets for infants/small children Malaria/dengue VHVs could be taught to make these from used mosquito netting and bamboo. Design so that can nets can be used on both flat surfaces and hammocks. mosquito coils, repellent Malaria and dengue PSI malaria kits Malaria Villages without drug sellers water jar covers Dengue New water permeable ones are available from the national Program larvicide Dengue safe birth kits, iron Pregnancy and delivery Iron/folate Pregnancy and delivery scabies soap/scabicide Scabies/skin infections povidone iodine , gauze or swabs Wound care gentian violet and swabs Skin infections C. Health System Strengthening 1. If other NGO assistance for the Referral Hospital does not materialize, PFD should seek the necessary funds and staff to provide this. At least one expatriate manager and two experienced local clinicians (one of them a secondary midwife) would be needed. 2. Planned training of midwives in HCs should go forward, but serious consideration should also be given to inclusion of hospital midwives. 3. PFD should explore alternatives to HC staff travel to villages for immunization by piloting means of subsidized transport to bring villagers to the HC on specific das of the month per village, using locally appropriate means of transport: ox-cart, motorcycle cart, boat etc. Provision of means of transportation to a villager selected by the community in exchange for providing this transportation on specified days (in lieu of payment) might be considered. 36 4. The present VSO nurse position is essential and should be retained. Consideration should be given to changing it to a salaried position in the interests of continuity since the incumbent will complete a three year volunteer stint in April. 5. PFD should continue to advocate with the PHD for allocation of additional staff to Damrei Pong HC and consider assisting in provision of relocation costs or housing if a potential transfer can be identified. 6. PFD should work actively with the PHD and central MOH to ensure that the conditions in Snuol HC catchment area are addressed in the forthcoming revision of the Health Coverage Plan. 7. PFD should pursue the possibility of a World Bank-funded equity fund for the hospital (planned in the next WB Project with locations not yet determined). D. TBA Training 1. TBA training should be undertaken as soon as possible in interventions that do not require availability of emergency obstetric care, with particular attention to immediate care of the newborn to establish a patent airway and prevent hypothermia, hypoglycemia. 2. If and when emergency obstetric care is available, specific referral and partnership mechanisms linking TBAs with the hospital should be developed, including arrangements which allow the TBA to maintain a (non-medical) role during the hospital delivery E. Program Management 1. Integration of the RH and VHV volunteer functions should be promoted through utilizing VHVs whenever new or replacement RH workers are needed and vice versa. 2. Program management should be integrated so that each PFD Program Officer monitors and supervises all activities in a geographical area: RH, VHV education, malaria. 3. PFD national staff should receive training in supervisory techniques and management skills, preferably provided at the work site, and there should be a gradual devolution of management responsibilities to national staff over the remainder of the grant period, so that by the end of the current grant the expatriate functions in a primarily advisory capacity F. Monitoring and Evaluation 1. Supervisors should be instructed to review all deaths in the past year with their VHVs and ensure they are recorded on the register, and continue to ask about deaths on a regular basis. 37 2. Verbal autopsy for maternal and child deaths should be conducted annually after compilation of mortality statistics from VHVs. However, the current should be substantially revised to ascertain the exact age at death and include detailed questions appropriate to each age group (neonatal, post-neonatal and child). Immunization status should be included along with duration and severity of symptoms and temporal relationship of each symptom mentioned to death. Diarrhea should be described in terms of quantity, duration and type (watery, mucoid etc). Presence of chills with fever, cough, and any type of skin lesion should be specifically probed. Verbal autopsy forms should be completed by the same staff who will conduct the analysis and assignment of cause of death. These need to be experienced clinicians and expatriate TA will be required, either from the VSO nurse or a consultant. 3. In Hearth villages, growth monitoring should be continued after the end of NERPS throughout the grant period and careful records kept of anthropometrical data. In addition, mortality data should be maintained specific to those villages, based on VHV registers with a rapid house-to-house survey for verification. Baseline data should be constructed in the same fashion using VHV registers from May 2001 – April 2002, verified by household survey. 4. Future household surveys should be done as a random sample stratified by HC catchment area with number of households in each proportional to the population but including sufficient households to be statistically representative at HC catchment level. This will produce a self-weighting estimate of indicators for the population as a whole plus “lot” specific estimates allowing comparison between supervision areas and identification of low performers. G. Other: 1. Community health education should be conducted using techniques such as drama or puppet shows on topics that require a broad community response to address, e.g. environmental sanitation issues. Entertainment should be followed by group discussion and analysis of the village situation, perhaps using videos or photographs of households and market areas, followed by an action plan to rectify identified problems. 2. The PFD Water and Sanitation Program should shift emphasis in Chhlong from water to sanitation, in order to reduce the incidence of dysentery and typhoid, which between them account for almost 10% of under5 deaths. 3. The planned iron distribution pilot should (a) use an iron/folate preparation rather than iron alone, given the contribution of malaria to anemia in this population; (b) be limited to approaches with a realistic potential for scale up if successful; and (c) keep the design as simple as possible, preferably just one intervention which could be revised if needed based on experience rather than multiple approaches. 38 Annex I: Scope of Work PARTNERS FOR DEVELOPMENT United States: 1320 Fenwick Lane, Suite 406, Silver Spring, MD 20910 Tel: 301-608 -0426, Fax: 301-608-0822, E-Mail: pfdinfo@partnersfordevelopment.org Cambodia: No. 24, Street 294, BP 551, Phnom Penh, Kingdom of Cambodia Tel./Fax: 855-23-213-335, E-Mail: PFD.Cambodia@ BigPond.com.kh Location: Chhlong OD, Kratie Province. Duration: September 2002. (LOE: 15 working days) Supervisor: Country Representative/Program Manager for Health. I SUMMARY Northeast Cambodia Child Survival Program (NCCSP) Goal: To improve the health of children less than five years old in Chhlong OD, by focusing on control of diarrheal disease, immunization, nutrition and micronutrients, and promotion of breastfeeding. In 1998, PFD expanded its focus on health interventions in Kratie, with implementation of a USAID-funded Child Survival Program in the Chhlong Operational District, with a total population of 129,000 covering a target population of 14,500 children under five and 27,000 women. This coverage accounts for 48% coverage of the target population of Kratie Province. The aim of the Child Survival Program is to improve the health of children under five years through a three-pronged approach – by educating mothers on improved care-taking and preventive behaviors; by training health center staff on the treatment and management of childhood illnesses; and by building capacity of the operational district managers, to plan and implement health center and outreach activities. The Child Survival Program's key interventions focus on immunization, control of diarrheal disease, nutrition, and breastfeeding. The Program is funded to March 2004 by USAID Global Health Bureau in Washington DC. The CS Program is coordinated by a full-time expatriate coordinator based in Chhlong OD, with a current staff complement of 1 VSO Public Health Advisor, 10 Program officers and 2 administrative staff. Historically, (PFD) implemented its two-year entry-level Child Survival Program, the Northeast Cambodia Child Survival Program (NCCSP), from October 1, 1998 to September 30, 2000. The accomplishments and constraints of the first year of the Program are described in the annual report, submitted in October 1999 to USAID. During the second year of the entry grant, PFD focused its efforts on finalizing Program planning, consolidating partnerships, and conducting the community organizing work needed to implement the 42-month Child Survival Program, that succeeds the entry grant. 39 In October 2001, PFD commenced Phase 1, for full implementation of the Child Survival Program. The Detailed Implementation Plan outlines the overall framework for the full CS Program while also detailing the annual workplan and timeframes proposed. The Annual Report for Phase 1 (Oct 2000- September 2001) describes the achievements and lessons learned to date. Northeast Cambodia Reproductive Health Program Goal : to improve the health status of men and women in Chhlong OD by increasing access to and availability of high quality reproductive health education and services at community level, and by reinforcing linkages between the community and the district health system. To complement PFD's Child Survival Program and to address the reproductive health needs expressed by community members during PRA activities, PFD initiated the Northeast Cambodia Reproductive Health Program (NCRHP) in Chhlong Operational District in April 1999. NCRHP started as a two-year CBD pilot Program funded by UNFPA to serve as an operations research tool for Ministry of Health. The CBD project is implemented in collaboration with health centers in the catchment areas, whereby HC staff serve as trainers and supervisors for the Program. The aim is to develop the capacity of health center midwives and operational district staff to implement and manage the Program in the longer term. The Program has now gone to scale with expansion to 100 target villages in Chhlong OD with the inclusion of a Male CBD Program to complement the original Female CBD Program. The Female CBD Program is conducted by trained agents from each community, who were selected by the respective communities, using MOH selection criteria. The Program provides training on HIV/AIDS and STDs for all CBD agents, using training materials specially developed and adapted to the cultural context, by the PFD staff. The training manual will be shared with other NGOs and training agencies to strengthen the national training resources for reproductive health and HIV/AIDS with a unique focus on the rural population of Cambodia. Following intensive training, the CBD agents conduct regular group education sessions in their village on birth spacing, HIV/AIDS transmission and prevention and where to seek medical treatment for STDs. The male CBD workers educate men about birth spacing, counsel men about male involvement in birth spacing, and promote STD/HIV prevention. The Program is managed by a Khmer Doctor who works with two other staff to implement the Program. The Program is funded by UNFPA and executed by MOH through to December 2003. In addition to field implementation. PFD participate actively in contributing to shaping government policy for IMCI and Reproductive health through the MOH IMCI working groups. This forum provides the opportunity for PFD to feedback from lessons learned in the field and contribute models of best practice for both public and private healthcare practitioners. PFD also participate as a representative voice for implementing NGOs in the Rollback Malaria Technical Advisory group. This provides the opportunity to advocate for the remote rural populations that PFD serve and represent their interests and needs for malaria control while equally contributing to improvement in the technical design of malaria Programs. 40 II SCOPE OF WORK A. Purpose:: 1. Determine the achievement, or progress towards achievement, of the Child Survival Program’s quantitative and qualitative objectives and planned outputs. 2. Identify factors contributing to and constraining the achievement of same. 3. Make appropriate recommendations to address any identified weakness and/or constraints. B. Specific Tasks: In achieving the stated Purpose, the Evaluation Team will specifically respond to each of the following questions, including explanation of factors contributing to or constraining performance and recommendations for improvement where appropriate: 1. Are the village health workers (VHVs) effective as a channel for community health education? 2. Are the training, support, incentives and supervision provided to the VHVs by PFD effective and appropriate in enabling them to carry out their functions in the community? 3. Are the Project’s activities, strategies and approaches effective in enabling target communities to effect improvement in their own health, through improved household practices and improved health care seeking behaviors? 4. How and to what extent have Project activities and approaches contributed to the following: · community participation and community empowerment? · linkages between the community and the health system? · activities of other VHVs, TBAs and FBCs at village level? · strengthening of the Operational District health system? · OD health staff perceptions of their roles and responsibilities in terms of their work with the communities? 6. How effective has PFD been in providing the following to national staff and associates: · Training appropriate to Project job requirements? · On- the- job coaching and mentoring? · Understanding of the goals and objectives of the Child Survival and related Reproductive health Program, including both the desired Program impact, and the broader goals of sustainability and community empowerment?. 41 · Appropriate and manageable workloads? · A positive attitude towards their work for PFD, the target communities, and the public health system? 7. Have constructive working relationships been established and fostered among and between PFD managers, PFD staff, associate staff, government counterparts and community leaders? 8. Do the initial needs assessments and baseline data provide an adequate base for development and implementation of Project activities, and have they been effectively employed for this purpose? 9. Is the Project’s Community Surveillance system: · User-friendly and well understood by both staff and village health workers?. · Designed so as to yield reliable information for Project monitoring? · Effectively collected, recorded, reported and used by PFD and Health System staff? · 10. Assess the level of integration and synergy between the Child Survival Program and the Reproductive Health Program, and make any necessary recommendations to increase it. 11. Has the Project effectively coordinated with the public sector (PHD, PDRD, POE), INGOs, LNGOs and other organizations working in the field of Child Survival and IMCI? 12. What is the likelihood of and feasibility of Program benefits being sustained beyond the period of PFDs assistance, and how might this be enhanced? 42 Annex II: Documents Reviewed Ministry of Health, Guidelines for Operational Districts, MOH, Phnom Penh 1997. Ministry of Health, Health Coverage Plan, MOH, Phnom Penh 1996. Ministry of Health and Ministry of Planning, Cambodia Demographic and Health Survey 2000, MOH/MOP, Phnom Penh 2001. Ministry of Planning, Cambodia Socio-Economic Survey 1999, MOP, Phnom Penh 2000. Ministry of Planning, Cambodia Poverty Assessment, MOP, Phnom Penh 1999. Ministry of Planning, General Population Census of Cambodia 1998, MOP, Phnom Penh 1999. Partners for Development/Cambodia, Annual Report of Child Survival Program, Kratie Province, Cambodia, PFD, Phnom Penh 2002. Partners for Development/Cambodia, Health Center Resource Survey, Chhlong Operational District, Kratie Province, PFD, Phnom Penh 2001. Partners for Development/Cambodia, Hearth Model Lessons Learned, PFD, Phnom Penh 2002 Partners for Development/Cambodia, Hearth Model Trainer’s Guide, PFD, Phnom Penh 2002. Partners for Development/Cambodia, Northeast Cambodia Child Survival Project, Additional Annex to Detailed Implementation Plan, PFD, Phnom Penh 2000. Partners for Development/Cambodia, Northeast Cambodia Child Survival Project Baseline Survey on Knowledge, Practice and Coverage (KPC), PFD, Phnom Penh 1999. Partners for Development/Cambodia, Northeast Cambodia Child Survival Project Detailed Implementation Plan, PFD, Phnom Penh 2000. Partners for Development/Cambodia, Revised Proposal to UNFPA and the Cambodian Ministry of Health for the Northeast Cambodia Reproductive Health Program, PFD, Phnom Penh 2001. Partners for Development/Cambodia, VHV Trainer’s Guide: Immunization, PFD, Phnom Penh 2001. Partners for Development/Cambodia, VHV Trainer’s Guide: Diarrhea, PFD, Phnom Penh 2001. Partners for Development/Cambodia, VHV Trainer’s Guide: Nutrition and Breastfeeding, PFD, Phnom Penh 2002. White, Patrice, Crossing the River: Traditional Beliefs and Practices of Khmer Women During Pregnancy, Birth and Post-Partum, PFD, Phnom Penh 1995. World Bank, A Poverty Profile of Cambodia, World Bank Discussion Paper No. 373, Washington DC 1997. 43 Annex III: LIST OF PERSONS MET Ministry of Health Kratie Provincial Health Department Dr. Cheam Saem Director of PHD Dr. Chuong Sengly Deputy Director of PHD Ngy Bulen PHD EPI chief Long Mardy PHD Continuing Education Training Chief Pech Seima PHD MCH chief Mao Sophalla Deputy Chief or PHD Bureau Technique Hem Sothy PHD HIS Chief Ministry of Health Chhlong Operational District District Management Team Chhneang Sovutha Director of Chhlong OD Ros Rithy OD Health System Cheng Sam Ath OD EPI chief Health Centers Meav Ramo Chambok HC Chief (VHV Supervisor) Tuon Rinda ChamBak HC staff (VHV Supervisor) Por Songly ChroyThmor HC Chief (VHV Supervisor) Chhun Samoeun Chroy Thmor HC staff (VHV Supervisor) Koh Sok Heang Chroy Thmor HC staff Chak Chin Damrei Pong HC Chief (VHV Supervisor) Thiv Tharith Kanchor HC Chief (VHV Supervisor) El Rany Kanchor HC Staff (VHV Supervisor) Moth Sam An Ksach Andett HC Chief Thuch Hong Kry Ksach Ande HC staff (VHV Supervisor) Heng Sokha Ksach Ande HC staff (VHV Supervisor) Nuon Nearng Ksim HC Chief (VHV Supervisor) Khen Chy Ksim HC Chief (VHV Supervisor) Hong Narin Pong Ror HC Chief (VHV Supervisor) You Narith Pong Ror HC staff (VHV Supervisor) Morm Vichett Prek Prosob HC Chief Leng Chanthy Prek Prosob HC staff (VHV Supervisor) Seng Chantrea Prek Prosob HC staff (VHV Supervisor) Orng Vannara Snoul HC Chief Theam Phanith Snoul HC staff (VHV Supervisor) Chhiv Tithya Snoul HC staff (VVH Supervisor) 44 Annex III: LIST OF PERSONS MET - continued Ministry of Health Chhlong Operational District Health Centers – continued - Prak Chan Nareun Snoul HC staff (VHV Supervisor) Men Sovann Tamao HC Chief (VHV Supervisor) Partners For Development Ann Canavan Health Program Manager, PFD Phnom Penh Michelle Lang Child Survival Program Coordinator, PFD Chhlong Dul Setha Child Survival Team Leader, PFD Chhlong Maria Doyle VSO Public Health Advisor, PFD Chhlong Chim Hen Child Survival Program Officer Hoeum Heoun Child Survival Program Officer Ngoun Ly Health System Strengthening Program Officer, PFD D Chhlong Yim Nady Child Survival Program Officer Ya Saroeun Child Survival Program Officer, PFD Chhlong Uch Sarom Health System Strengthening Program Officer, PFD Chhlong Keang Sochany Child Survival Program Officer Dr. Kiv Sokha Child Survival Physician Kim Seng Yada Child Survival Program Officer 45 Annex IV: Analysis of Mortality Data Methodology Village Health Volunteers maintained a register of all mothers and children in their catchment area (approximately 50 households), including pregnancies, births and deaths. The Program began in April-May 2001, so that by the end of 2001 data had been collected for about an 8 month period. VHVs, assisted by supervisors from the Health Center, completed a form for each death based on interviews with the mother or other caretaker for a total of 164 deaths in children under the age of five. The forms elicited the details in case of an accidental death and the presence or absence of the following symptoms in the case of deaths due to disease: fever, convulsions, diarrhea, dysentery, difficulty breathing, and malnutrition. Other observations and notes were written in on many forms. In addition to the 171 deaths of infants and children under the age of five, 8 deaths to women during pregnancy, delivery or in the post-partum period were reported, with description of the time of death relative to the pregnancy and symptoms. The total number of under five deaths recorded (171) is about 20% lower than the number that would be expected if the birth rates and mortality rates found in the 1998 census have continued unchanged. Some underreporting is therefore likely, although the difference may also reflect changes in fertility and mortality since the 1998 census. The number of maternal deaths reported is consistent with the generally accepted national estimate of 400 deaths per 100,000 live births, suggesting little underreporting of maternal death. The forms were reviewed by a nurse with a Masters degree in Public Health and in Nursing and extensive clinical experience in the region, and probable cause of death assigned based on the age of the child at death, the symptoms reported, the time of year, and prevalence of specific diseases in the village as reported by villagers, PFD staff, and Health Center staff. In cases where the assignment of probable cause of death was difficult, the reviewer consulted with three PFD staff, an expatriate registered nurse and two Cambodian secondary nurses familiar with local disease patterns and local terminology/beliefs. Findings The following tables show the probable cause of death for under fives and pregnant/postpartum women. It should be kept in mind that these are probable causes of death only, based on a review of verbal autopsy information, and that some errors are inevitable. Detailed notes on the criteria used for assigning deaths to each cause follow each table. 46 Table 1: Deaths to Infants and Children Under the Age of Five Chhlong OD May- December 2001 Probable Cause of Death Number Percentage ARI 43 25% Peri-natal 20 12% Meningitis/encephalitis 19 11% Malaria 17 10% Neonatal Tetanus 15 9% Typhoid/other infectious 10 6% GI Other 8 5% Accidents 7 4% Diarrhea 6 4% Dysentery 5 3% DHF 4 2% Malnutrition/Starvation* 3 2% Sepsis 3 2% Unknown 11 6% TOTAL 171 100% * Refers to malnutrition as the sole cause of death. Malnutrition as a contributing factor to death from infectious disease is not reflected and, based on known nutritional status in this population, can be assumed to be considerable. Notes: ARI: Deaths were classified as due to ARI if difficulty breathing, with or without high fever, was described in the absence of other likely causes. Although cough was not included in the list of symptoms on the verbal autopsy form, a number of the cases wrote it in under “other”. There is no way of determining how many of the ARI deaths were due to post-measles pneumonia or other immunizable diseases. It is possible that in some cases the difficulty breathing was an agonal event rather than a primary symptom and the death actually due to non-respiratory causes. However, the number of cases is so large that the reviewer feels confident in stating that ARI is clearly the most common cause of death. This impression is consistent with reports from villagers and health facilities. Peri-natal causes: the 20 deaths ascribed to peri-natal causes include 17 in which the infant died within a few hours of birth, one which died within 24 hours, and 2 who died after 3-4 days with the notation that the baby had been weak with no sucking reflex from birth. A number of the deaths immediately after birth included notations that the baby was cyanotic at birth. No information was available regarding length of gestation or birth weight. 18 of the births occurred at home and two in a private clinic or home of a private midwife. Meningitis/Encephalitis: deaths were assigned to this cause if the child had convulsions, with or without high fever, was beyond the age for neonatal tetanus, and either did not live in a village with high prevalence of malaria or died during the season when malaria is rare. While meningitis/encephalitis are noted as significant causes of death in national facility-based death statistics and in the only other report based on rural surveillance (RACHA 2000), the number is surprisingly large. Even allowing for some misclassifications (e.g. cerebral malaria occurring off￾season, or febrile seizures accompanying another infectious disease process), it would seem 47 that there is an unusually high prevalence of such deaths in this population and further investigations are warranted, particularly in terms of identifying possible transmission of Japanese B encephalitis. One of the reports mentioned the death of another family member with the same symptoms. Pig farming is widespread in the area. Malaria: Deaths were classified as probable malaria if high fever was reported without respiratory symptoms and the death occurred during the rainy season in a village known to have high endemicity. A few of the cases specifically noted “malaria” in the margin and one had been to a health facility and had a positive malaria test. Most died at home without a blood smear or dipstick done. Some misclassification is likely and the actual number of deaths due to malaria may be either somewhat higher (due to deaths from malaria occurring “off season” or in villages with low prevalence, which will have been classified as “other infectious”) or lower (due to some of the deaths ascribed to malaria actually being from other infectious causes). However, the reviewer feels confident in citing malaria as one of the six major causes of infant/child death for this population, whatever the exact percentage. Neonatal tetanus: of the 15 cases classified as probable neonatal tetanus, 13 of the forms specifically stated the death was due to neonatal tetanus (in a child of the right age) and two others described death from convulsions in an infant less than one month of age. It is possible that a few were actually meningitis or encephalitis deaths, especially since the forms did not specify exact age in days in most cases. Typhoid and Other Infectious Diseases: the ten deaths ascribed to this group featured high fever as the sole symptom, occurring either in a non-malarious area or in the cool or dry season when malaria is infrequent. Typhoid is known to be hyper-endemic in Cambodia, and more than half of these cases reported continuous high fever of several days duration. In the absence of laboratory tests it is not possible to be sure, but it is probable that some were indeed due to typhoid or paratyphoid, with the remainder other infectious diseases. It is also possible that some malaria, occurring either “off season” or in areas of usually low prevalence, is misclassified in this group (see under Malaria above). Gastro-intestinal – other: deaths in this category consist of infants and children without diarrhea but with clear GI symptomatology, e.g. hard and rigid abdomen, abdominal distention, abdominal pain, severe vomiting .Causes may include intestinal obstruction or peritonitis. Accidents: the seven deaths in this category include four drownings, one fall from a bicycle, and two poisonings Diarrhea: Deaths were ascribed to diarrhea if the child was reported to have had diarrhea, with or without fever, prior to death and lacked signs of respiratory infection or convulsions. The number is relatively low, which is consistent with villager and Health Center reports that diarrhea is not a major cause of mortality. PFD has installed deep tube wells in most of these villages under it’s Northeast Cambodia Community Development Program. Dysentery: Deaths were ascribed to dysentery if the child was reported to have had bloody stools, with or without fever, prior to death. The term for dysentery in Khmer is quite specific and distinct from the term for watery diarrhea, making confusion between the two unlikely. Dengue Hemorrhagic Fever: DHF has moved into the rural areas of Cambodia over the past two years and is widely reported to be a significant health problem in the villages. The four 48 deaths classified as DHF featured high fever and hemorraghic manifestations, e.g. petechiae or ecchymosis. One child was reported to have vomited blood. Malnutrition/Starvation: the deaths in this group include a child with cleft lip/palate, a baby who was bottle fed, and a 7 day old infant with severe thrush from birth which prevented intake. All three of these cases clearly died from primary malnutrition/starvation and/or dehydration. However, malnutrition as a contributing factor is not reflected in this data and, based on what is known about nutritional status in the population, probably played a role in many of the deaths from infectious disease. Sepsis: Deaths classified as sepsis featured fever, blistering or rash on the skin, or “black color on the body” occurring at ages or in seasons when DHF was unlikely. Some misclassification between DHF, sepsis, and “other infectious” is likely. Unknown: 11 deaths could not be classified based on the information available. In 10 cases, the only symptom noted by the family was crying. In one case, the only symptom noted was blisters on the lips on the day of death. All cases were less than one year of age. Table 2: Maternal Deaths, Chhlong OD May-December 2001 Probable Cause of Death Number Direct Obstetric Cause: prolonged/obstructed labor; possible uterine rupture 4 Eclampsia 1 Post-partum Infection 1 Severe Anemia/cardiac 1 Non-maternal causes 1 Notes: The four women classified as dying from direct obstetrical causes all died during labor, one of them while undergoing an emergency Cesarean section at the hospital. The woman classified as eclampsia died during labor with edema and convulsions. The woman classified as post-partum infection died 8 days after delivery with a high fever. One woman died 37 days post-partum having had significant generalized edema for the entire postpartum period. In the reviewer’s Cambodian experience, such women are usually severely anemic and the death has therefore been classified as due to anemia/heart failure, although this is purely speculative and other pathology (e.g. kidney disease) cannot be ruled out.