Medical Care Development International Ndwedwe District Child Survival Project Cooperative Agreement No. FAO A-00-97-00025-00 FINAL EVALUATION REPORT Project Location: Ndwedwe District, KwaZulu Natal, South Africa Project Duration: September 25, 1997 – September 30, 2001 Submitted to: USAID/BHR/PVC Child Survival Grants Program Ronald Reagan Building 1300 Pennsylvania Avenue Washington, DC 20523 Submitted by: Medical Care Development International 1742 R Street, NW Washington, DC 20010 USA Tel: (202) 462-1920 Fax: (202) 265-4078 Email: mcdi@mcd.org March 2002 Prepared By: Dr Edmund McGrath, External Evaluator Ndwedwe District Child Survival Project Cooperative Agreement No. FAO A-00-97-00025-00 FINAL EVALUATION REPORT March 2002 Ndwedwe District, KwaZulu Natal, South Africa Dr Edmund McGrath, External Evaluator Contact Person at Headquarters: Contact Person in the Field Office: Joseph Carter, Director Dr Farshid Meidany, Project Manager Medical Care Development International Ndwedwe District Child Survival Project 1742 R Street, NW 1401 Maritime House, 143 Victoria Embankment Washington, DC 20010 USA Durban, South Africa Tel: (202) 462-1920 Tel: 27-31-304-0365 Fax: (202) 265-4078 Fax: 27-31-304-0386 Email: mcdi@mcd.org Email: mcdi@mweb.co.za TABLE OF CONTENTS ACRONYMS AND ABBREVIATIONS ....................................................................................1 ACKNOWLEDGEMENTS ........................................................................................................2 A. SUMMARY...........................................................................................................................3 B. ASSESSMENT OF RESULTS AND IMPACT OF PROGRAM............................................9 1. SUMMARY CHART ................................................................................................................9 2. TECHNICAL APPROACH.......................................................................................................11 a. Overview........................................................................................................................11 b. Interventions...................................................................................................................13 i. Immunization...............................................................................................................14 ii. Control of Diarrheal Disease .......................................................................................16 iii. Pneumonia Case Management....................................................................................18 iv. Maternal and Newborn Care.......................................................................................20 v. HIV/AIDS...................................................................................................................24 c. New Tools and Approaches ............................................................................................28 3. RESULTS: CROSS-CUTTING APPROACHES............................................................................29 a. Community Mobilization and Capacity Building ............................................................29 b. Communication for Behavior Change .............................................................................32 c. Capacity Building Approach...........................................................................................35 i. Strengthening the PVO.................................................................................................35 ii. Strengthening Local Partner Organizations..................................................................36 iii. Health Facilities Strengthening...................................................................................42 iv. Strengthening Health Worker Performance/Strengthening Training............................44 d. Sustainability Strategy....................................................................................................46 C. PROGRAM MANAGEMENT.............................................................................................47 1. PLANNING..........................................................................................................................47 2. STAFF TRAINING.................................................................................................................48 3. SUPERVISION OF PROGRAM STAFF.......................................................................................50 4. HUMAN RESOURCES AND STAFF MANAGEMENT..................................................................50 5. FINANCIAL MANAGEMENT..................................................................................................51 6. LOGISTICS ..........................................................................................................................52 7. INFORMATION MANAGEMENT.............................................................................................52 8. TECHNICAL AND ADMINISTRATIVE SUPPORT.......................................................................53 9. MANAGEMENT LESSONS LEARNED .....................................................................................55 D. OTHER ISSUES IDENTIFIED BY THE TEAM.................................................................57 E. CONCLUSIONS AND RECOMMENDATIONS.................................................................58 1. CONCLUSIONS ....................................................................................................................58 2. RECOMMENDATIONS...........................................................................................................59 ATTACHMENTS.....................................................................................................................63 1. FINAL EVALUATION TEAM MEMBERS .................................................................................64 2. ASSESSMENT METHODOLOGY.............................................................................................65 4. SPECIAL REPORTS...............................................................................................................68 a. Summary of Rapid Test Cost-Effectiveness Study b. Summary of the DramAidE External Evaluation c. Full Report: Investigating HIV/AIDS-impacted Health Beliefs and Behaviors Regarding Pediatric Pneumonia Ndwedwe District Child Survival Project Final Evaluation 2001 1 ACRONYMS AND ABBREVIATIONS AIDS Acquired Immune Deficiency Syndrome BCC Behavior Change Communication BHR/PVC Bureau for Humanitarian Response/Private Voluntary Cooperation CABA Children Affected by AIDS CHC Community Health Committee CHW Community Health Worker CHWP Community Health Worker Program CS Child Survival CSP Child Survival Project DHSMT District Health System Management Team EPI Expanded Programme on Immunisation GIS Geographic Information System HBC(V) Home-Based Care (Volunteer) HIS Health Information System HMIS Health Management Information System HIV Human Immuno-Deficiency Virus IEC Information, Education and Communication KPC Knowledge, Practice and Coverage KZN KwaZulu Natal (Province) LAS Local Authority Structures LOE Level of Effort MCDI Medical Care Development International MOH Ministry of Health NDCSP Ndwedwe District Child Survival Project NPPHCN National Progressive Primary Health Care Network PATH Program in Appropriate Technologies for Health PEP Perinatal Education Program PLWA Persons Living with AIDS RSA Republic of South Africa SHC School Health Club STI Sexually Transmitted Infection TREE Training Resources in Early Education TAA Tribal Authority Area TT Tetanus Toxoid (vaccination) USAID United States Agency for International Development WRA Women of Reproductive Age Ndwedwe District Child Survival Project Final Evaluation 2001 2 ACKNOWLEDGEMENTS The external evaluator wishes to express sincere gratitude to all who participated in the evaluation and who facilitated the process in so many ways. The entire project team embraced the evaluation with a remarkable degree of openness, honesty and enthusiasm which aided the evaluation enormously. The willingness of busy DOH and partner representatives to engage with the evaluation process was indicative of the close co-operation that had been developed with the NDCSP and this was reflected in the eagerness for the continuation of the project expressed by all persons interviewed. Special thanks are due to Thuli Ngidi and Alyssa Wigton for their special contribution to the evaluation and production of the report. The evaluator wishes to thank MCDI and USAID for the opportunity to be involved with this exciting project set in a wonderful part of KwaZulu Natal. Ndwedwe District Child Survival Project Final Evaluation 2001 3 A. SUMMARY The Ndwedwe District Child Survival Project (NDCSP) was initiated as a two year planning grant under USAID/BHR/PVC funding in 1995, which led to a full-scale project between 1997- 2001. The goals of this project were to reduce morbidity and mortality amongst children under 60 months of age and to improve the health status of women of childbearing age by reducing the risks associated with reproduction. In 2000, additional funding was secured to expand the project’s initial HIV/AIDS activities in response to the rapidly escalating HIV/AIDS epidemic in South Africa, KwaZulu Natal Province (KZN) being at the epicenter of the epidemic. The project is located in the Ndwedwe District, KwaZulu Natal Province, South Africa and includes some of the poorest deep rural communities found in the country. The principal partner during the project period was the Department of Health at the provincial and district levels, particularly the District Health System Management team (DHSMT). Productive partnerships were forged with both US and locally based non-governmental organizations (NGOs). The NDCSP was recently awarded a four-year cost-extension to continue with the existing Child Survival (CS) and Women of Reproductive Age (WRA) interventions with important new elements and more intensive HIV/AIDS/STI interventions in an extended project area resulting from the newly established District boundaries. The project’s objectives and achievements by intervention area are summarized in Table 1. Some of the Project’s main accomplishments, lessons learned, and challenges are outlined below, along with conclusions and recommendations from this evaluation. Main Accomplishments District Health System Development · The NDCSP contributed in large measure to the substantial progress that has been made in the establishment of the community base of a District Health System (DHS) in support of the Community Health Worker Program, launched by the Provincial DOH in 1999-2000. · The NDCSP is fully incorporated in the operations of the DHSMT and solid partnerships have been developed with key DOH programs at both regional and district levels. · The NDCSP is well integrated with other key partners of the DOH, operating within the framework of the DHSMT. · A highly motivated and experienced project team has been assembled which is poised to play a major role as a lead partner of the DOH in the further development of DHS in the new sub￾District during the extended phase (Phase II) of the project. Improving Health Status, Knowledge, and Behaviors · The majority of project targets were met during the lifetime of the project and substantial achievements were made in most, but not all, intervention areas where targets were not met. · HIV/AIDS activities have been incorporated into all primary project activities and a significant amount of training in HIV/AIDS for health workers and community volunteers has been conducted. Ndwedwe District Child Survival Project Final Evaluation 2001 4 · Home Based Care Volunteers (HBCVs), deployed in selected areas of the District, have been regularly supervised and HIV/AIDS awareness has been introduced to ten secondary schools. Community Development · The core group of Traditional Birth Attendants (TBAs), trained by the NDCSP as one of the primary project activities, remains very active and their role has evolved with the project. · Significant progress has been made in engaging with communities through church groups and tribal structures, both of which are highly influential in traditional Zulu culture. Strengthening Health Facilities · Progress was made in improving health worker performance and in this process the NDCSP helped to introduce methodologies and procedures for service supervision in a District where supervisory structures are fragmented and not yet fully developed. · Rapid tests for HIV have been introduced in two hospitals and for syphilis in one clinic. Major Lessons Learned · The importance of engaging with communities through the prevailing principal authoritative structures from the outset. · The need to define clear methodologies and procedures for community engagement and the need to work to an agreed Action Plan when multiple agencies are simultaneously involved. · The importance of ensuring that communities are adequately prepared when changes in health worker practice are introduced. · The need for continuous follow-up and on-site supervision in promoting the development of new skills and simultaneously helping health workers to ‘un-learn’ the old skills and methods being replaced. · Ensuring that persons selected to engage with communities have an adequate level of training in health messages and strategies, especially if they do not have a health background. · The need for partners to be aware of DOH Business Plans when engaging with communities on behalf of the DOH. · The importance of including the whole project team in decision making and of assigning responsible roles to individual members when designing and implementing the project Action Plan. Major Challenges That Need to be Addressed during Phase II · Rolling out the Phase 1 core activities to the newly extended project area which has a poor health service infrastructure. · Ensuring sustainability of the DHS through capacity building at community level of CHCs, CBOs, church groups etc. and institutional strengthening of these structures. · Coping with the competition for authority at community level between the Traditional structures and the newly established Local Government structures. · Rolling out key quality improvement initiatives such as IMCI and PEP to all service delivery points. · Translating the gains achieved in immunization, CDD and Maternal and Newborn Care to other key intervention areas, particularly Pneumonia Case Management and HIV/AIDS. · Developing and implementing a comprehensive community training program in the sub￾District. Ndwedwe District Child Survival Project Final Evaluation 2001 5 · Developing strategies to reduce the vulnerability of orphans and young women who are at particular risk in the escalating HIV/AIDS epidemic. · Adapting Behavior Change Communication (BCC) to counteract prevailing attitudes and practices that enhance the risk of HIV infection. · Developing strategies aimed at improving care for PWLAs and OVCs in the community. · Developing strategies to tackle the very high prevalence of child abuse in the District. · Harnessing the potential contribution of community health workers, both paid and voluntary, in the development of a community database. Priority Conclusions Resulting from This Evaluation 1. This evaluation fully endorses the decision of USAID to extend the NDCSP for a further four years, based on the success of the project to date in terms of design, implementation and outcomes. The Provincial DOH has high expectations of the extended Project and has specified some key areas for particular emphasis. 2. All core Project activities were assiduously pursued, substantial progress is evident in all key areas and high levels of success were attained with most but not all, Project targets. 3. Commendable success is evident in the manner in which the HIV/AIDS components were merged with the primary components of the project and the entire project team is clearly focused on the challenges presented in merging the two components. 4. The NDCSP has made a large contribution to the creation of a functional DHS through capacity building for health at community level. The essential organizational structures have been established and community based health cadres, paid and voluntary, are active. Although both the structures and cadres are still fledgling, it is the Provincial intent that the Community Health Worker Program (CHWP) should be fully consolidated and the NDCSP has been entrusted with a major role in this process by the DHSMT during Phase II. 5. The DOH has indicated that the NDCSP will be entrusted with the development and introduction of systematic procedures and protocols for the operations of CHCs and CHWs. The CHCs will be trained to monitor and supervise community workers and community based health activities and to manage community resources, including community funds. It is the evaluator's opinion that the project team is fully capable of delivering this objective with appropriate technical support. 6. In contacts with the DOH and other principal partners it is evident that the NDCSP is highly regarded and that good interpersonal and professional relationships have been established. The planned on-going contribution of the project to the DHSMT during Phase II, across a spectrum of activities, is universally welcomed. 7. A number of key Provincial DOH developments under-pin project initiatives and enhance the likelihood of sustainability. These include: a) the Provincial CHWP Policy, b) the recent appointment of a Deputy Director for DHS Support to a post that had been vacant and c) the stated commitment of the Superintendent-General to the DHS initiative. 8. An experienced and dedicated project team is in place and the project is well positioned to expand its role as a principal partner of the DHSMT by helping to consolidate the community base of the DHS; by lending operational support to a number of key Programs – HIV/AIDS; Maternal and Child Health and Youth, Sexuality and Women’s Health in particular and by helping to improve health worker performance through supervision of the PEP and IMCI training initiatives. Ndwedwe District Child Survival Project Final Evaluation 2001 6 9. It is in the area of community empowerment for health that the NDCSP can make a major sustainable contribution through the introduction of community oriented health training. In this initiative the TBA training module developed by the NDCSP can be merged with the Bergville CSP developed IMCI Community Module and the standard HIV/AIDS training module to create a comprehensive community health training package that could become the Provincial standard. 10. The project team includes three highly experienced and dedicated health professionals who are well capable of leading the initiative outlined above. The senior of these has been with the project from the outset and a clearly defined role in health worker training has already been agreed with the DOH Program Principals for this officer during Phase II. The other two officers were recruited as supervisors of HBCVs in the District as a key element of the HIV/AIDS component. However the latter role by itself is not sustainable and an extended role needs to be developed to maximize the complementary skills that they bring to the project. They are eminently suited to develop and implement the community training initiative being planned for the District in addition to further developing the Home-based Care Program of the HIV/AIDS Program. 11. The two Community Organizers came to the project without any experience (one was identified through an internship placement with the project as a Sociology undergraduate) but have grown in confidence and experience and are now well versed in their roles and are respected for their contribution to the community empowerment process. As CHC mentor and Social Welfare intermediary, respectively, their contribution in Phase II can be substantial. 12. The replacement Project Manager comes with a substantial national program management background and is an experienced epidemiologist in the South African context. The level of engagement with DOH and other partners achieved in the brief time he has been with the NDCSP and his inclusive and responsive interaction with team members augurs well for the continued successful development of the project. Priority Recommendations Target Area and Project Foci 1) As indicated by the DOH, the NDCSP should continue to operate for the present within its existing boundaries, even though a part of the southerly area no longer falls within the newly established Ndwedwe sub-District of the Ilembe District (KZ 29). At the same time the NDCSP has been requested to extend its field of operations to the newly incorporated area of the sub-District to the northwest, which includes a District Hospital and 3 clinics. 2) The NDCSP offices are located in Durban, outside the Ndwedwe District and therefore removed from its sphere of operations, with the newly incorporated areas even further distant. In order to establish a higher community profile for the project it is recommended that an operations base should be established within the District. The newly constructed Ndwedwe Health Center would be an ideal location, given that it is sited at the heart of the District and now incorporates a ‘community facility’. The old Ndwedwe Clinic has been used throughout by the project for meetings and assemblies and the Superintendent General has offered to facilitate the establishment of a dedicated NDCSP office on this site. Ndwedwe District Child Survival Project Final Evaluation 2001 7 3) The core objectives of the NDCSP should remain the same with prime emphasis on child survival and reproductive health initiatives but with re-defined strategies to take account of the escalating HIV/AIDS epidemic. 4) The principal focus of the NDCSP in Phase II should be on consolidating the District Health System at community level by developing methodologies for specifying, quantifying and certifying the operations of the various structures and cadres operating at community level. In consultation with its partners in the DHSMT, the NDCSP should develop and introduce within the District, protocols and procedures appropriate to this process. Data for Monitoring and Surveillance 5) The Provincial HIS intends that a community database will be established as a key component of a functional HIS. Such a database will allow the geographic profile of the CHWP to be readily determined and to match the distribution, content and capacity of community and clinic-based health services with the socio-economic, demographic and epidemiological profile of the community. This will allow the DHSMT to monitor the DHS in terms of equity, productivity, efficiency etc. The Provincial HIS expects the NDCSP to play a lead role in this initiative in partnership with the DHSMT and the Provincial HIS. 6) The Superintendent General has a vision of a “community-data circuit” that would underpin the operations and management of the DHS. This information system would include demographic data (e.g. household size and structure), sociological data (e.g. persons living with HIV/AIDS (PLWAs) and orphaned and vulnerable children (OVCs)), epidemiological data (e.g. sentinel disease surveillance), vital data (e.g. unregistered births and deaths) and other key related data (e.g. maternal deaths) that could be used to generate a community health index . It is recommended that the Project, in co-operation with the DHSMT and the Provincial HMIS, should weigh the feasibility of piloting the instrument(s) for this data system in the District, as an essential tool for management, monitoring and evaluation within the DHS. Innovative Tools and Strategies 7) In developing and delivering health messages in the community the NDCSP should use the latest available tools e.g. the IMCI ‘community module’ now being developed by the Bergville CSP and new BCC methods. 8) In its continuing role in strengthening Health Worker performance the NDCSP should foster linkages between the health services and the community in every way possible so that a solid CHC-CHW-Clinic axis is established at the core of the DHS. 9) All future health worker training programs should be accompanied by a parallel program for CHWs, HBCVs and TBAs to ensure that communities can be informed of any intended changes in health worker attitudes and performance as an outcome of the training. 10) All available evidence suggests that the escalating HIV/AIDS epidemic will have a dramatic impact on child survival and quality of life for survivors. The vulnerability of children is further aggravated by reports of very high levels of child abuse in the District. The Project should develop strategies for quantifying and addressing these major negative influences on child survival and well-being. 11) Young women are especially vulnerable to HIV infection, carrying at least 1.5 times the risk of infection as their male counterparts. Their vulnerability is further increased by the accelerated disease process associated with pregnancy. The Project should help to address Ndwedwe District Child Survival Project Final Evaluation 2001 8 this major problem which further impacts on the status of OVCs through the design and conduct of KPC studies and the design and implementation of appropriate BCC strategies. Sustainability 12) The DOH should be encouraged to identify local counterparts for each Project staff member in the interest of capacity building and sustainability. Project Management 13)Responsibility and accountability should be delegated to each Project staff member for project initiatives and activities that fall within their scope of operations. Formal supervisory structures should be established now that the project team has grown in size and complexity. Any deviance from agreed action-plans should be cleared with the project Manager or delegated supervisor. 14) All Project staff should be engaged in developing and reviewing work-plans that impact on their areas of activity and responsibility and they should be kept informed of all developments that may impact on the project. 15) Home office interventions (e.g. special studies/analyses etc.), which directly impact on Project activities should be devised and implemented in consultation with local Project staff. 16) The Home Office Project Coordinator should have a regular local presence and visit the Project at least once yearly and have the opportunity to develop a technical as well as an administrative role in the project. Ndwedwe District Child Survival Project Final Evaluation 2001 9 B. ASSESSMENT OF RESULTS AND IMPACT OF PROGRAM 1. Summary Chart Table 1 Part 1: Original Program Grant Objectives (1996-2001) Objectives Baseline Final Immunization 1. Increase to 71% the % of children immunized against measles 51% 53%i 2. Increase to 50% the % of children 12-23 months who are fully immunized per vaccination cards 26% 47%i 3. Reduce to 25% the % of children 12-23 months who defaulted between the DPT1 and DPT3 doses 48% 8% 4. Increase by 20% the % of women who receive at least two TT vaccinations See note ii below 86% Control of Diarrheal Disease 1. Increase to 66% the % of mothers who give the same or more than usual amount of liquids during their child’s diarrheal episode 46% 71%iii 2. Increase to 65% the % of mothers who give the same or more food than usual amount of food during their child’s diarrheal episode 45% 57%iii 3. Decrease to 5% the % of mothers of children 0-23 months who say they don’t know what to do following a diarrheal episode 20% 7%iii Pneumonia Case Management 1. Increase to 55% the % of mothers who are able to recognize danger signs of pneumonia 35% Not available 2. Reduce to 35% the % of mothers who reduce or stop feeding for a child with rapid or difficult breathing 55% 75%iii 3. Increase to 95% the % of mothers who seek care at health facilities for children with rapid or difficult breathing 86% 86% Maternal and Newborn Care 1. Increase to 50% the % of mothers who know the danger signs of delivery that require giving birth at a health facility 10% 78% 2. Decrease to 50% the % of women who give birth outside health facilities 65% 15% 3. Among women seeking prenatal care, increase by 15% those who make at least two prenatal visits during the pregnancy 9% 75%iii 4. Increase to 70% the % of mothers who initiate breastfeeding within an hour after giving birth 51% 48%iii 5. Increase to 85% the percent of mothers who breastfeed within the first eight hours after birth 77% 96%iii 6. Increase to 45% the % of mothers of children under 4 months who are exclusively breastfeeding 33% 32%iii 7. Increase to 42% the % of mothers who are not pregnant, do not want another child in the next two years or are not sure, and are using modern methods of contraception 32% 67% Note: For Part I, Original Program Grant Objectives, the source for Baseline data is the 1996 KPC unless otherwise noted. Source for Final data is the 2000 KPC unless otherwise noted. Whenever possible, the most recent data available (from the Cost Extension 2001 Baseline KPC) has been used. i In the 2000 KPC survey, immunization rates for the children over one year of age were calculated on the assumption that a child without a Road to Health card is entirely un-immunized. Unfortunately, 40% of the caregivers surveyed could not produce a Road to Health Card (it is likely these cards were lost, misplaced, or discarded when the care of the child passed from the biological family to the caregiver). This suggests that many children with at least partial immunization may have been missed due Ndwedwe District Child Survival Project Final Evaluation 2001 10 to missing Road to Health Cards. Further, the NDCSP participated in a national measles immunization campaign during which approximately 16,000 children received measles immunizations (92% of a total 17,400 children under the age of 5 in the district). None of these measles immunizations were recorded on the children’s Road to Health Cards as a matter of national policy. Thus, it is assumed that measles vaccination coverage and full immunization coverage recorded during the final KPC do not reflect actual immunization rates. ii Maternal health cards (on which tenanus toxoid vaccinations are recorded) are kept by the clinics in which delivery took place. During the final KPC, estimation of TT coverage was based on maternal recall. iii Source: December 2001 KPC. Table 1 Part 2: HIV/AIDS Component Grant Objectives (2000-2001) Objectives Baseline (2000) Latest Data Available (2001) 1. 80% of high school students in which School Health Clubs (SHCs) are active have adequate knowledge of HIV/AIDS prevention as demonstrated in their ability to name at least two strategies of prevention Baseline data not collected by DramAidE 74%i 2. 50% of high school students with active SHCs report adoption of one of three strategies of HIV/AIDS prevention (abstinence, being faithful, condom use) Baseline data not collected by DramAidE 90%i 3. 80% of mothers can recognize at least two known ways in which a mother can transmit HIV/AIDS to her child 70% 71% 4. 70% of non-maternal caregivers can recognize at least 2 known ways in which a mother can transmit HIV/AIDS to her child 40% 59% 5. 60% of mothers can name two or more symptoms of an STI 40% 33% 6. 50% of caretakers can name two or more symptoms of an STI 30% 22% 7. 35% of mothers will report the use of a condom the last time she had sexual intercourse 14% 22% 8. 80% of high school students in which SHCs are active have adequate knowledge of the care of family members with AIDS Baseline data not collected by DramAidE 84%i 9. 80% of households with persons with chronic severe illness are receiving regular visits (according to protocol that will define number of visits) from HBCVs Data to measure this indicator not available due to time lag in implementing HBC monitoring system 10. 100% of HBCVs demonstrate improved counseling skills Data to measure this indicator not available due to time lag in implementing HBC monitoring system 11. 70% of mothers report that they would allow their child to play with a child who has the AIDS virus 56% 63% 12. 65% of non-maternal caregivers report that they would allow their child to play with a child who has the AIDS virus 47% 59% 13. 50% of families caring for orphans and vulnerable children in communities with active CHCs have correctly applied for government benefits Data to measure this indicator not available due to time lag in implementing HBC monitoring system Note: The HIV/AIDS program component began in 2000 so “Latest Data Available” results are based on a one year implementation period. The source for Baseline data for the HIV/AIDS objectives is the 2000 KPC unless otherwise noted. The source for “Latest Data Available” objectives is the Cost Extension 2001 Baseline KPC. i Source: 2001 DramAidE Evaluation. Ndwedwe District Child Survival Project Final Evaluation 2001 11 2. Technical Approach a. Overview The Ndwedwe District Child Survival Project (NDCSP) was initiated as a planning grant under USAID/BHR/PVC funding in 1995. This grant supported a two-year planning period and arising from this, a second grant was awarded to support full-scale Child Survival Project activities between 1997-2001. The goals of the project are to reduce morbidity and mortality among children under 60 months of age in the target area and to improve the health status of women of childbearing age by reducing the risks associated with reproduction in the same population. The intervention areas emphasized for the original project, and their respective level of effort allocations, were: Primary Immunizations (EPI) (25% LOE), Control of Diarrheal Diseases (CDD) (20% LOE), Pneumonia Case Management (PCM) (25% LOE), and Maternal and Newborn Care (30% LOE). Subsequently, additional funding was sought and secured to expand the project’s limited HIV control activities in response to the rapidly escalating HIV/AIDS problem in the district. In 2000, a multi-faceted HIV/AIDS intervention was funded that focused on three types of initiatives: 1. preventing new cases through community and school-based prevention programs; 2. strengthening the capacity of families and communities to meet the needs of orphans and children affected by AIDS; and 3. support and supervision of home-based care for people living with AIDS (PLWA). The NDCSP was recently awarded a four-year cost-extension to continue with the existing child survival (CS) and Women of Reproductive Age (WRA) interventions with important new elements and more intense HIV/AIDS/STIs interventions in the extended project area generated by newly established district boundaries for Ndwedwe. The project is largely located in a rural area of the Ndwedwe District, KwaZulu Natal Province, South Africa. The District includes some of the poorest deep rural communities found in South Africa. The principal partner during the project period was the Department of Health at the provincial and district level, particularly the District Health System Management Team (DHSMT). Productive partnerships were forged with PATH as well as with the following local non-governmental organizations (NGOs): Oakford Clinic and Priory, The Valley Trust (TVT), National Progressive Primary Health Care Network (NPPHCN), DramAidE, Training and Resources in Early Education (TREE), Diakonia Council of Churches and Sinisizo. The project’s results-based objectives by intervention are: Immunization 1. Increase to 71% the % of children immunized against measles 2. Increase to 50% the % of children 12-23 months who are fully immunized per vaccination cards 3. Reduce to 25% the % of children 12-23 months who defaulted between the DPT1 and DPT3 Increase by 20% the % of women who receive at least two TT vaccinations Ndwedwe District Child Survival Project Final Evaluation 2001 12 Control of Diarrheal Disease 1. Increase to 66% the % of mothers who give the same or more than usual amount of liquids during their child’s diarrheal episode 2. Increase to 65% the % of mothers who give the same or more food than usual amount of food during their child’s diarrheal episode 3. Decrease to 5% the % of mothers of children 0-23 months who say they don’t know what to do following a diarrheal episode Pneumonia Case Management 1. Increase to 55% the % of mothers who are able to recognize danger signs of pneumonia 2. Reduce to 35% the % of mothers who reduce or stop feeding for a child with rapid or difficult breathing 3. Increase to 95% the % of mothers who seek care at health facilities for children with rapid or difficult breathing Maternal and Newborn Care 1. Increase to 50% the % of mothers who know the danger signs of delivery that require giving birth at a health facility 2. Decrease to 50% the % of women who give birth outside health facilities 3. Among women seeking prenatal care, increase by 15% those who make at least two prenatal visits during the pregnancy 4. Increase to 70% the % of mothers who initiate breastfeeding within an hour after giving birth 5. Increase to 85% the percent of mothers who breastfeed within the first eight hours after birth 6. Increase to 45% the % of mothers of children under 4 months who are exclusively breastfeeding 7. Increase to 42% the % of mothers who are not pregnant, do not want another child in the next two years or are not sure, and are using modern methods of contraception HIV/AIDS/STIs 1. 80% of high school students in which School Health Clubs (SHCs) are active have adequate knowledge of HIV/AIDS prevention as demonstrated in their ability to name at least two strategies of prevention 2. 50% of high school students with active SHCs report adoption of one of three strategies of HIV/AIDS prevention (abstinence, being faithful, condom use) 3. 80% of mothers can recognize at least two known ways in which a mother can transmit HIV/AIDS to her child 4. 70% of non-maternal caregivers can recognize at least 2 known ways in which a mother can transmit HIV/AIDS to her child 5. 60% of mothers can name two or more symptoms of an STI 6. 50% of caretakers can name two or more symptoms of an STI 7. 35% of mothers will report the use of a condom the last time she had sexual intercourse 8. 80% of high school students in which SHCs are active have adequate knowledge of the care of family members with AIDS 9. 80% of households with persons with chronic severe illness are receiving regular visits (according to protocol that will define number of visits) from HBCVs Ndwedwe District Child Survival Project Final Evaluation 2001 13 10. 100% of HBCVs demonstrate improved counseling skills 11. 70% of mothers report that they would allow their child to play with a child who has the AIDS virus 12. 65% of non-maternal caregivers report that they would allow their child to play with a child who has the AIDS virus 13. 50% of families caring for orphans and vulnerable children in communities with active CHCs have correctly applied for government benefits. The objectives of the NDCSP are being addressed through a package of maternal and child health care interventions operating at the facility and community level, within the context of an integrated, coherent and effective District Health System. The NDCSP has ensured that its training of clinic workers is adapted to IMCI approaches, which have recently been adopted by the Department of Health. Besides the integrated approach of IMCI, the project has worked to foster a DHS that emphasizes the importance of working to improve the health of the catchment population, rather than remaining facility-based, and the importance of enrolling all possible actors to play complementary roles in improving the community’s health. The project therefore focuses on improving quality of, access to, and demand for health services by addressing the Ndwedwe health system at four levels: · At the caregiver level: maternal education through contact with project-trained TBAs, clinic staff with improved communication and education skills, and through creche child-to-child health education · At the community level: training of TBAs and daycare workers to improve their knowledge of preventive health care and danger signs and ability to serve as community educators, support to community health committees which mobilize communities for health activities, implementation of prevention and education programs in local schools in collaboration with an indigenous NGO; and development of a home-based care program for the district · At the health care facility level: training of nurses to provide quality and accessible care through training in IMCI, PEP, STI diagnosis and treatment, community outreach, and better management of prenatal and postnatal care · At the district health system level: support for creation of a functioning DHS structure by ensuring that systems created provide an adequate drug and vaccine supply, training and supervision of clinic workers, supervision by clinics of community workers and community health committees, and coordination of current fragmented government efforts. b. Interventions The project’s immunization strategy focuses on reducing problems identified through the KPC, such as caregivers’ lack of familiarity with recommended schedules of immunization, and on problems identified at the clinic level such as missed opportunities and the persistence of specific "immunization days" despite a policy change favoring continuous availability. Activities included training of clinic-based nurses and development and dissemination of health messages through TBAs, local crèches and clinics, and schools. The project’s control of diarrheal disease intervention includes development and implementation of IMCI training and refresher modules for PHC nurses on management of watery diarrhea, persistent diarrhea and dysentery, and Ndwedwe District Child Survival Project Final Evaluation 2001 14 communications programs for communities emphasizing ORS, hygiene and prevention, and supportive care and feeding during and after illness. The project’s pneumonia case management intervention strategy includes training PHC nurses responsible for pneumonia cases in IMCI/standard case management procedures and health communications activities which utilize multiple channels to disseminate messages to caregivers emphasizing early detection of pneumonia symptoms and prompt care-seeking. The project’s maternal and newborn care activities have included a TBA training program, training of PHC nurses in perinatal care, and upgrading of existing nurse midwives through the self-guided Perinatal Education Programme (PEP) training, including a project-supported clinical practicum. Through health communication activities, the project has worked to familiarize area women with the danger signs of pregnancy and encourage them to make advance arrangements for transport in case of nighttime delivery and/or complicated delivery. In addition, because of the extremely high rates of STIs in pregnant women (an estimated 30% of prenatal clinic attendees are seropositive for syphilis), and the skyrocketing rates of HIV in the Durban area, the NDCSP introduced rapid syphilis and gonorrhea dipstick tests for use at rural clinics, as well as raising community awareness of the means of transmission and prevention of HIV/AIDS/STIs. Rapid testing for HIV/AIDS is currently being carried out at the district hospitals. Evaluation findings for each of these intervention areas are outlined below. i. Immunization Immunization-Related Indicators Indicator Baseline KPC Final KPC 1. Increase to 71% the % of children immunized against measles 51% 53% 2. Increase to 50% the % of children 12-23 months who are fully immunized per vaccination cards 26% 47% 3. Reduce to 25% the % of children 12-23 months who defaulted between the DPT1 and DPT3 doses 48% 8% 4. Increase by 20% the % of women who receive at least two TT vaccinations 10% 92% From the table above, it would appear that most of the project’s immunization targets have not been met, especially those reflecting childhood immunization. However, evidence from sources other than the KPC suggest that substantial success has been achieved in improving the immunization status of infants and young children in the District during the lifetime of the project. Firstly, measles cases (measured in the KPC survey by reviewing Road to Health Cards which did not include immunizations obtained during mass campaigns) have in fact declined dramatically as reported in interviews with health workers at all health facilities visited during the evaluation. This reflects the average coverage rate of 98% for measles vaccine achieved in the Province during the Expanded Programme on Immunisation (EPI) Mass Immunization Campaign of 2000. The Ilembe District, which now incorporates the Ndwedwe sub-District, had the lowest coverage in the Province, achieving only 89%. High coverage rates were also achieved in the previous mass campaign, according to the EPI Program Manager. High average rates of coverage were also achieved for polio vaccinations during the 2000 campaign (97% in first round and 70% in second round) with the Ilembe District again the lowest at 86% and 60% respectively. Ndwedwe District Child Survival Project Final Evaluation 2001 15 These vaccinations were not recorded on RTH cards and thus are not captured in the KPC analysis. The absence of a RTH card was equated with non-immunized status for the purposes of the KPC study of 2001, on which the summary data in the Table is based. This methodology would have had an added negative bias on the determination of completed immunization status. Taking all factors into account it is the considered opinion of the evaluator that the project target of 71% for measles immunization was in all probability met. It is likely to take some time before the higher levels of measles immunization are reflected in the Provincial or Regional HIS data sets to provide evidence for this conclusion. It can also be reasonably assumed that the project target of 50% completed immunization at 23 months was also met since the commonest default is with measles immunization and the rates of default between DPT1 and DPT3 were reduced to 8%, which was well below the project target of 25%. Additionally, completed immunization at 12 months stands at 73% for KZN Province currently, but disaggregated figures for the Districts are not available (personal communication, EPI Program Manager). The project target of increasing by 20% the number of pregnant women who receive at least two doses of Tetanus Toxoid during antenatal care has most likely been met as well. Although the baseline estimate of TT coverage was not deemed to be reliable, the final KPC revealed that 86% of women surveyed fulfilled this criterion. However the substantial gains in immunization outlined above allied to the documented increase in attendances for ANC provides supporting evidence of target attainment. In addition, the EPI Program Manager revealed that as part of EPI Disease Surveillance a survey of hospital records confirmed that no cases of neonatal tetanus had been recorded in recent years. The evaluation suggests therefore that all NDCSP targets for immunization were fully met and this finding reflects the appropriateness of project initiatives in promoting demand for and enhancing the delivery of services for immunization throughout the District. The evidence of a relatively lower uptake of immunizations in the District of Ilembe during the mass campaign of 2000 is disappointing, but it should be noted that the District includes some geographic areas that have a poorer health infrastructure than Ndwedwe Sub-District (the project target area) and also includes a fairly large metropolitan population. These factors could conceivably have counter￾balanced the achievements of the project, which invested a high level of effort in promoting demand for immunization services through the TBA initiative and on the supply side through the IMCI training and supervision of health workers. The role of the TBAs in mobilizing households and communities is highly regarded and the EPI Program Manager agreed that the high TT immunization levels and the elimination of neonatal tetanus could be largely attributed to the activities of the project trained TBAs. It is intended that these indigenous community volunteers will be targeted for inclusion in the community training initiative of the extended project. Every effort should be made to recruit to this training active TBAs in the project areas where planned TBA training was cancelled following the MTE as well as in the communities of the two additional clinics being incorporated in the newly extended Ndwedwe Sub-District. EPI management plan to conduct a review during 2002 of the capacity of Ndwedwe District Child Survival Project Final Evaluation 2001 16 CHWs and indigenous volunteers to interpret RTH cards as a measure of their ability to promote immunization. EPI is a prime component of IMCI and the increased exposure of clinic nurses to this training would be expected to have a dramatic effect on immunization status when fully implemented. The implementation to date in the Province has only been on a pilot basis and success has been limited, as the Health Facility Survey of May 2001 reveals. Although the index of availability of vaccines, supplies and equipment to support vaccination was very high throughout the Province’s IMCI pilot clinics, health worker performance as reflected in the Index of Integrated Assessment was lowest in KZN among the four provinces in the Republic of South Africa (RSA) which were evaluated. This latter assessment confirmed that missed opportunities for catch up immunization are common and this problem will need to be addressed when IMCI training and supervision resumes. Coverage levels in excess of 90% are eminently attainable if the twin objectives 1) of mobilizing demand in the community through CHWs, the CHCs and trained indigenous volunteers like TBAs and 2) of improving service provision through training and supervision of health workers in IMCI can be achieved. Valuable lessons that have been learned to date are a) that the success, which is already evident in the project's community initiatives, must be consolidated and b) that much can and must be done to strengthen the performance of health workers and reduce missed opportunities. Given that the immunization infrastructure in the Province is very adequate and amongst the best in the RSA and given that the core activities of the Phase II work-plan will address the lessons learned, the NDCSP can make a major contribution to attaining universal immunization coverage in the District within the life-time of the project. ii. Control of Diarrheal Disease Diarrheal Disease-Related Indicators Indicator Baseline KPC Final KPC 1. Increase to 66% the % of mothers who give the same or more than usual amount of liquids during their child’s diarrheal episode 46% 71% 2. Increase to 65% the % of mothers who give the same or more food than usual amount of food during their child’s diarrheal episode 45% 57% 3. Decrease to 5% the % of mothers of children 0-23 months who say they don’t know what to do following a diarrheal episode 20% 7% As can be seen from the table above, the project targets for control of diarrheal disease were partially attained. Over 70% of mothers practice oral re-hydration during episodes of diarrhea, up from 46% at the start of the project, while 57% of mothers continue feeding or give additional food during diarrheal episodes. The latter figure is a big improvement although still some way short of the project target of 65%. The percentage of mothers who don’t know what to do during or following a diarrheal illness has been reduced from 20% to 7% just short of the project target of five percent. Although much progress has been made in this intervention area, diarrhea remains the predominant illness affecting communities, with the problem of watery and mucoid/bloody diarrhea compounded by cholera, which has now become endemic in the project area and other Ndwedwe District Child Survival Project Final Evaluation 2001 17 locations in the province. On a site visit during the evaluation, one clinic had processed 15 cholera cases that day and special facilities had been established at all clinics to cope with this particular problem. The escalating HIV/AIDS epidemic, with recurring or protracted diarrhea being a prominent manifestation of infection, is compounding the problem. The possibility of implementing activities to promote the prophylactic use of cotrimoxazole for AIDS affected women and children (to prevent opportunistic infections) should be explored during Phase II with the Department of Health. NDCSP initiatives aimed at control of diarrheal disease has focused on creating awareness of preventive strategies, on establishing appropriate home management practices and on improving health seeking behavior at community level and on improved health worker performance at clinic level. Activities at community level were conducted through the TBAs (and more recently through the HBCVs) by the inclusion of CDD messages and skills in their supervisory contacts with project personnel and with health workers at facility level. The CDD experience and skills of these volunteers are enhanced by their attendance as assistants in the ‘cholera tents’ currently attached to most clinics. The IMCI training was the principal vehicle for improving health worker performance and the current focus on cholera, with all clinics regularly practicing and being updated in case management, has aided and abetted this process. Although not all CDD project targets were met, the evaluation found no evidence of default in the project initiatives to date. It was also evident during the evaluation that the DOH is responding aggressively to the threat presented by the prevalence of cholera in the province, which appears to have reached endemic proportions. MCDI’s Health Facility Assessments conducted during 2000 reveal that the enabling infrastructure for proper case management of diarrheal illness is adequate at all clinics in the District. All clinics have appropriately resourced ORT corners and all have separate designated facilities for case management of cholera. However both case management by health workers and the counseling of caregivers received low ratings during the IMCI Evaluation. All clinics provide access to treated water and water treatment for communities as well as appropriate supervision and instruction to communities. The inclusion of indigenous community volunteers like TBAs in this process is a very positive indication of the health services – community interaction being fostered in the effort to contain and control the cholera epidemic. CDD messages and skills are also re-enforced at every contact between project personnel and TBAs, HBCVs and other community contacts. The main lesson learned from efforts at CDD to date is that much remains to be done in this critical area of disease control. The foundations for an integrated three-pronged approach to CDD have been laid and the project workplan for Phase II includes activities which are central to this effort. The success achieved to date by the DHSMT, with crucial NDCSP support, in the formation of CHCs, engagements with Tribal Leaders (Amakosi) and church groups, both of whom exert great influence in Zulu culture, and the training and supervision of indigenous volunteers has created the required infrastructure for the community training activities that will be a feature of the extended Project. In addition, the community health information system database which the project will help to establish will be key tool that will enable the DHS to monitor the epidemiology and impact of diarrheal disease in the community and to mobilize and target resources in response to defined needs. Finally, the planned rollout of the IMCI initiative to include all health workers and CHWs will be the critical element that will underpin the entire Ndwedwe District Child Survival Project Final Evaluation 2001 18 effort. From the evidence outlined above this latter activity is the one which requires most attention if the gains made in CDD are to be consolidated. iii. Pneumonia Case Management Pneumonia Case Management-Related Indicators Indicator Baseline KPC Final KPC 1. Increase to 55% the % of mothers who are able to recognize danger signs of pneumonia 35% Not available 2. Reduce to 35% the % of mothers who reduce or stop feeding for a child with rapid or difficult breathing 55% 75% 3. Increase to 95% the % of mothers who seek care at health facilities for children with rapid or difficult breathing 86% 86% Pneumonia Case Management (PCM) is the only intervention area where all targets were not met. Reliable data are not available to evaluate progress with the first target of increasing the percentage of mothers able to recognize danger signs of pneumonia. The findings of the KPC 2001 suggests that the percentage of mothers who stop or reduce feeding for a child with rapid or difficult breathing now stands at 75% whereas the baseline indicator for this target was 55%, and the target was a reduction to thirty five percent. The percentage of mothers who seek care at a health facility for children with rapid or difficult breathing remained high at 86% but this was also the baseline figure, indicating that no progress was made. The overall results with PCM are very disappointing when compared with the success achieved in the other intervention areas which used identical strategies and methodologies. There is no ready explanation for the failure to achieve set targets in this intervention area; the project used the same TBAs as community educators and the same clinic nurses provided treatment and care￾giver instruction. The PCM intervention area is relatively new when compared to EPI and CDD. Core activities in these latter intervention areas have had prominence in both community education initiatives and health care practices for nearly two decades. PCM has had a more recent focus of attention within the health care process, having been introduced through ARI programs in many countries in the early nineties and through the IMCI initiative launched in KZN Province in 1999, and then, only on a pilot basis. Communities have an awareness of pneumonia as a disease entity (it is called Amahlaba – sharp pains in the chest, in Zulu culture) and a variety of traditional remedies have been relied on over time for the relief of these symptoms. The common usage of these remedies has been compounded by traditional beliefs and practices that led to restriction of both fluid and food in a child with rapid or difficult breathing as well as to delays in seeking help at a health facility. TBAs would be very familiar with these traditional practices and would in all probability have condoned, if not promoted them. The cadre of project trained TBAs have been exposed to PCM messages during quarterly supervisory sessions with the project’s Training Coordinator in parallel with the project’s role in supervising IMCI trainees. It is likely that change in attitude Ndwedwe District Child Survival Project Final Evaluation 2001 19 and knowledge among these TBAs has not had sufficient time to filter through to communities in a manner that would be reflected in increased awareness and changed behavior among mothers. Additionally, no attempt has been made to date in the District to engage with other influential elements in the community in support of PCM, as has happened so successfully over time with EPI and CDD. This omission is acknowledged by DOH Program personnel as one of the main reasons for the relative lack of success with IMCI introduction in the province to date. The TBAs would, in these circumstances, have been isolated as promoters of change at community level in this intervention area. The problems arising from failure to engage adequately with communities in support of PCM have been compounded by persisting weaknesses in health worker performance with PCM. This weakness is understandable since the systematic approach to case management, which is inherent in IMCI, had only been introduced to a small proportion of clinic nurses in the District at the time of this evaluation. Site visits during the evaluation also revealed that few clinic nurses in service at this time have had formal PHC training. Weaknesses in health worker performance with the various elements of IMCI were highlighted in the Health Facility Survey Report of May 2001. In this assessment, KZN nurses who had been trained in IMCI performed less well than their counterparts elsewhere in the RSA, missing 20% of cases presenting to the clinic with pneumonia while their counterparts in KZN who had no formal exposure to IMCI missed 40% of cases presenting with pneumonia. Correct treatment for pneumonia which was correctly diagnosed was given in only 44% of cases and clinic nurses in KZN did not compare favorably with their counterparts in other Provinces when assessed for the quality of counseling given to caregivers during the health contact. Valuable lessons learned from this intervention area include 1) the importance of checking that message carriers, in this instance the TBAs, have adequately absorbed the messages that are expected to effect change in entrenched traditional beliefs and practices in the community, especially when the message carriers have until recently operated entirely within the traditional ethic and 2) that substantive changes in health worker practice are more difficult to introduce when only one health worker is trained in an institution since the new trainees may feel ‘isolated’ in their new practice even with regular supervision and support. It is the opinion of the evaluator that there is little more that the project could have done to attain the targets set in the PCM intervention area. The results of the IMCI evaluation have only recently become available and the weaknesses discovered in the implementation process have not yet been fully addressed. In discussion with the Program Manager, it is obvious that there is widespread disappointment with the results to date and that an accelerated implementation plan is being developed which will implement the twin components of health worker training and the IMCI Household and Community Module in parallel. The latter module has been piloted in the Bergville District by the World Vision CS Project and the process was being evaluated at the time of the NDCSP Final Evaluation. It is intended that the NDCSP will play a major role in the current plans of the DOH to introduce the IMCI Household and Community Module to communities in the Sub-District and to extend IMCI training and supervision much more widely among health workers. As mentioned previously, the NDCSP is well positioned to undertake this role in the extended project. Ndwedwe District Child Survival Project Final Evaluation 2001 20 It is also very important that the KPC findings that demonstrate persistence in undesirable beliefs and practices with regard to PCM in the community be brought to the attention of the CHCs and community groups. Comparisons with achievements in other intervention areas should be highlighted so that underlying reasons for failure to reach targets in PCM may be explored and appropriately addressed. If the strategies outlined can be successfully implemented, there is every expectation that achievements in this intervention area during the extended life-time of the project can be comparable to those in the other intervention areas described in this section of the report. iv. Maternal and Newborn Care Maternal and Newborn Care-Related Indicators Indicator Baseline KPC Final KPC 1. Increase to 50% the % of mothers who know the danger signs of delivery that require giving birth at a health facility 10% 78% 2. Decrease to 50% the % of women who give birth outside health facilities 65% 15% 3. Among women seeking prenatal care, increase by 15% those who make at least two prenatal visits during the pregnancy 9% 75% 4. Increase to 70% the % of mothers who initiate breastfeeding within an hour after giving birth 51% 48% 5. Increase to 85% the percent of mothers who breastfeed within the first eight hours after birth 77% 96% 6. Increase to 45% the % of mothers of children under 4 months who are exclusively breastfeeding 33% 32% 7. Increase to 42% the % of mothers who are not pregnant, do not want another child in the next two years or are not sure, and are using modern methods of contraception 32% 67% As can be seen from the table above, success in this intervention area has been impressive to say the least, with all project targets except one exceeded, most by a large margin. The number of women who recognize danger signs at delivery that require giving birth at a health facility has risen from 10% to 78%, well above the project target. This new knowledge and awareness is matched by the comparable decline in the numbers of women who deliver at home – down from 65% to 15%, with the target of 50% well exceeded. The dramatic decline in home births is paralleled by a decline in clinic births in the District, which has fallen from 828 to 184 between 1996 and 2000. A review of birth statistics at Osindisweni and Montebello District Hospitals during the evaluation confirmed a recent increase in births at both hospitals, the figure having doubled at the former during the last 2 years. There is also anecdotal evidence of women electively seeking to give birth in other hospitals, either in those situated close to the District or in Durban. This suggests that women in seeking health facility births are bypassing clinics (some apparent reasons for this practice will be discussed later). The number of pregnant women who make at least two ANC visits during a pregnancy has increased from 9% to 75% during the lifetime of the project – a dramatic success, well beyond the set target. As mentioned in the Immunization Section above, it is acknowledged that the activities of the trained TBAs have been a major factor in the elimination of neonatal tetanus. It is reasonable to expect that the TBA training module on “the Handful of Danger Signs for Ndwedwe District Child Survival Project Final Evaluation 2001 21 Newborns” would have contributed further net gains in child survival, but no supporting evidence is available for this contention. The target for initiation of breastfeeding within eight hours of birth shows heartening improvement with the target surpassed by a large margin at 95%. However, the targets for initiation of breastfeeding within one hour of birth and for exclusive breastfeeding were not attained. Further gains within this target area may well depend on initiatives at institutional level to reinforce maternal practices, given the rise in institutional births. Substantial gains have been achieved in modern contraceptive usage, with 67% of women using these methods, which is again well above the project target of forty two percent. The contribution of the NDCSP to the gains in Maternal and Newborn Care is very large. There were three key project initiatives in this intervention area - the training of TBAs in obstetric and newborn care, the supervision provided to nurse-midwives undertaking PEP and the Family Planning training provided to clinic nurses. TBA training was a core activity of the project from the outset and 39 active TBAs were trained from the catchments of 4 of the 5 clinics in the District. It was decided not to proceed with the planned TBA training in the fifth clinic catchment area, following the MTE and to re-program the training resources to the CHC development process then gaining momentum. It was felt that the likely gains in this latter area would be more sustainable in the long-term. The TBAs at this stage were also supervising fewer home births in compliance with the Provincial policy of encouraging health facility births and also because of their own awareness of the risks of exposure consequent on the increasing prevalence of HIV among antenatal clinic attendees. The decision to suspend TBA training was a reasoned one at the time. However it would appear that the true value of the TBA in the modern era lies in their role as advocates at community level for better quality maternal and newborn care, rather than their traditional role of home birth supervision. It is the opinion of the evaluator that the TBAs training and supervision by the NDCSP Training Coordinator contributed greatly to the gains outlined above and that this component of the project was one of the most productive in terms of tangible and sustainable benefit. They became such good advocates of health facility births that some clinic principals commented during site visits that the TBAs were probably responsible for the decline in clinic births, the perception being that they were encouraging pregnant women to bypass the clinic in favor of hospital as the desired place of birth. In focus group discussion, TBAs dispute this viewpoint and laid the blame on prevailing attitudes among some clinic staff that discourage mothers from clinic deliveries. Problems with transport for syphilis serology tests were also cited as a factor in the swing towards hospital births. Women are frequently encouraged to attend hospital for the first ANC visit to ensure that syphilis testing is performed since clinics cannot always ensure test availability. It was felt that, as a result, women tended to continue attending the hospital and deliver there. The latter explanations are probably the more correct. Although clinics are very well staffed by Regional standards, the required skills mix for quality perinatal care is still not widely available in clinics and discussion with clinic staff during site visits revealed that problems with the referral of women presenting problems in labor are common and adversely affect the attitudes of staff towards deliveries at the clinic. The latter problems result from the lack of easy access to emergency transport for referral during labor and from the stated Ndwedwe District Child Survival Project Final Evaluation 2001 22 reluctance on the part of some hospitals to readily accept such referrals. These are systemic issues outside the remit of the project, which require attention. Although there is no argument in favor of resuming training of TBAs in its original format, there is every reason to seek out active TBAs in the new areas of the sub-District and in those project areas where the original training was abandoned for inclusion in the planned community training which will be a major component of the extended project. The role of these indigenous community volunteers is already well established and they command respect in the community. There is little doubt that this group can make a major contribution in project-led efforts to promote improvements in antenatal care for pregnant women and in perinatal care for mothers and newborns. The trained TBAs still supervise home births in emergency situations and there is anecdotal evidence that young pregnant schoolgirls are reluctant to attend clinics and, as a result, seek obstetric care from the TBAs instead. Their demonstrated willingness and capacity for broader training (c.f. Newborn Care, IMCI and HIV/AIDS) makes them a vary valuable resource in the extended NDCSP in pursuit of its core CS and WRA objectives. The NDCSP Training Coordinator has supervised the PEP self-directed midwifery training in the District. While this task has been performed diligently, entry to the program is voluntary and up to now there has been little incentive for nurses to pursue the training because practical support within the service (e.g. protected study time) has been lacking and it would appear that there has been little encouragement from some service managers for nurses pursuing the course. During the evaluation it was stated by the Provincial Maternal Health Program Manager that the PEP will be afforded much greater status in future with the likelihood that PEP will be considered a pre-entry requirement for Advanced Midwifery Training, a formal career path qualification. Wider uptake of PEP will without doubt enhance the quality of perinatal care considerably and help to consolidate the gains made to date in Maternal and Newborn Care. The NDCSP will have a major role to play in this undertaking. Family Planning training for nurses was motivated, planned and conducted in all clinics and hospitals in the District by the project Training Coordinator and the massive gains in modern contraceptive usage is testament to the success of this initiative. Testing for syphilis is routine but not for gonorrhea in antenatal clinics throughout the Province, although no reliable data on coverage and outcomes is available. Staff at most of the clinics visited during the evaluation stated their contentment with the testing procedures in place, which are standard throughout, and were confident that the presence of syphilis in antenatal attendees is, in general, accurately determined and adequately treated despite the difficulties outlined below. The NDCSP introduced rapid testing for syphilis at Oakford Clinic, where staff found the test process to be a very efficient tool in identifying and treating syphilis in women attending the antenatal clinic at a single visit. It was stated in interview with the nurse in charge that the on-site test helped to raise awareness among clinic staff about syphilis and that quality assurance cross￾testing at Osindisweni Hospital helped to emphasize that the prevalence of syphilis among antenatal attendees at the clinic was higher than anticipated, as well as confirming the reliability of the test process. None of the Government clinics in the District were willing to introduce the test, preferring to continue with the existing process of sending the blood samples to their ‘mother’ hospital for analysis. Although this process requires a follow-up visit a week later and Ndwedwe District Child Survival Project Final Evaluation 2001 23 is frustrated by transport problems which leads to wastage and repeat testing the commonest reason advanced for not introducing the rapid test process was “concern about the reliability of the test.” There are likely to be other factors hindering the acceptance of an alternative efficient and reliable test process that require exploration and remedial intervention. MCDI staff are encouraged to carry out their planned activities to address this issue during Phase II. The Maternal Health Program Manager rates the contribution of the NDCSP to the Program very highly and anticipates a very important role for the extended project in supporting the Program in a number of key areas. The main lesson learned is that the TBAs are a very valuable resource in the field of Maternal and Newborn Care. They have demonstrated their worth in generating beneficial changes in attitudes and behaviors with regard to antenatal care, place of birth and newborn care as outlined above. Their role as the link between the community and health services in this important area will be developed further during Phase II with particular emphasis on preparing communities for acceptance of a number of important new initiatives. These include the introduction of the “kangaroo” method of newborn care and extending the process of enquiry into maternal deaths to community level. It is planned to build on the successes derived from the TBA training of the project to date but with less emphasis on supervising deliveries and more emphasis on their role as community educators, especially the promotive/preventive aspects of maternal and newborn care. It is also planned to recruit more participants to PEP and to include a Neonatal Resuscitation Module with the Difficult Birth Module and the Clinical Practicum, which were added with project support to the standard PEP. It is hoped that site￾centered Advanced Midwifery Training might be introduced in the Ndwedwe Sub-District as a logical extension of PEP training. The Maternal Health Program is keen to extend the Rapid Syphilis Test to more clinics but recognizes that there may be opposition to this move as mentioned earlier and further analysis of the whole process of syphilis testing is required. The problems surrounding referral of patients with complicated labor, mentioned earlier, will need to be addressed in the context of the extended PEP training. In addition, the Baby Friendly Hospital Initiative will need support from the project to consolidate and enhance the gains made with breastfeeding, given the high percentage of mothers who now deliver in hospital. Recent developments in the Province suggest that anti￾retroviral drugs to prevent Mother to Child transmission of HIV will be more widely available in the near future but the role of the project in this venture, if any, remains to be determined. The Provincial Maternal Health Program has high expectations of NDCSP support in these ventures and anticipates further strengthening of the highly productive partnership already established. Ndwedwe District Child Survival Project Final Evaluation 2001 24 v. HIV/AIDS HIV/AIDS-Related Indicators Indicator Baseline (2000) Latest Data Available (2001) 1. 80% of high school students in which School Health Clubs (SHCs) are active have adequate knowledge of HIV/AIDS prevention as demonstrated in their ability to name at least two strategies of prevention Baseline data not collected by DramAidE 74% 2. 50% of high school students with active SHCs report adoption of one of three strategies of HIV/AIDS prevention (abstinence, being faithful, condom use) Baseline data not collected by DramAidE 90% 3. 80% of mothers can recognize at least two known ways in which a mother can transmit HIV/AIDS to her child 70% 71% 4. 70% of non-maternal caregivers can recognize at least 2 known ways in which a mother can transmit HIV/AIDS to her child 40% 59% 5. 60% of mothers can name two or more symptoms of an STI 40% 33% 6. 50% of caretakers can name two or more symptoms of an STI 30% 22% 7. 35% of mothers will report the use of a condom the last time she had sexual intercourse 14% 22% 8. 80% of high school students in which SHCs are active have adequate knowledge of the care of family members with AIDS Baseline data not collected by DramAidE 84% 9. 80% of households with persons with chronic severe illness are receiving regular visits (according to protocol that will define number of visits) from HBCVs Data to measure this indicator not available due to time lag in implementing HBC monitoring system 10. 100% of HBCVs demonstrate improved counseling skills Data to measure this indicator not available due to time lag in implementing HBC monitoring system 11. 70% of mothers report that they would allow their child to play with a child who has the AIDS virus 56% 63% 12. 65% of non-maternal caregivers report that they would allow their child to play with a child who has the AIDS virus 47% 59% 13. 50% of families caring for orphans and vulnerable children in communities with active CHCs have correctly applied for government benefits Data to measure this indicator not available due to time lag in implementing HBC monitoring system HIV/AIDS activities which were specified in the original Project DIP were largely incorporated in the Maternal and Newborn Care intervention area. However it soon became apparent to the Project Team that the impact of the escalating HIV epidemic necessitated an expansion of these activities and, as a result, a HIV/AIDS sub-component was added to the Project in 2000. The available indicators for this intervention area are shown in the table above. These suggest that significant progress has already been made in improving knowledge and establishing healthy behaviors with regard to HIV/AIDS among high school students, which is a very welcome finding. However, it is evident that less progress has been made in changing established beliefs and practices among older persons at risk of infection. These are not surprising findings at this early stage of implementation of the HIV/AIDS Project component and some deep rooted constraints to progress exist which need to be addressed if targets in this intervention area are to be met. As described earlier, the evaluator was impressed with the manner in which activities of the HIV/AIDS sub-component were merged with those of the core Project. HIV/AIDS has Ndwedwe District Child Survival Project Final Evaluation 2001 25 become a very prominent, if not dominant, issue in all Project activities with strategic messages relating to the disease incorporated in every contact with individuals, households and community groupings. The KPC 2000 Survey revealed that HIV/AIDS awareness was high since 94% of mothers and 88% of caregivers had heard of the disease, but only 74% stated that they knew it could be prevented. While knowledge about the role of condoms in reducing the risk of transmission was high at 86%, knowledge about other key risk reduction strategies was very low, e.g. limiting sexual partners (8%), avoiding sexual contact with persons who have multiple partners (4%) and avoiding sexual contact with commercial sex workers (2%). Over 80% of mothers knew about the possibility of mother to child transmission (MTCT) but only 22% knew that MTCT could be prevented. Belief that MTCT was inevitable when a mother was HIV positive was found to be prevalent, as was the belief that an infected infant presented a danger to potential foster or adoptive families. Knowledge about other STIs was also very low with only 40% of mothers and 29% of caregivers having such knowledge. Only a small proportion of men and women had knowledge about specific signs and symptoms of STIs, and condom use during the last act of intercourse was very low, at 7%, despite the high levels of awareness of the benefits of condom use. More worrying was the low condom usage among female respondents, despite high levels of awareness among these women that their husbands/partners had other partners (35% knew of such activity and 43% suspected but did not know for sure). The KPC Survey found that 3.5% of children under 15 years were orphaned due to maternal death and 11.5% due to paternal death, and that a high proportion of parental deaths had occurred recently – 41% of maternal deaths in the previous 15 months and 72% of paternal deaths in the previous four years, highlighting the increasing mortality from HIV/AIDS. HIV infection is the most common cause of maternal mortality in South Africa at this time, being responsible for 29.7% of such deaths.1 The KPC Survey also revealed evidence of widespread stigmatization associated with HIV/AIDS. About one third of mothers and caregivers stated that they were unwilling to care for a relative suffering from AIDS, a finding at variance with the strong ethos of caring for ill relatives that exists in traditional Zulu culture. Roughly half of mothers and caregivers interviewed would not let their child play with an HIV infected child and a quarter stated that they would not let a teacher with the infection continue to teach their children. As part of its baseline assessment for the HIV/AIDS project, the NDCSP conducted focus group discussions with respondents representing various ages and the full socio-demographic spectrum from deep rural to semi-urban to assess levels of understanding of HIV/AIDS in the community, to explore attitudes towards PLWA and children affected by AIDS (CABA) and to identify traditional community structures and institutions that might support a community-based home care program. Local religious leaders were also interviewed to assess community attitudes towards affected persons and willingness to offer assistance to such people in the community. The findings of these discussions led to the identification of a number of key factors which helped to determine the pertinence of the objectives of the NDCSP in this intervention area. 1 Department of Health. (2000). Third Interim Report of Confidential Enquiries into Maternal Deaths. Ndwedwe District Child Survival Project Final Evaluation 2001 26 These were: 1. While HIV/AIDS awareness was high, misconceptions were common about both sexual and non-sexual risk of infection. 2. Awareness of the benefits of condoms was high and condoms were freely available but condom use was found to be extremely low. 3. Economic dependency severely undermined women’s ability to motivate for condom usage and sexual fidelity. 4. Exploitation of school girls was found to be a problem in the District. 5. Community resources – social, financial and informational, for care and support of PWLA were generally limited. It was found that access was poor to grants for orphan care available from the Department of Welfare. 6. However, a potential for mobilizing such resources in the community was identified among church congregations in particular and all the newly created CHCs identified HIV/AIDS as one of their priority health problems. The NDCSP focused on 3 core activities: a) those aimed at preventing new cases, b) those aimed at strengthening the willingness and capacity of families to care for OVCs and 3) those aimed at supporting and improving home-based care of PWLAs. In implementing these activities the Project has worked in a close partnership with the regional HIV/AIDS program and activities are coordinated through the DHSMT and the District AIDS Forum. The operational capacity of the DHSMT should be enhanced when current plans to allocate the annual Provincial budget for HIV/AIDS on a district basis are implemented. The achievements to date in implementing the activities outlined in the DIP are impressive and at the time of the evaluation the entire project team was largely engaged with these activities when not directly involved in the evaluation process. A number of activities being conducted by the Project which were highlighted by the regional HIV/AIDS Program Manager and the District Manager as being unique to the Ndwedwe District will be described. These include: · The engagement of the Project trained TBAs in support of the HIV/AIDS intervention strategies was considered opportune. This cadre of indigenous volunteers had their established status in the community enhanced through the Project training. All those who are currently active were given additional training to improve their ability to deliver HIV￾related messages to households and communities and a proportion of their number were trained as HBCVs. It was apparent from focus group discussions with TBAs during the evaluation that they have tremendous potential to contribute to the home-based care initiative in the District. As a group, they stated their eagerness to acquire the skills and to be provided with the commodities that will enable them to realize this potential. There is a widely held perception that the TBAs, with their traditional accessibility to homes, can help overcome the barriers to home-based care that derive from the high levels of stigmatization highlighted in the KPC. The Project has the opportunity and capacity to roll out this extended scope of operations to active TBAs in the new Project area as well as to those TBAs for whom primary training was not conducted during Phase I. · The level of supervision and support provided to the HBCVs in the District by the Project’s two highly experienced Supervisors is not matched elsewhere in the Province. Ndwedwe District Child Survival Project Final Evaluation 2001 27 The MCDI Home Office, at the instigation of the Project’s Training Coordinator, facilitated the engagement of both as members of the Project Team when it was realized that coordinated supervision would be required if the trained HBCVs who had been deployed in the District were to function in any meaningful way. Such supervision is not universal to date in the Home Based Care component of the HIV/AIDS Program throughout the Province. While the supervisory and support structures provided directly by Project are excellent, they are unique and extravagant in their present format and as such not sustainable. It is to the credit of both Supervisors that they themselves had already realized this extravagance when the issue of adjustments to their current scope of operations was discussed with them during the evaluation. A key objective of the NDCSP during Phase II will be to develop and implement supervisory and support systems and methods for the Home Based Care Program which are not dependant on Project support but which can reasonably be sustained through linkages with the systematic supervision and support structures of the DHS, especially those operating at community level. In addition, the HBCP will need to be extended to the newly incorporated areas of the Ndwedwe sub-District. It is also anticipated that the Project HBC Supervisors, with the Project Training Coordinator, will play a major role in developing and implementing in the District, the community training program which will be based around the IMCI Community Module, but will have a major HIV/AIDS component. The Supervisors will also play a lead role in developing strategies for care of OVCs and CABA at community level and exploring the potential role that the cadre of HBCVs can play in this process. GIS mapping of vulnerable homesteads and the collection of vital demographic and socio-economic data essential to the DHS in planning and delivering home based care could be included in the scope of operations of the HBCVs, as well as in those of the TBAs. Plans are being developed to pay these volunteers a stipend from the HIV/AIDS budget and it is intended that the Nutrition Program will support home based care by enabling the HBCVs to deliver nutritional support to vulnerable homesteads. Both of these developments will help to consolidate the operations of the HBCP, as the lack of financial reward and the inability of the volunteer care providers to contribute materially to impoverished households affected by HIV/AIDS were highlighted in focus group discussions with the HBCVs as key constraints to their overall effectiveness. · The Project’s efforts in engaging with the community have been extensive and highly inclusive. Contacts have been established throughout the District with all elements of the Traditional Authority which is still the dominant influence in Zulu culture although its dominance has been challenged and eroded in recent years, as part of the political changes that have occurred in the RSA. Contact with churches, both formal and informal, has been extensive because of the strong influence of the church in traditional Zulu culture. These wide-ranging contacts under-pinned the project’s successful efforts at facilitating the creation of CHCs, and thereby the creation of the community base of the DHS. Through participatory learning activities, the Project’s Community Organizer has helped the CHCs and other community structures - church groups in particular, to place HIV/AIDS at the core of their Health Action Plans. In a functional DHS the health agenda at community level will be largely driven by the CHCs and other local structures. The success of the Project to date in engaging with these agencies may well be pivotal in mobilizing and enabling community responses to the challenges presented by the escalating HIV/AIDS epidemic. Ndwedwe District Child Survival Project Final Evaluation 2001 28 · When it became apparent to the Project Team that the available Department of Social Welfare (DSW) grants for orphan care are largely inaccessible to families in need, a Social Worker was recruited by the Project as an additional Community Organizer during the past year. A training module has been developed which is being widely introduced in the community which outlines the procedures to be followed by claimants of these grants. This is being matched by efforts to develop linkages between the DSW and the Traditional Authorities that will help to remove major constraints to grant access that currently exist. The major obstacle to grant access, especially in deep rural communities, is the failure to formally register many births and deaths with the result that lineage cannot easily be established when a grant application is made. Most of these births and deaths will have been ‘registered’ with the Tribal Authority and efforts are being made to integrate the two systems. The TBAs, as well as the HBCVs, can play a pivotal role in this process by ensuring that the statutory registration required for grant access is completed in vulnerable households while parents are still living. They may also have an important role to play as mediators between the traditional and civil authorities in facilitating the registration of ‘unregistered’ orphans for grant purposes, which is a difficult and tedious process. Additionally, they have a key sentinel role to play in ensuring that this vital data enters the community data base which the Provincial HIS plans to develop in the District with NDCSP support. · The NDCSP and its partners face major obstacles to the achievement of its core objectives in this intervention area as outlined in the KPC findings above. However a range of initiatives already being implemented or planned by the NDCSP and its partners including: 1) the substantial groundwork which has been accomplished with communities to date; 2) the level of engagement with churches; 3) the initial success evident with the school based activities; 4) plans to expand the capacity of the HBCP; and 5) new initiatives in orphan care suggest that project core objectives are realistic and achievable. c. New Tools and Approaches IMCI Community Component GAPS Analysis: During the final year of Phase I, the NDCSP pilot tested a GAPS Analysis exercise in two communities of the Sub-District. The GAPS Analysis, based on a method under development by the KZN Provincial AIDS Task Team and World Vision, is a technique for identifying priority focal areas by assessing the degree to which prevailing community practices deviate from a set of Key Family Practices being promoted as part of the community component of IMCI. The pilot test established that the GAPS Analysis, in addition to spotlighting problem areas for BCS emphasis, was unexpectedly useful in identifying factors that commonly impede the adoption of improved practices. An NDCSP medical anthropologist consultant is now in the process of formalizing this function of the GAPS Analysis and expanding it into a comprehensive behavior change assessment tool for promoting improved Key Family Practices (including the HIV/AIDS-related practices designated by the Provincial AIDS Task Team as appropriate to the needs of KZN). The project’s Community Liaison staff will be trained in the use of the instrument so that they can lead the Community Health Workers in carrying out a GAPS Analysis exercise in all clinic catchment areas of the expanded project area. The expected outcome of the GAPS Analysis exercises will be the Ndwedwe District Child Survival Project Final Evaluation 2001 29 identification of: 1) Priority Practices for special emphasis during Phase II, and 2) factors currently limiting or supporting the households’ ability to adopt the Priority Practices. The NDCSP Wealth Ranking for CABA Needs Assessment: Community Health Committees throughout the project area are in the process of compiling and/or updating Orphans Registers that monitor the households caring for children affected by AIDS (CABA) in their communities. A purpose of these registers is to identify CABA households with the highest level of need for community and NDCSP assistance (including organization and training related to income￾generating activities). Since cash income alone is seldom an adequate indicator of material well￾being in rural Ndwedwe, the NDCSP is now adapting a PRA technique (the Wealth Ranking Exercise) to enable CHCs to select the neediest CABA households for special assistance. Wealth Ranking for this purpose was pilot tested in Shangase Tribal Authority in May 2001 with encouraging results. To broaden the use of this technique to the rest of the project area, the NDCSP is developing a formalized Wealth Ranking Instrument and guidelines for use. NDCSP Community Liaison staff will be trained as master trainers in use of the instrument. They in turn will train CHWs to carry out Wealth Ranking exercises in all communities with Orphans Registers, and they will lead the CHCs in choosing high-need households based on the results of the exercise. In addition, MCDI’s study of the Cost-Effectiveness of HIV/AIDS Rapid Tests have been described elsewhere in this report. A summary of the study is included as Attachment 4a. 3. Results: Cross-Cutting Approaches a. Community Mobilization and Capacity Building Achievements to date in the area of Community Mobilization and Capacity Building are impressive. From the outset the NDCSP has had a primary focus on the community and a substantial level of effort was generated through the project initiatives to engage with communities. This focus was driven by the fact that the NDCSP was sited in the Ndwedwe District by the Provincial DOH because the area historically had inferior health services and included some of the poorest communities in the region. It was anticipated that community empowerment for health would help to address these inequities which were inherited from the previous apartheid administration. The Provincial commitment to the development of a DHS, which evolved during the lifetime of the project, created the essential enabling environment for the community focused initiatives of the project and, given its operational capacity in this arena, the NDCSP in turn became a principal enabling partner of the DHSMT in the process. Initial efforts at CHC development were largely unproductive, being constrained by difficulties in determining authoritative structures among the many groupings that were found to exist in the community. These difficulties were compounded by the fact that multiple agencies were simultaneously engaging with communities without a clearly defined plan of action. However, during December 2000 the DHSMT, together with the NDCSP, TVT, NPPHCN and other partners in the initiative, created a Task Team to conduct an ad-hoc situation analysis in the light Ndwedwe District Child Survival Project Final Evaluation 2001 30 of the above difficulties and tackle the problems identified. The Task Team established protocols for community engagement as well as standardized definitions and scopes of work for the variety of community health workers currently operating in the District. Guidelines were also developed to help communities in the selection of CHCs. These tools were then introduced in all TAAs. Arising out of this process, a much more concerted and successful effort was made by the DHSMT to consolidate the structure and operations of CHCs, equitably distributed throughout the District, as the bed-rock of a sustainable DHS. The Task Team evaluated the status of CHCs in the District in October 2001 and found that only four out of an anticipated 19 had not been formed at that time, but by the end of the year all had been established. The foundations have also been laid for the creation of a District Health Forum that will be constituted by representation from all CHCs, and other community structures as appropriate, and through which the DHSMT will engage with communities in all matters pertaining to health. The current NDCSP Community Officer has played a lead role in this process, working under the direction of the District Manager. An emerging constraint is the potential for conflict between the traditional Tribal Authorities and the newly established Local Authorities, created following the Local Government elections which were held in South Africa at the end of 2000. Currently CHCs are constituted under Tribal Authority and the DHSMT acknowledges that merging both structures may be difficult, as evidenced by recent experience in the Bergville District where conflict has arisen between the two Authorities in the selection of CHCs. However progress with the merging process has been made in one geographic area in the Ndwedwe District and this augurs well for the future. The NPPHCN has been assigned responsibility for training selected members of the CHCs in organizational and operational methodologies and the NDCSP undertook the training of communities in CHC selection and in health related matters. Through Participatory Learning Activities (PLA), the NDCSP Community Organizer facilitated communities in introducing standardized procedures for CHC selection. CHCs were also facilitated in determining their own health priorities and developing action plans to address their main health problems. The placement of TBAs and HBCVs, trained by the project, in most communities contributed to the on-going process of helping communities to develop a health perspective, with particular emphasis on Child Survival, Reproductive Health and HIV/AIDS. The TBAs and HBCVs are incorporated in most, if not all, CHCs and further enhance the contribution of the NDCSP towards community mobilization and empowerment. CHWs (oNompilos) are deployed in all Tribal Authority Areas and are paid a stipend by the Provincial DOH. The DHSMT Task Team selected TAAs for CHW placement, based on the status of existing health services and previous CHW allocations. Eighty-two have been deployed in the District to date, from a Provincial total of 2500, with a provisional plan for a final deployment of 10,000. All existing non-volunteer community health workers have been absorbed in the CHW cadre, thus creating uniformity throughout the Province. The TVT and NPPHCN have been contracted by the Provincial DOH to select and train CHWs and to employ and manage the cadre. This process has been subject to two very positive independent evaluations during 2001. The TVT has intimated its intention to engage other Ndwedwe District Child Survival Project Final Evaluation 2001 31 partners in monitoring and supervising the CHWP throughout the Province, in line with Provincial policy to devolve responsibility for the entire community component of the DHS to the communities themselves. The NDCSP is ideally positioned to support this initiative by developing and testing methodologies and criteria for monitoring, evaluating and certifying the performance of CHCs and CHWs in the Ndwedwe District that can be extended to other Districts in time. It is intended that the NDCSP would support local NGOs/CBOs to undertake this responsibility in the interest of long term sustainability. The Ndwedwe District is unique in that it has two trained indigenous volunteer cadres which are actively involved in health promotion and basic health care delivery in the community. No other District in the Province has trained TBAs; in addition, the level of HBC supervision has not to date been matched in any other District. The NDCSP is responsible for this unique situation, having trained the TBAs as one of its primary project activities and initiated the supervision of HBCVs in the District as a key element of the HIV/AIDS intervention of the project. During this evaluation, service principals suggested that the contribution of these two cadres to the development of a DHS in the Ndwedwe District constituted a significant advantage over other Districts. Focus group discussions with representatives of each cadre revealed them to be highly motivated, eager to acquire greater skills and to have more resources in their kits to enhance their role, which is especially targeted towards vulnerable households. This motivation was also apparent in discussion with untrained inter-faith group representatives. Probably because of their tradition of voluntary service, the TBAs did not at any stage in discussion mention remuneration whereas the more recently recruited HBCVs felt that some form of financial support was desirable (this is currently under consideration by the DOH through an allocation from the HIV/AIDS budget). Two NDCSP trained TBAs and some 30% of the HBCVs supervised by the project team have been elevated to CHW status within Ndwedwe District. This potential career path serves as a motivation for volunteer community workers like TBAs and HBCVs. It is widely agreed that this aspiration is a reasonable one although the educational requirements for CHW selection are beyond the reach of most volunteers. In focus group discussions, both TBAs and HBCVs voiced their concern about the selection process for CHWs and felt that their substantial voluntary efforts should be given a weighting in the selection process. It was revealed in discussion with service principals that some of the earliest CHWs trained in the Province had less than adequate literacy (e.g. ability to read and interpret a RTH card) and as a result, a standard 8 education is now a basic requirement for selection as a CHW. The experience gained by the project team in training and supervising TBAs and HBCVs strongly positions the NDCSP to undertake a lead role in the extended community training initiative currently being planned by the DHSMT and the Directors of key Provincial Programs. It is a recommendation of this evaluation that, in addition to the IMCI Community Module being refined in the Bergville CSP, the curriculum should also include the TBA Training Module developed by the NDCSP Training Coordinator in Phase 1 and the Social Welfare Module currently being developed by the project’s Community Organizer with responsibility for Welfare and Social Services. The inclusion of the HIV/AIDS module currently used to train HBCVs as well as key KPC findings would create a comprehensive curriculum that the NDCSP would help to deliver to the community in a manner that would generate a cascade effect. All current Ndwedwe District Child Survival Project Final Evaluation 2001 32 project team members should have a role to play in the refinement of this curriculum and in devising and implementing delivery and supervisory methodologies. When the Community dataset has been agreed upon by the DHSMT, the community training initiative will provide an ideal vehicle for its implementation. Community Health Facilitators (CHFs) directly supervise the CHWs. The Provincial DOH has retained responsibility for the training and employment of this cadre. Current plans are for CHCs to select Supervisors from their members to monitor CHWs. The CHWP evaluation of September 2001 has raised doubts about the effectiveness of this process especially since all CHWs are paid while all CHC members are volunteers and these plans will need to be re-visited. It is eminently feasible that the CHCs should monitor and support the voluntary cadres and the introduction of systematic procedures, protocols and methodologies for this process should be an anticipated output of Phase II of the NDCSP. Overall, considerable progress has been made in mobilizing and empowering communities throughout the District. The NDCSP, particularly in the last year of Phase 1, contributed in large measure to this entire process. Valuable experience has been gained and important lessons learned by the NDCSP and its partners in this process, as evidenced by the highly productive changes in operational methodology introduced in the past year. The principal lessons learned in the above activity include: 1) the importance of engaging with communities through the prevailing authoritative structures in the community from the outset; 2) the importance of developing standard procedures and methodologies for engaging with communities; 3) the importance of developing a clear action plan with defined roles and responsibilities when multiple agencies are pursuing the same process. In the evaluator’s opinion, achievements to date in Community Mobilization and Empowerment are highly sustainable for a number of reasons. a) the process is driven by a clearly articulated Provincial DOH Policy as well as a determination by senior DOH management to develop a functional DHS which will operate largely through the CHCs and indigenous community workers intimately linked to the clinic based health services through CHWs; b) the prevailing positive attitude amongst middle management, service managers and service providers towards CHWs, CHCs and community volunteer cadres; and c) the evident enthusiasm for the whole process found among community volunteer personnel interviewed. Further development and consolidation of this process can and should be a major output of Phase II of the NDCSP. b. Communication for Behavior Change The NDCSP behavior change assessment program has included both qualitative research and surveys which were aimed at establishing priority areas for behavior change interventions, assessing existing practices and beliefs, and identifying influencing factors that constitute barriers to and opportunities for behavior change. Ndwedwe District Child Survival Project Final Evaluation 2001 33 In addition to the project’s baseline KPC, the NDCSP was the site of a pilot test of the KPC 2000. The study included a KPC survey for Ndwedwe Sub-District designed to assess knowledge, attitudes, coverage and practices on child survival issues relevant to project interventions, including the HIV/AIDS sub-project that was set to commence in early 2000. As a complement to the sample survey, project staff in collaboration with DramAidE implemented an in-depth qualitative behavior change assessment of individual and community attitudes and practices related to HIV/AIDS. This assessment used focus group discussions and semi￾structured interview methods to examine factors contributing to the runaway spread of HIV in the project area, including prevailing sexual norms and constraints to condom use. Community attitudes towards and willingness to provide care for PLWA and CABA were also assessed. The combined quantitative-qualitative study identified audience segments in need of special attention (e.g. non-maternal caregivers), and offered recommendations for behavior change messages and strategies (such as collaboration with interested church groups). In addition, during 2001, the NDCSP pilot tested a GAPS Analysis instrument in two communities of the sub-District. The GAPS Analysis, based on a methodology under development by the KZN Provincial AIDS Task Team and World Vision, is a technique for identifying priority intervention areas by assessing the degree to which prevailing community practices deviate from a set of Key Family Practices currently being promoted as part of the community component of IMCI. The pilot test established that the GAPS Analysis, in addition to spotlighting problem areas for BCC emphasis, was unexpectedly useful in identifying factors that commonly impede the adoption of improved practices. A medical anthropologist who is a consultant to the NDCSP is currently in the process of formalizing this function of the GAPS Analysis with the objective of expanding it into a comprehensive behavior change assessment tool in the promotion of improved Key Family Practices. These include HIV/AIDS-related practices which the Provincial AIDS Task Team wishes to encourage throughout KZN at this time. It is recommended that the project continue to use this tool, particularly in conjunction with community partners, to carry out GAPS Analyses in the catchment of those clinics in the newly incorporated project areas. This will enhance the project’s ability to identify priority practices for special emphasis during Phase II, as well as factors currently limiting or supporting the ability of households to adopt desired priority practices. Formative research completed during Phase I also suggested that BCC activities should include the creation of additional School Health Clubs in order to positively influence social norms related to high-risk HIV/AIDS behaviors among youth as well as mobilizing tribal and church leaders in the community to support this effort. DramAidE participated in the KPC 2000 qualitative HIV/AIDS assessment, and this project partner is well positioned to ensure that the research findings are reflected in the songs, dramas and other activities which the School Health Clubs utilize to promote healthy, low-risk behavior among school children and out-of school youth. It is recommended that this be a key focus of the BCC strategy during Phase Two. Efforts should also be made to share the results of formative research with those local faith-based organizations that are working with the NDCSP in attempting to modify the behaviors that are contributing to the spread of HIV/AIDS in the District. The NDCSP’s multi-level behavior change interventions during Phase I included an analysis of barriers to implementation of the Integrated Management of Childhood Illness. One lesson Ndwedwe District Child Survival Project Final Evaluation 2001 34 learned is that some aspects of the IMCI protocol, specifically those that discourage the dispensing of unnecessary medicines and antibiotics, are not popular either in the community or among clinic nurses. During Phase II, it is essential that efforts designed to improve compliance with IMCI protocols will be made concurrently at both health facility and community levels. These efforts must include regular follow-up supervisory visits to clinics to provide support to nurses who have completed IMCI training as well as an exploration of community perceptions with regard to the changes that IMCI protocols would bring to current knowledge and practices. The planned community training initiative will provide the appropriate platform for the latter activity. Achievements to date in meeting project targets in EPI, CDD and Maternal/Newborn Care intervention areas suggest that Project’s approach to behavior change has been successful. However these successes are counter-balanced by apparent failure to effect similar beneficial change in PCM and HIV/AIDS. As has been suggested in the earlier Technical Section, these are newer areas of behavior change intervention for the NDCSP and its partners and long established attitudes and behaviors in these intervention areas have not been challenged over as protracted a period as those in the more ‘traditional’ MCH intervention areas. In broad terms the behavior change objectives of the Project were a) to facilitate the dissemination of information as widely in the community as possible and b) to explore obstacles and constraints to behavior change in order to develop more successful BCC methodologies. Overall the Project has made substantial progress in meeting these objectives. Extensive channels of communication have been developed to reach communities through trained information disseminators - health workers, the community volunteer cadres (CHWs, TBAs, HBCVs) and creche workers; through school based activities and through community groups, principally CHCs but also churches and faith based groups. Three partners of the NDCSP have contributed in large measure to these activities – TREE, DramAide and Diakonia. The extensive research efforts described above have identified a number of key behavior change issues which will require special attention during Phase 11. These include high levels of misinformation and stigmatization related to HIV; prevailing cultural norms that increase the vulnerability of women to infection and traditional practices relating to birth and death registration that act as constraints to care for OVCs and CABA. One of the Project’s HBC Supervisors has recently undergone training in the BEHAVE Framework and will be responsible for introducing this new BCC methodology to all NDCSP intervention areas. The training module developed by the Project Community Organizer with responsibility for Social Welfare, which is currently being introduced in all community contacts, should enhance access to available grants. The major lessons learned were: a) the value of active community participation in operations research aimed at determining priority BCC intervention areas, as shown in the IMCI Key Family Practices Gaps Analysis; b) the capacity of the CHCs, with support from CHWs and CBOs, to develop prioritized Health Action Plans when appropriately facilitated in the process and c) the value of the Gaps Analysis instrument in the planning and assessment of BCC interventions. Ndwedwe District Child Survival Project Final Evaluation 2001 35 The many project interventions aimed at empowering the DHS, which have been described in the previous section under Community Mobilization, will be enhanced by the BCC capacity building efforts of the project at community level. The extent to which BCC interventions methodology has been established at community level to date augurs well for the sustainability of this key intervention following project closure. If the methodology can be established and consolidated in each CHC which has been formed with project support throughout the District and extended to the new project areas, the project plans for sustainability are both realistic and achievable. The NDCSP is well positioned and equipped to fulfill this objective during the project extension. c. Capacity Building Approach i. Strengthening the PVO This project is fully consistent with and contributes directly towards the BHR/PVC Strategic Objective of increased capability of PVC’s PVO partners to achieve sustainable service delivery, as elaborated in the BHR/PVC Strategic Plan. Moreover, the project has supported the institutional strengthening objectives referenced in the DIP to include supporting the technical strengthening of the CSST, enhancing MCDI’s participation in CORE Group activities and in successfully networking with collaborating agencies implementing child survival related programs. Home office participation in the implementation contributes directly to strengthening the capability of MCDI to more effectively implement child survival projects in a manner consistent with BHR/PVC Strategic Objectives. The technical capacity of the home office has been strengthened directly through the participation of CSST members in the myriad CS training activities organized by the BHR/PVC, BASICS, CORE Group and CSTS. Clearly, technical expertise related to the implementation of the IMCI strategy at both the facility and household levels has been reinforced, as a result of MCDI’s participation in the IMCI Working Group and the HH/C-IMCI Workshop in Baltimore, January 2001. Close technical collaborations with the Maternal Health, EPI, and IMCI Programme Managers at the provincial level, and the HIV/AIDS Coordinator at the Regional level have all contributed positively to MCDI’s efforts to improve its design and implementation of maternal/neonatal, Control of Diarrheal Disease, Pneumonia Case Management, and HIV/AIDS/STI interventions. These Programme Managers have been extremely supportive of MCDI’s efforts to incorporate innovative activities into its portfolio (for example, HIV rapid testing protocols, community-based models of care, and wealth ranking tools) which has enabled MCDI to acquire significant knowledge and expertise apropos of the design and implementation of prevention and care programs. These activities are currently being utilized in other, related MCDI projects, such as our new Prevention of HIV/AIDS among Umbutfo Swaziland Defence Force Personnel project. In an effort to improve its overall capacity and organizational performance, MCD headquarters in Augusta, Maine conducted a series of strategic planning workshops in 1998, in which the International Division participated. Using a synthesis of organizational assessment methods (e.g., Strategic Planning Workbook for Non-Profit Organizations, Miller Foundation; Partnering to Build and Measure Organizational Capacity, CRWR Committee; Management and Organizational Sustainability Tool, (MOST)), both the domestic and the international divisions Ndwedwe District Child Survival Project Final Evaluation 2001 36 identified strengths and weaknesses and assessed their impact on the organization’s ability to carry out its mission statement. As a result, in 2000, the Board of Directors mandated the creation of an inter-divisional Technical Advisory Group, comprised of staff members from both the international and domestic divisions, that provides technical support to the international division. It is worth noting that the dissemination of technical knowledge and lessons learned occurs throughout the entire organization via the Technical Advisory Group. During the first quarter of 2002, MDCI will embark on a capacity assessment using the Institutional Strengths Assessment (ISA) tool developed by CSTS. To carry out this assessment MCDI is collaborating with CSTS to utilize the ISA methodology and survey instrument. The assessment will include the participation of the field team in South Africa; the results of this survey will be used to formulate an organizational capacity development plan to address the weakness identified at the home office. Subsequently, it is MCDI’s intent to adapt the ISA to assess the capacity of the field office and its partners. ii. Strengthening Local Partner Organizations Partnership with the DOH - Creation of the District Health System The DOH has been the principal local partner of the NDCSP throughout and the project Manager and the project Community Organizer were members of the DHSMT from the outset. The Program Manager served as the District Information Officer for a period and subsequently mentored a local counterpart in that office. During this time the clinic data register developed by the Bergville CSP was pre-tested in the District but this process was overtaken by the development of a Provincial HIS which is now well established but not yet fully operational. The NDCSP introduced a data set, which included some additional indicators for Project monitoring purposes. These were collected quarterly from all clinics in the District with an excellent level of compliance and this discipline would have contributed to the development of the HIS. Because the Provincial HIS is now capable of providing the project with the datasets required for monitoring purposes it is no longer necessary to collect Project specific data from the clinics. The NDCSP provided substantial support to the DHSMT in the development of the community infrastructure of the DHS, through the creation of CHCs throughout the District. Standardized protocols, procedures and methodologies for community engagement have been developed and introduced. These initiatives are described in detail in Section 3a above. The groundwork has been completed for the formation of a District Health Forum by which communities and the DHSMT will interact and the roles and responsibilities of the various partners of the DOH, including communities, have now been clearly defined. Planning for the further development and consolidation of these community structures is well advanced with the NDCSP playing a lead role in the process. The DHSMT developed its current functionality during the lifetime of the project. The community engagement experience of the NDCSP, especially that gained in the identification of existing TAs, the process of selecting and the subsequent training program of the selected TAs was used to good effect by the DOH in the DHS development process. Two other local partners with long experience in community work in the Province – TVT and NPPHCN, also contributed greatly in the process. The District Manager effectively harnessed and merged the skills and experience of the major partners in the operations of the DHSMT, which will have a much wider scope of operations during Phase II of the project. The new Ndwedwe District Child Survival Project Final Evaluation 2001 37 District of Ilembe (KZ 29) will incorporate three Districts as currently constituted, including Ndwedwe , which with its newly defined boundaries, will become a sub-District of KZ 29. All the major partners have been invited to membership of the larger DHSMT and this will help the NDCSP to introduce those community initiatives for which it is highly regarded to a wider audience. Partnership with the DOH – Program Support The NDCSP supported the activities of key Provincial Programs, most notably Maternal Health, EPI, IMCI and HIV/AIDS. It was evident during the evaluation that current and projected NDCSP support to the Programs is welcomed and appreciated. Relationships between Program Principals and the project team are very cordial and other Programs, such as Youth and Women’s Health, are keen to forge partnerships with the NDCSP. This is largely because of the project’s solid performance to date in engaging with communities through CHCs, religious groups, schools, community health workers and health professionals and also because of its contribution to the operations of the DHSMT. All the Program personnel interviewed were enthusiastic about the benefits which derive from a partnership with the project. Support to the Maternal Health Program was provided through the training of TBAs, supervision of nurse-midwives undertaking PEP and training of clinic nurses in Family Planning. The training of TBAs was a unique and successful undertaking of the NDCSP. Criteria for selection for training were established, the training module written, and 39 TBAs were trained (in four training groups) and quarterly supervision maintained thereafter, both by the project trainer and clinic nurses. The whole initiative was very successful and over 75% of the trainees remain active and constitute a most valuable cadre of voluntary community workers. Although some were lost through moving out of area for political and economic reasons and two elderly but active members of the group died, some were also promoted (two were recruited as CHWs) and a number received further training as HBCVs. Their core role also changed and TBAs contributed to the dramatic shift in recent years towards institutional births in line with Provincial policy and they currently undertake supervision of home births in emergency situations only. They have all been trained in HIV/AIDS, STIs, early recognition of pneumonia, management of diarrhea and the danger signs of illness in the neonate. Their eagerness for additional training and their demonstrated capacity to absorb new health messages suggests that they should be among the first to be trained under the new community training initiative to be implemented during Phase II of the NDCSP. It is also anticipated that the active TBAs in the extended District area will be selectively identified for training under this initiative although it is not intended to repeat the ‘formal’ TBA training process which occurred during Phase 1 of the project, as described earlier. The self-directed Perinatal Education Program (PEP), which was developed in the Province, is highly regarded. It consists of three modules – obstetric care, newborn care and HIV/AIDS and will most probably be designated as a pre-requirement for Advanced Midwifery training in the Province. The NDCSP made a significant contribution to the uptake of PEP in the District and has been supervising at least one nurse-midwife in each of the five clinics and in one hospital in the District. The Project Training Coordinator visits each of these midwives monthly to monitor Ndwedwe District Child Survival Project Final Evaluation 2001 38 progress, answer questions, offer advice and, in conjunction with the Maternal Health Program Manager, she delivers a clinical practicum to augment the Program’s written material. The NDCSP offered support to EPI in a number of activities. As well as promoting EPI objectives in all its CS and Maternal Health initiatives, the project has offered logistical and personnel support to the two EPI mass campaigns which were conducted during the lifetime of the project. These campaigns were very successful in reaching over 90% of target populations for catch-up measles and polio vaccinations. Completed immunization rates are higher in the District than those reflected in the KPC results, since mass campaign vaccinations were not recorded in RTH cards. The EPI has now been subsumed into the IMCI initiative and the NDCSP has been prominent in this initiative from its 1998 launch and was largely responsible for the Ndwedwe District being chosen as one of the pilot Districts for the introduction of IMCI in the Province. The Project Training Coordinator was one of the original IMCI trainees of the Provincial DOH and also received training as a supervisor. The Training Coordinator subsequently acted as the District IMCI Supervisor. The core IMCI messages were also included in the TBA quarterly supervisory schedules, thus extending the reach of the initiative. The IMCI initiative in the Province was evaluated through a Health Facility Survey conducted in May 2001 in KZN as well as in three other Provinces – Mpumalanga, the Northern Province and the Northern Cape. The overall results suggest that the implementation of IMCI in KZN to date has not been as successful as in the other three Provinces. However substantial improvement in the performance of health workers in applying IMCI practices was apparent in KZN and valuable lessons were learned for the accelerated implementation phase. These include: strengthening the training team; placing more emphasis on nutritional assessment of the child; regular on-site follow-up supervision after training; and preparing communities for the changes in both health worker and caregiver practices being introduced in the IMCI initiative. The NDCSP will continue to support the implementation of IMCI in the District by assuming responsibility for the introduction of the IMCI Community Module as well as for providing supervision of health workers following training. The level of support provided by the NDCSP to the HIV/AIDS Program has grown considerably with the implementation of the HIV/AIDS component of the project. (see Technical Section above). Partnership with Other Local Partners The NDCSP worked in partnership with four local organizations – Oakford Clinic and Priory, TREE, DramAidE, and Diakonia, during Phase 1 of the project. Oakford Clinic and Priory is a Catholic Mission facility which operates large primary and secondary schools as well as a crèche, in addition to the Clinic. The latter operated as a fee￾paying facility during the early part of the project, at a time when all Government health facilities provided free health care. Because it was perceived to be a repository of cash income it became the subject of repeated robberies and was forced to close down completely for most of Ndwedwe District Child Survival Project Final Evaluation 2001 39 2001, having previously curtailed after-hours services for security reasons. At the time of the evaluation the clinic had resumed a limited daytime service with a reduced staff complement, following receipt of DOH funding. This funding arrangement is an interim one and is not sufficient to allow former services levels or staffing levels to be restored. Even this curtailed service is not guaranteed in the long-term. The Sister in charge stated that she was not sure what the future held for the mission clinic or for how long operations could be sustained since at the time of the evaluation the DOH had not reached a decision on the long-term future of the facility and its services. There are adequate alternative public facilities in the area but the mission had recently donated land to the Government and it was anticipated that some 1000 households would be developed on this land. This projected expansion in the clinic’s catchment population may secure its future, but perhaps most likely as a DOH facility. The Clinic is situated on the fringes of the project’s area of operations but was chosen as the base for the TBA training for a number of reasons. These include: 1) the willingness of the clinic to facilitate the project initiative because it was providing supervision to TBAs at that time and these would be incorporated in the training program, 2) the mission had the required hostel facilities for the trainees and was able to generate income from their use by the project, and 3) the clinic was at that time conducting deliveries on-site and the nearby District Hospital at Osindisweni, with its busy labor ward, provided an alternative location for additional practical experience for the trainees. The clinic proved to be an ideal location and three of four planned TBA trainings were conducted there. The Clinic staff felt that they had all benefited by association with the TBA training program although they were only indirectly involved. However the partnership with the NDCSP brought an unexpected benefit when the Clinic was chosen as the pilot site for the Rapid Syphilis test. Two clinic nurses were trained in the implementation of the test process and clinic staff were very impressed with the efficiency of the on-site test which guaranteed that effective treatment was given on the day to women attending antenatal care who tested positive. It was found that the prevalence of positive test results was higher than that which had been the norm with the previous off-site test process. This finding led the District Hospital to review its test procedures and quality assurance processes. As stated earlier, a study conducted by the project confirmed the cost effectiveness of the rapid tests (see Attachment 4a for a summary of this study). The partnership between Oakford Clinic and the NDCSP was very cordial and mutually beneficial. During the evaluation, clinic staff stated that they hope that an on-going relationship with the NDCSP will still be possible, however, they are concerned that the current parlous state of clinic operations might create difficulties. The Priory, however, is currently being considered as a site for a ‘model’ crèche to cater for OVCs and CABAs which the NDCSP hopes to develop in co-operation with Siniziso, another Catholic organization which is the principal trainer of HBCVs in the Province. Training and Resources in Early Education, (TREE) is the largest Early Childhood Development (ECD) resource and training organization (RTO) in South Africa. It was founded in 1984, has extensive experience in its field of operations, is fully accredited and is a major contractor to the Department of Education. It operates throughout KZN which has an estimated 1.5 million children under six years of age. TREE’s core activity is the training of trainers of Ndwedwe District Child Survival Project Final Evaluation 2001 40 crèche operators. Many trainers come from remote rural areas and tend to operate in those areas following training. Supervisory structures are based around four regional teams and eight area managers, distributed throughout the Province. TREE’s trainees form an extensive network with tremendous potential to influence health related behavior in the community and it is hoped that this potential can be further developed with additional project support. TREE approached the NDCSP for help with the inclusion of child health issues in its training curriculum. The project trained twelve Master Trainers in four intervention areas – immunization, diarrheal disease, pneumonia and maternity care. As well as learning the health information content of these interventions, the trainers were also given instruction in training materials production and curriculum development as well as pre- and post-test methodology. The NDCSP also conducted a Capacity Assessment of TREE in 2001 which highlighted organizational strengths and weaknesses. Three priority areas requiring urgent attention were identified: 1) The need for greater long term financial sustainability, given TREE’s dependency on overseas financial support. Greater emphasis will be placed on the income generating capacity of the organization to address this problem. 2) The need to develop a greater staff capacity. Systematic staff training is being planned and management training is to be sought to remedy this deficiency, possibly from the NDCSP. 3) The need to develop a HIV/AIDS policy and to include psycho-social issues pertaining to the care of children in its curriculum. Support was to be requested from its partners, including the NDCSP, to facilitate this development. The current project workplan includes HIV related training for TREE Facilitators and other areas of support from the NDCSP should be developed as appropriate strategies and opportunities evolve. The partnership with TREE will used in the proposed model crèche development and TREE’s network of area managers and regional coordinators will be used to promote improved care for OVCs, including their psycho-social needs. DramAidE (Drama AIDS Education) was established in 1992 and is based at both the University of Zululand and the University of Natal (Durban), the major tertiary institutions in KZN. It is a donor-funded independent organization which promotes the use of participatory drama and other interactive educational methodologies for HIV/AIDS, life-skills and sexuality education. The methodologies used are culturally sensitive, are accessible to all ages and levels of education and are multi-cultural. DramAidE operates mostly in schools and tertiary colleges but also in the community, where it trains teachers, nurses, care-givers etc., and works with CBOs and church groups. It also works in the field of health communication, promoting the use of participatory methodologies in the development of health promotion campaigns for schools, colleges, municipalities etc., with special emphasis on HIV/AIDS prevention and care. The NDCSP agreed to sponsor DramAidE over a two-year period to engage with rural schools in the project area and facilitate the development of School Health Clubs in each school with the aim of fostering healthy behavior and AIDS awareness among pupils and out of school youth. The planned activities were conducted in eight secondary schools, chosen by the Schools Inspectorate for their isolation, during the grant period. Ndwedwe District Child Survival Project Final Evaluation 2001 41 An independent evaluation was conducted during 2001, which concluded that the intervention had successfully achieved its major goal of attitudinal and behavioral change with respect to HIV/AIDS among pupils in the target schools (see Attachment 4b for a summary of the DramAidE evaluation report). A significant increase in knowledge about healthy sexual behavior and practices and about the risk of HIV transmission was demonstrated among these pupils. The out of school youth engaged by DramAidE as program activity presenters also benefited to a significant degree. Teachers gained confidence in their ability to discuss issues related to HIV infection with pupils and learned not to discriminate against infected persons and also learned about caring for those infected and affected with HIV. Overall the school level activities were considered to have been “remarkably successful” by the independent evaluator. The anticipated cascade effect, whereby the Health Clubs established in target schools would then initiate the formation of such clubs in neighboring schools and among community groups, however, did not take place. A number of reasons were advanced for this failure, which include: 1) the geographic isolation of the schools (ironically, the main reason for their selection), 2) the limited time available since the activity of the Clubs is given low priority in the school time￾table, and 3) inadequate mentoring by school principals and teachers to facilitate engagement with the wider community and lack of funds to support outreach activities. The evaluation suggested that the limitations described are typical of the constraints that affect rural interventions, largely because of inherent lack of cohesion and infrastructure in rural communities to support the cascade effect. A number of problems were highlighted in interviews with DramAidE principals. From the organization’s viewpoint major constraints included: 1) NDCSP procedures allow for payment in arrears only and this affected the operational capacity of the organization, 2) interaction with the project was poor with the result that required, and anticipated, technical support from project personnel was not forthcoming, and 3) training provided by the project, e.g. in HBC, was delayed. From the project viewpoint, team members felt that they were not afforded the opportunity to influence the content of DramAidE’s messages. The appropriateness and accuracy of DramAidE’s messages were also questioned by the Youth and Adolescent Program Manager at the provincial DOH, who expressed the desire to co-operate with the NDCSP in reviewing and monitoring the content and impact of DramAidE’s activities. There appeared to have been very little contact between DramAidE and the Project Team, and little institutional awareness about the partnership remained in the NDCSP following the departure of the outgoing Project Manager. It was agreed that these issues would be addressed during the extended partnership. A higher level of technical input from the project is planned and a technical contact person would be designated within the project team. The message content of DramAidE’s activities will require continual monitoring and review by the project in conjunction with key DOH Programs and impact evaluation methodologies will need to be developed and implemented. A limited co-operative partnership has been developed between the Project and DIAKONIA, a Durban based Council of Churches. Both parties benefit from this co-operation – the NDCSP is being facilitated in its efforts to engage with churches in mobilizing support for the community- Ndwedwe District Child Survival Project Final Evaluation 2001 42 based initiatives that are central to the project and DIAKONIA is being introduced to the smaller churches and faith based groups (both of which are mainly indigenous) with whom it would not have previously engaged. This partnership will be strengthened further during Phase 2 of the project as both partners attempt to build on the existing motivation and capacity of religious leaders and their churches to support HIV/AIDS related activities in the community. At the time of the evaluation, discussions were taking place between the NDCSP and Sinosizo in respect of the crèche developments described above and with Tsu Chi, a Taiwanese Foundation, about the possibility of developing income generating projects for women caring for OVCs and CABAs. Overall, the partnerships between the NDCSP and the variety of local partners described have been very productive. The partnerships have, individually and collectively, facilitated the implementation of core elements of the project and each of the partners has, in turn, benefited from its engagement with the NDCSP. The Capacity Assessments conducted within TREE and DramAidE have identified areas of organizational and operational weakness and remedial plans have been developed which will be supported by the NDCSP during the extended project. Partnerships with local organizations are essential elements of an externally funded and managed project, both to enhance the capacity of the project to achieve project objectives and more importantly to ensure long-term sustainability. The NDCSP should devote sufficient resources to foster existing and projected partnerships to these ends during Phase II. It would appear that project inputs to key partnerships have not been adequate in this respect during Phase I, and is highlighted as one of the few weaknesses detected in the management of the project to date. However, the workplan for Phase II seeks to redress this deficiency by designating responsibility to an individual project team member for liaison and co-ordination with each partner organization. iii. Health Facilities Strengthening The NDCSP did not have any direct responsibility for Health Facilities Strengthening in the Project DIP, and was therefore not involved in facility management and operations. Facility level interventions were focused on improving health worker performance (described below) and the introduction of rapid HIV testing capacity at two regional hospitals and rapid syphilis testing at one clinic in the District. The Project also initiated and facilitated a review of the logistics of essential supplies to clinics, with particular emphasis on vaccines and the EDL, as part of its responsibility for IMCI supervision. Health workers in all 5 clinics and both hospitals were given follow-up supervision in IMCI; supervision of the self-directed PEP and training as HIV/AIDS Counselors. This contributed significantly to the progress made in achieving Project targets in core activities. In addition, the rapid HIV testing process was introduced by the NDCSP, in collaboration with PATH. PATH personnel provided training in the rapid HIV test process to selected laboratory staff and to clinic nurses in two hospitals in the District and supervised the establishment of test procedures and protocols. Training in the test process was conducted only if pre-conditions agreed by the Project partners were fulfilled at the hospital. These included a) an established capacity for pre- and Ndwedwe District Child Survival Project Final Evaluation 2001 43 post-test counseling, b) informed consent for testing routinely obtained and c) patient confidentiality assured. During site visits at the time of the evaluation staff at Osindisweni and Montebello hospitals welcomed the rapid HIV test process for its ability to reduce the turn￾around time for HIV test results and reduce the test-client mismatches, which are common when sample analysis is done at a distant site and results are not immediately available to service providers. The cost efficiency of the test process was determined in a study conducted by MCDI in March 2000. Attachment 4a provides a summary of this study. The rapid test procedure was not being conducted at the time of the evaluation in either hospital because test reagents were out of stock. This default occurred because of a delay in placing the order for replacement stock, occasioned by the un-expected departure of the Project Manager in late 2001. The shortage of supplies was compounded by wastage that occurred in supplies at Osindisweni Hospital due to evaporation of diluent fluid in some test kits, an occurrence which remains unexplained. The rapid syphilis test was introduced at Oakford Clinic and was considered to be a very important contribution in ante-natal service provision. By ensuring that appropriate treatment was administered to patients testing positive on the day, the risk of a patient defaulting essential treatment was significantly reduced, thus ensuring a much more efficient and effective service provision. It is hoped that this test capacity can be extended to other clinics during Phase II although there is widespread resistance to the process, as discussed in the Technical Section above. A key element of the IMCI supervision was the joint visits which were made to all clinics by the Regional Pharmacist and the Project Training Coordinator, which were planned and facilitated by the latter. During these visits the status of essential commodity stocks held at the clinics was evaluated and procurement procedures for essential supplies were revised and strengthened. During the Health Facility Survey conducted as part of the IMCI initiative it was clear that these visits were productive as all clinics in the Ndwedwe District, as well as in the other participating Districts, achieved a maximum score in respect of the availability of essential commodities (vaccines and drugs), though only half of all clinics rated as highly in respect of essential equipment and supplies (weighing scales, timers, ORS mixing bowls, charts etc.). The NDCSP should endeavor to address the latter deficits through the DHSMT, as a key element of its training and supervisory roles at health facility level. The Hospital and Clinic Assessment Reports of March 2001 suggest wide variation among facilities across a spectrum of parameters – defined catchment populations, service organization, staffing levels, service mix, skills mix, commodities supply, supervisory structures and support services, among others. Field visits to the various facilities during the evaluation confirmed these variations and it is the evaluator’s opinion that observed wide variations in health worker productivity and performance would also be confirmed if a critical performance and productivity analysis is performed. An instrument being developed in Lesotho presently by MCDI could be usefully employed in this regard and the Superintendent General expressed interest in this process. Ndwedwe District Child Survival Project Final Evaluation 2001 44 A solid foundation for a functional and productive CHC–CHW–Clinic operational axis appears to be have been established. The evaluator found support at all levels for this axis and all clinics supervise CHWs and most also provide supervision or support to TBAs and HBCVs operating within their catchments. The same applies to primary care activities at the two District Hospitals (Osindisweni and Montobello) and at Kwa-Mashu Polyclinic. Each of these three institutions provides supervision and logistics support to designated clinics in the District and is also responsible for the delivery of all outreach services to the Health Posts where community workers, both paid and voluntary, have their closest interaction with service providers. It is the stated intent of the Provincial DOH that this axis of interaction between services and communities will be strengthened to create a functional DHS. As described earlier, all elements of the community arm of this axis will receive support during the extended NDCSP, with particular emphasis on consolidating both the CHCs and the CHW cadre as well as the indigenous community volunteers. The supervisory capacity of the clinics will also be strengthened and regulatory systems will be developed and implemented to enhance the functionality of the CHBP. The NDCSP has been assigned a major role in this development. The NDCSP also provided financial support to The Valley Trust, a local NGO, for health post construction during Phase 1 (the equivalent of one half of the cost of one facility). Further support is planned for Phase II. iv. Strengthening Health Worker Performance/Strengthening Training One of the primary objectives of the NDCSP from the outset has been strengthening health worker performance through training. The Training Coordinator was the project’s earliest recruit and remains the most experienced team member. Initial training efforts focused on the supervision by clinic nurses of project trained TBAs operating in their clinic catchment. This was an essential adjunct to the quarterly supervision and in-service training provided to the TBAs through the project. The engagement of clinic nurses in this manner contributed to the overall performance of the TBAs, which appears to have grown over time as new elements have been added to their training as well as forging closer relationships between the health services and community workers. Supervision of the Perinatal Education Program was a principal responsibility of the Training Coordinator and to date at least one nurse from each institution has been supervised at various stages of the three-module Program. This activity is likely to be given added importance by the Maternal Health Program during Phase II, as it is likely that completion of PEP will be considered an essential step for nurses contemplating Advanced Midwifery Training. Following her attendance at a workshop on Integrating Family Planning in Existing Projects held in Malawi in 1997, the project Training Coordinator arranged with the Maternal Health Program to train nurses in Family Planning. This was done at Osindisweni District Hospital and all clinics selected at least one nurse to attend the training course. Initial attempts by the project to introduce Pneumonia Case Management training were overtaken by the Provincial decision to implement the IMCI initiative. The NDCSP facilitated this process to a large extent in Ndwedwe as described elsewhere in this report and it is envisaged that the Ndwedwe District Child Survival Project Final Evaluation 2001 45 project Training Coordinator will play a key role in the planned extension of IMCI in the sub￾District in Phase II. The Training Coordinator has conducted training for nurses in all 5 clinics and each of the two District Hospitals in HIV/AIDS Counseling as a major initiative within the HIV/AIDS Project Component. The latest among these Counselors were being certified while the project Evaluation was taking place. Overall the NDCSP contribution to Strengthening Health Worker Performance through training activities in the District can be considered significant, despite the fact that few of the project impact indicators, which were discussed in Section 2b above, are directly attributable to the health worker performance during client contacts. Although the estimated level of coverage achieved in project-led health worker training initiatives is probably less than 20% overall, a solid start has been made and much of the training in all probability would not have been possible in the absence of the Project. Additionally, all the training initiatives were motivated by the Project Training Coordinator and delivered in the context of partnerships between the NDCSP and individual Programs, rather than as a mandated core project responsibility. The evaluator considers that the project approach has been an effective one considering the circumstances in which training and supervision of health workers was conducted, as outlined above. Contact with trainees was largely on-site in a practical setting, was repetitive and evaluative and in most instances ended in a certification process ensuring that performance objectives were met. Valuable lessons were learned and adopted along the way. Among these were the need for repetitive contact in introducing new skills; the appreciation that old skills are difficult to shed; that communities need to be prepared for changes in health worker attitudes and practices that are anticipated outcomes of training; and that when coverage levels attained in training are low, trainees can feel isolated when new skills imply changes in attitude and practice. The evaluation suggests that the project is very well endowed with health worker training capacity, which can be harnessed by the DHSMT in support of the many key health worker training initiatives currently being planned. During the lifetime of the extended project, the three highly qualified nurses and experienced trainers who form the core of the project team can make a major contribution to this process and help to ensure sustainability through skills transference and capacity building. The design and implementation of performance assessment tools that will facilitate the continuing assessment and certification of both health personnel and the various community workers already deployed will be a major challenge for the Project. Experience to date with these tools in project-led training suggests that this objective is achievable. Gaps between performance standards established during training and actual performance were identified during the Health Facility Survey conducted as part of the IMCI Evaluation. This finding was a cause of major disappointment to all concerned with IMCI training and supervision, and strategies to address the problem are under consideration. These include: a strengthened training team, an institutional focus with broader training coverage, more rigorous supervision, and parallel community training in the catchment of the target facility. Ndwedwe District Child Survival Project Final Evaluation 2001 46 d. Sustainability Strategy Sustainability has been interpreted by the NDCSP to mean that not only will measurable improvements be achieved in health status, health awareness and health care but that these improvements will be attained by strengthening existing systems and structures and creating new ones that will persist and be functional after MCDI has completed its work in Ndwedwe District. In recognition of the pre-eminent requirement for sustainability: · all NDCSP interventions have been conducted in partnership with the DOH, its key Programs and major partners, and in full compliance with current DOH policies and priorities. · To this end, the project has been a founding member of the DHSMT in the District and has become a key contributor in the effort to develop a functional DHS, the rudiments of which hardly existed in Ndwedwe at the commencement of the NDCSP. · In addition, the Project has contributed in large measure to the substantial progress that has been made at community level to establish the essential base for a functional DHS. · This effort has been described earlier in this report but it is worth emphasizing the extensive engagement with communities that the project has spearheaded, culminating in the creation of CHCs in all Tribal Areas, as well as the widely dispersed network of indigenous community volunteers that the project has trained and mentored. These achievements and the inclusive manner in which they were attained suggest that NDCSP initiatives in this arena are highly sustainable. The fact that these efforts supported the implementation of the Community Health Worker Program and the Home-based Care component of the HIV/AIDS Program, both of which are essential initiatives of the Provincial DOH, further augments the expectation of sustainability. The project has recently demonstrated that methodologies for health needs assessment, priority intervention planning and BCC can be successfully transferred to the newly mobilized community structures and a raft of capacity building activities at community level which are planned for the extended project will contribute further to the likelihood of sustainability of these efforts for community empowerment. The project has contributed to efforts focused on strengthening existing skills and transferring new skills to health workers and establishing procedures for supervision and monitoring of health workers following training. Notable among these efforts was the training in Family Planning and AIDS Counseling that was unique to the Project as well as the supervision provided to PEP and IMCI trainees. These efforts were conducted within the framework of the operations of key DOH Programs and the support which is planned during the extended project, especially to HIV/AIDS, IMCI and Maternal Health as well as to the HIS, will contribute further to sustainability. As stated above, the project has also contributed to efforts aimed at strengthening Health Facilities through the introduction of new rapid test methodologies for HIV and syphilis and also helped to re-organize the logistical support systems to clinics for IMCI activities as part of its supervisory brief. The NDCSP contributed to the sustainability of its local partners by supporting both TREE and DramAidE to: Ndwedwe District Child Survival Project Final Evaluation 2001 47 · undertake Organizational Capacity Assessments to identify both weaknesses and strengths, and · to develop plans to redress weaknesses (such as critical training needs) and to identify long-term strategic plans. Continued efforts to enhance the organizational development of local partners during Phase II will only further contribute to the NDCSP’s objectives of ensuring that collaborating partners will be able to continue important initiatives and procedures after the end of the project. The project has closely collaborated with the DOH and other local partners in efforts aimed at strengthening health services and creating an integrated locally based system of health care that intimately involves communities and indigenous systems and structures. This collaboration must be further developed during the project extension in order that the DHS may develop the capacity to appropriately respond to the new health and social challenges that are emerging, particularly as a result of the escalating HIV/AIDS epidemic. Overall NDCSP goals for sustainability have been met to date and there is every likelihood that a fully operational DHS with dynamic community participation can be achieved during the lifetime of this project. C. PROGRAM MANAGEMENT 1. Planning Planning for the NDCSP involved stakeholders within the District at community and health facility levels, as well as regional and provincial authorities. At the community level, groups consulted included the Amakhosi (traditional chiefs), their Ndunas (headmen) and councilors. Project goals and objectives were also discussed with the community in focus groups and at community meetings. All project activities from the outset have been fully incorporated in the DOH planning process for the District. The NDCSP has been a member of the DHMT, which has played a crucial role in ensuring integrated program planning and the implementation of project activities at the local level. All major stakeholders and partners were briefed about project goals and objectives. At the community level, project goals and objectives were discussed with traditional leaders and in community meetings. The formation of CHCs and the engagement of project trained and supervised volunteer cadres as well as church and tribal groupings enabled communities to be party to the planning process and to begin to take ownership of project initiatives local level. During the evaluation it was evident in discussions at every level that all the Project Team members are highly regarded by their counterparts and relationships are mutually respectful and without exception convivial. It was apparent that all project activities and initiatives were well known to DOH and partners reflecting the high levels of integrated planning and engagement with partners that has been established as the norm in NDCSP management. Ndwedwe District Child Survival Project Final Evaluation 2001 48 Project objectives have been used for the planning of quarterly activities, in order to ensure that the work plan is practical and stays on track. During Phase I, the Project Manager also made quarterly reports to the DHMT to solicit their input regarding ongoing and new activities. Progress towards project objectives is discussed regularly both informally and formally, and project staff regularly report to the project manager on activities within their areas of responsibility. These reports are integrated into the progress reports prepared for the Home Office in Washington. In addition, the project manager has made frequent field trips with professional staff to observe their activities and to provide supervision and technical assistance. Project work plans are developed which provide timelines for the implementation of project activities. Objectives for the completion of specific activities are established and work schedules of individual staff are planned accordingly. It was apparent during the evaluation that a high level of integration of activities has been established as the norm within the project, maximizing the use of project resources and opportunities to extend project initiatives. Progress toward meeting these objectives is tracked through the year. The evaluation suggests that Project planning was highly inclusive throughout and this is reflected in the degree to which the DOH and partners are familiar with project initiatives and activities and in the degree to which the NDCSP contribution to each of the key Programs with which it interacts is acknowledged. This high level of awareness was also perceived at health facility level and among community representatives who were interviewed. Given that no HIS, CHWP, DHS or DHMT existed in the District at the time that the DIP was prepared and reviewed, the DIP work plan can be considered highly practical and this practicality is confirmed by the fact that the core activities remain central to the DIP work plan being developed for the project extension. In the circumstances that prevailed at the time it would appear that planning for implementation was both skilled and intuitive and the MCDI Home Office, the initial Project Manager and the Project Training Officer deserve due credit for this. The challenge presented by HIV was not planned for in the DIP but as the influence of the escalating epidemic became apparent the project adjusted to the challenge by designing and successfully launching the HIV/AIDS sub-project component. 2. Staff Training Requirements for improvement in staff knowledge, skills, and competencies are assessed in relation to program objectives and job descriptions through staff meetings and discussions with the Project Manager. A substantial level of training, both formal and in-formal has been provided to the project staff over the life-time of the project. The three technical staff members are all nurses with multiple qualifications and extensive service records, who are highly skilled and experienced. The Training Coordinator was the first local person recruited to the NDCSP and has contributed greatly to the successful operations of the project and the degree to which these have been integrated into service delivery structures and into the operations of all the key Programs of the DOH. She has received training in a number of relevant areas - Family Planning; Vitamin A Deficiency; IMCI Training and Supervision; AIDS Counseling Trainer; PEP Supervisor – and Ndwedwe District Child Survival Project Final Evaluation 2001 49 she has been sponsored by the NDCSP to attend two major workshops in Washington which also enabled her to become familiar with MCDI Home Office operations and personnel. The two HBCV Supervisors have joined the project in the past year and both were supported by the NDCSP to undergo training in Project Management at the end of the project evaluation process. One Supervisor has undergone training in the BEHAVE Framework for BCC and the other has been earmarked for IMCI Training to facilitate the introduction of the IMCI Community Module. The Community Organizer with responsibility for CHC development has received informal training in techniques and methodologies for engaging with communities and in the Participatory Learning Activity approach. This training has been productive as reflected in the level of community mobilization achieved, especially in the past year, as described earlier. Attempts are being made currently to identify an appropriate Diploma or Degree Program in Community Health in South Africa which might be appropriate further training for this highly motivated and capable team member that would enhance both his role within the project as well as his future employment prospects. The most recent NDCSP recruit is the Social Worker. She has only been with the Project for a short period and is a Sociology graduate, currently undertaking a Masters Program. She has not received any formal training to date but her in-service experience with the project is already substantial, including team-working, community engagement, operational planning etc. In addition, the project Administrator has received training in management of USAID grants by the country mission. All team members actively participated in the evaluation process and in the development of the DIP Action Plan for the project extension. All were intimately engaged in the design and conduct of the KPC that preceded the evaluation. Professional staff are also kept abreast of developments in their fields through access to technical materials from other organizations and through the internet. Immediately following the evaluation, the project team was scheduled to undergo training to improve their skills in the Microsoft Office package (i.e., Word and Excel) as well as to utilize email and internet services. There is abundant evidence that project staff have applied the skills learned to project activities and this is reflected in the successes achieved in the various intervention areas. It would appear that adequate resources were devoted to staff training and this was apparent from field visits and from discussions with all project staff during the evaluation. The overall lessons learned include: a) value of having high levels of ‘in-house’ training capacity (the project has been particularly well endowed in this respect from the outset); and b) the extent to which productive in-formal training of project staff depends on skilled, experienced and committed ‘mentoring’ from project principals. Again the project rates highly in this respect. Ndwedwe District Child Survival Project Final Evaluation 2001 50 3. Supervision of Program Staff The NDCSP field office is a small unit. The MCDI policy manual provides the reference point for administrative and supervisory procedures. In the field, the managerial style is collegial and consultative. Progress towards project objectives is discussed regularly both informally and formally, and project staff regularly report to the project manager on activities within their areas of responsibility. These reports are integrated into the progress reports prepared for the Home Office in Washington. In addition, the project manager has made frequent field visits with professional staff to observe their activities and give feedback and technical assistance. Project workplans are developed which provide timelines for conducting project activities. Objectives for the completion of specific activities are established and staff then develop their own work schedule based upon this guideline. Progress toward meeting these objectives is tracked through the year. 4. Human Resources and Staff Management MCDI’s personnel policies and procedures in the field largely mirror those established for the Home Office, with the exception of a few minor policies to better reflect local South African human resource policies (e.g., salary payment of a 13th month). The field team is highly motivated and experienced and is poised to play a major role as a lead partner of the DOH in the further development of DHS in the new sub-District during Phase II of the project. Turnover in the project has been minimal, and new team members added under the HIV/AIDS sub-project have brought on board additional much needed expertise as the project team attempts to respond to the crisis created for many families in the District by the epidemic. Morale, cohesion and working relationships were found to be excellent during the evaluation and these characteristics were particularly demonstrated during the last quarter of 2001 when the Team successfully planned and conducted the final KPC, moved offices and prepared for the final evaluation, all without a Project Manager. This was a taxing time for the project but the team was rightly proud of its achievement and all felt they had benefited from the experience. Morale had been boosted unexpectedly by the decision of the project team to employ those HBCVs who are supervised by the project as field personnel during the KPC. The project team were pleased to be able to show loyalty to a dedicated group of volunteers with whom they had forged strong working relationships and the volunteers were very appreciative of the opportunity to participate and the income generated. Two staff members have left the project during its lifetime. The founding Project Manager resigned in late 2001. He had contributed greatly to the formative development of the project with the Training Coordinator and was highly regarded by senior colleagues at the DOH and among partner organizations for his experience and abilities. In addition, the contract of the first recruit to the post of Community Organizer was terminated in December 2001 because of non￾compliance with project management directives to engage with all the partners of the DHMT when organizing communities. His actions contributed to the delays that occurred with the development of CHCs, and in many cases the work he was employed to carry out necessitated repitition by the project’s other Community Outreach Organizer and the Social Worker. The eventual success of the CHC venture was built on valuable lessons learned, as described under Ndwedwe District Child Survival Project Final Evaluation 2001 51 Community Mobilization, earlier. The reasons for this team member’s waywardness are not clear but probably derive from personal as well as political motivation. No plans have been developed to date for staff transition because of the project extension but consideration is being given to the process as outlined, under Staff Training, above. The project team is also keenly aware of the need for enhanced efforts during Phase II to ensure the sustainability of the project beyond the cost-extension phase. The project team, in collaboration with the Home Office in Washington, continuously seeks and identifies other, external sources of funding to continue project activities beyond 2005. The NDCSP team has also cultivated collaborative relationships with partner organizations in South Africa (both government and non-government), which will foster opportunities for transition of staff to other employment opportunities, when the need arises. 5. Financial Management The financial management activities carried out by the Child Survival Project are consistent with the activities outlined in the DIP. At the home office, MCDI has a financial management system that is overseen by our Chief Financial Officer and is regularly audited. The financial management of the project funds is the shared responsibility of the field team and home office staff. In the field, the Administrator has the responsibility for implementing MCDI management protocols for field accounts. This protocol consists of a manual that outlines standard procedures for tracking all field expenditures and income fund transfers authorized by the Project Manager and the Director of the Division. All program expenditures are entered into an Excel spreadsheet and aggregated monthly. Field office expenditure reports are submitted on a monthly basis to CSST in Washington where they are reviewed and forwarded to the office of the CFO in Augusta, Maine. Field expenditures are tracked using a coded chart of accounts that corresponds to project grant line items. All accounts are reviewed and entered into an automated system and later merged with the main corporate system in Maine. In the field office, NDCSP operates a local expenditure account and only the Project Manager can authorize payments. Comprehensive financial reports are routinely compiled and forwarded from the field office to the Washington office where they are reviewed. If necessary, queries are sent to the field for clarification. While MCDI does not have a regional office in southern Africa to support the project in South Africa, members of the Home Office Child Survival Support Team provide on-going technical backstopping to the project. The Project’s financial management and accountability for project finances and budget were found to be very adequate and no budgetary adjustments were required. The project appears to be approximately on-target with expenditures, neither over nor under budget. No financial irregularities of any degree were apparent and the only avoidable losses occurred when a cash box containing some $200 was stolen and not recovered. Budgeting skills appear to be well developed and adequate supervision and support was provided from Home Office. No provision has been made to date for activities beyond the extended cooperative agreement but plans are in place to engage with new partners in intervention areas which will need on-going financial support e.g. care of OVCs. Ndwedwe District Child Survival Project Final Evaluation 2001 52 6. Logistics The logistical management of the NDSCP was excellent throughout although there was evidence of slippage at the very end. Recorded losses throughout the six years of the project amount to one cash box containing some $200, which was stolen and not recovered. There were no vehicle accidents and all vehicles procured are still serviceable at the time of this evaluation. This is quite a remarkable achievement, given that much of the accumulated mileage occurred on un￾surfaced roads, which are frequently impassable in the rainy season. Procurement and usage of other commodities was also very efficient and wastage was minimal (e.g. two printers discarded). In addition, some wastage occurred in one batch of the rapid HIV test materials due to reagent evaporation. This batch apparently was procured from India and the wastage could not be directly attributed to the project management. At the time of the evaluation these reagents were out of stock due to the failure to place an order late in 2001, with consequent service disruption. This delay was occasioned by the precipitous departure of the Project Manager and a failure to systematize the procurement process, which resulted in none of the remaining project staff being aware of the procedures for procurement. Apart from the latter glitch the project’s logistics systems are very well established and sufficiently strong to support the expanded Project activities anticipated during Phase II. 7. Information Management The project has made a significant contribution to the development of the HIS, which at this time operates on a Regional basis and serves a considerably larger geographic area than that covered by the Project. The former Project Manager served for a period as the District Information Officer on the DHMT and the project established the system of monthly returns from the clinics in the project area. These returns, which were still being made to the end of 2001, reflected service based activities being promoted and supported by the project and were merged with data pertaining to project activities to create the Quarterly Reporting Instrument (QRI) – the principal instrument for monitoring project performance on an ongoing basis. At the time of the evaluation, although sophisticated and responsive, the Regional HIS was unable to provide substantive evaluative data as denominator demographic data had not yet been fully entered. This HIS has developed a sound operational base and has solicited the support of the new Project Manager who is an experienced epidemiologist, with both a provincial HIS and a national Program Management background. This partnership will be mutually beneficial and data sets from the HIS should greatly enhance the QRI. The project participated in both the EPI and IMCI Evaluations conducted during 2000 and 2001 respectively. While valuable data were generated much of it could not be disaggregated to reflect project specific activity areas and target populations could not be accurately defined. Ndwedwe District Child Survival Project Final Evaluation 2001 53 The project has had to rely on surveys, particularly that conducted in partnership with MACRO during 2000 and the KPC of 2001 for the Final Evaluation. In addition a number of qualitative studies were conducted that also informed the project with evaluative data. The system to measure progress towards project targets was effective in that reliable data was available to allow a critical assessment of virtually all project goals and specific targets. The QRI informed the decision to suspend formal training of TBAs (3 of 4 planned trainings were conducted) when the decline in home births became apparent. Project funds were re￾allocated to the CHC initiative. The QRI does not adequately reflect activity at community level where the project is most active and the NDCSP has been requested by the DOH to develop a community based information system. It is intended that this system will be intimately linked to the HIS but will be structured to be the operational planning instrument of the DHS. The project is well positioned to achieve this objective. The qualitative studies conducted during Phase 1 provided very valuable information that will significantly influence project activities in key areas during the extended period. These include: 1) MCDI’s study which confirmed the cost effectiveness of the rapid HIV test, suggesting to the DOH that its more widespread use should be pursued; 2) the Institutional Strengths Analysis conducted among local partner organizations of the NDCSP which identified areas in which the project can offer institutional strengthening support; 3) the Health Facility Survey which helped to identify operational weaknesses that need to be addressed, and 4) the Gaps Analysis which demonstrated its value as a BCC assessment tool. The project staff are very proficient with data gathering and data use in operational planning and project monitoring. The additional epidemiological skills brought to the project by the new Project Manager will greatly enhance this process. It was apparent during the evaluation that there is a high level of awareness among DOH and partner principals of the achievements of the project and that there are also high expectations of the NDCSP in many operational areas, especially informatics. The development of monitoring and impact assessment tools for the CHWP and their systematic introduction should be among the main contributions of the project during its extended phase towards consolidation of the DHS. 8. Technical and Administrative Support The project receives regular backstopping and technical support and supervision from the Home Office Child Survival Support Team (CSST), and in particular the backstopping officers assigned to project. As part of its efforts to ensure the appropriateness and adequacy of its backstopping efforts, the CSST convenes at least monthly to discuss project activities and provide feedback to the field. In addition, the home office backstopping officers assigned to the project maintains frequent telephone and email contact with the Project Manager. These contacts are useful in providing assistance regarding project administrative and technical issues that arise. Home office technical support is provided by a host of channels to include field visits by home office technical staff, forwarding technical updates from various sources, e.g., CORE, BASICS, Ndwedwe District Child Survival Project Final Evaluation 2001 54 PAHO, UNICEF, etc., to the field staff and securing technical guidance from the CSTS. Regular updates of website contacts are provided as well. The headquarters office has mobilized technical assistance from several individuals with technical expertise relevant to the project. 1. Consultant Waverly Rennie assisted the field staff with development of the DIP in February 1998. 2. Consultant Bonnie Kittle visited the project in order to discuss a potential collaboration between the NDCSP and the Equity Project in the Eastern Cape Province. 3. Dr Barbara Parker assisted the staff in the design of qualitative research instruments and supervised the implementation of related focus groups and semi-structured interview studies. She also provided technical support for proposal development and DIP preparation, as well as the development and implementation of Gaps Analysis and wealth ranking exercises. 4. Wendy Ravano, External Evaluator for the Mid-Term Evaluation (conducted December 1999). 5. Consultant Pamela Putney, midwifery specialist, visited the project in 1996 to work with the project’s nurse/midwife on improving the TBA training component of the project. The consultant reviewed the TBA training curriculum and offered suggestions for revisions that would make it consistent with the Gold Standards for Maternity Care. She also introduced the “Handful of Danger Signs in Newborns” to the project. 6. Dr Richard Stokes prepared an assessment instrument for practicum training for the Perinatal Education Programme (PEP). He also reviewed maternity care services at the clinic level and made recommendations for changes in the provincial maternal health card. 7. PATH’s Dr Milton Tam and Ian Buchanan visited the NDCSP to facilitate the launch of rapid testing for STIs and HIV in health institutions serving Ndwedwe District. They trained hospital and clinic staff in use of the tests and developed a testing algorithm appropriate to the District. Dr Tam made a subsequent follow-up visit to provide continuing support and advice. 8. In 2000, the NDCSP worked with MACRO to pilot the new HIV section of the KPC Module. 9. In March 2000, Dr. Christopher Schwabe, MCDI’s Senior Health Economist, conducted a study to evaluate the cost-effectiveness of the new 2-step rapid HIV testing procedure being utilized by the project. Additional cost-effectiveness analyses will be critical during Phase II of the project to identify activities and strategies to ensure the financial sustainability of the project. See Attachment 4a for a summary of this report. 10. In 2001, Gita Gidwani, an independent consultant, was hired to conduct surveys of Ndwedwe facilities (both hospitals and clinics). 11. During July and August 2001, Chris Moore, a medical student at Yale University, conducted a qualitative study of pediatric pneumonia and health beliefs. Specifically, the research was undertaken to attempt to understand caregivers’ concepts of illness, appropriate care, pneumonia, and the personal response to the local HIV/AIDS epidemic. See Attachment 4c for a copy of this report. 12. June Kelly, a consultant with the University of Natal’s Child Development Programme, conducted an external evaluation of DramAide’s activities in October 2001. See Attachment 4b for a summary of this evaluation. Ndwedwe District Child Survival Project Final Evaluation 2001 55 In addition, several field visits have been made by the International Division Director and the Project Officer backstopping the project to provide both supervision and technical assistance. The evaluation team identified the following needs for future technical assistance: · Family support methodologies – to develop strategies to support vulnerable households · Children’s emotional needs – supporting OVCs, CABA and children as care-givers. · BCC methodologies – dealing with entrenched risk behavior. · Resuscitation and management of the ‘at-risk’ newborn. MCDI is currently undertaking an Institutional Strengthening assessment which includes a focus on how the organization can better plan for responding to the technical assistance needs of its field programs. Developing strategies for responding to needs identified, in combination with regular and frequent communications via email and phone, and in person will contribute to a better understanding of the needs of the NDCSP team in the field. 9. Management Lessons Learned A number of important management lessons have been learned which include: 1. The need for an inclusive collegial management style that involves all team members in key decision making, especially in those intervention areas with cross-cutting implications. This implies that opinions of all team members are sought and respected, especially when decisions made impact on their area of responsibility. Team members consistently complained that the management style of the out-going Project Manager was not sufficiently inclusive and that their opinions were not given adequate weight in the management process. The major example cited referred to the delay occasioned by the Project Manager’s reluctance to address the concerns expressed by team members about the conduct of their former colleague as Community Organizer. This delay had a significant negative impact on project performance in a key area and the team felt that the prevailing reclusive management style contributed to the problem. Another example cited was the reluctance of the Project Manager to be persuaded of the need for and value of engaging supervisors for the HBCVs who had been deployed in the District without support and supervision. The decision to engage the Supervisors, who are now regarded as key resources within the project team, was made by Home Office on the recommendation of the Training Coordinator. It is the opinion of the evaluator that the high levels of morale and team-spirit that was evident during the evaluation was derived in large measure from the successful manner in which the project team negotiated the difficult period following the departure of the first project manager. This was a stressful but productive period during which the KPC was completed, the project was re-sited in new accommodation and preparations were made for the final evaluation. All team members were intimately involved in this whole process, to their individual and collective benefit. They all deserve much credit for this effort, especially the project’s Nurse Training Supervisor who deputized as Project Manager during this time. Ndwedwe District Child Survival Project Final Evaluation 2001 56 2. The need to assign authority and responsibility to individual team members for core activities and in liaison with partners. The team felt very enthusiastic about this modus operandi and wished that it had been employed in project management to date. In the newly developed workplan a team member has been identified as the lead person for each core activity and a project contact has been identified for each partner of the NDCSP, both current and projected. Apart from the contribution that this process is likely to bring to the project, experiential benefit will also accrue to individual team members. Ndwedwe District Child Survival Project Final Evaluation 2001 57 D. OTHER ISSUES IDENTIFIED BY THE TEAM Key issues identified by the team include the following: 1. More Project input is needed when externally-led studies are being planned. The project team felt that they should be more involved when external led studies were being planned. They felt that it was unfair to expect them to ‘field’ a study for any purpose with which they were not familiar or in whose planning they had not participated. These sentiments are totally reasonable since any study has the potential for valuable learning experience for the project and individual team members, as well as the potential for negative displacement impact on a busy project. 2. Skills acquisition and training opportunities facilitated through the project that would enhance employability afterwards. All team members raised concern in this regard. It is essential that every opportunity for skills transference within project activities and initiatives be used to benefit team members. Such opportunities are legion in a project like the NDCSP and include methodological skills in areas as diverse as: communication, counseling, training, monitoring, supervision, assessment, evaluation, negotiation, arbitration, budgeting, accounting, reporting, management, project development, proposal writing, etc. The project management should give due consideration to the aspirations of project staff in this important initiative and facilitate team members where possible to avail of such opportunities. Core training towards staff development has been discussed earlier. 3. Enhancing the profile of the project in the community. Team members felt that the project would benefit from having an established operating base in the community. The project team is entirely based in Durban, some 50-80 km removed from its base of operations at present and it is felt that a higher community profile would be generated if a center of activity within the project operational area were established attached to one or more clinics or health centers. The Province’s Superintendent General for Health agrees with this sentiment and is willing to facilitate this development. Significant cost benefits may also accrue to the project. Ndwedwe District Child Survival Project Final Evaluation 2001 58 E. CONCLUSIONS AND RECOMMENDATIONS 1. Conclusions 1. This evaluation fully endorses the decision of USAID to extend the NDCSP for a further four years, based on the success of the project to date in terms of design, implementation and outcomes. The Provincial DOH has high expectations of the extended Project and has specified some key areas for particular emphasis. 2. All core Project activities were assiduously pursued, substantial progress is evident in all key areas and high levels of success were attained with all Project targets. 3. Commendable success is evident in the manner in which the HIV/AIDS components were merged with the primary components of the project and the entire project team is clearly focused on the challenges presented in merging the two components. 4. The NDCSP has made a large contribution to the creation of a functional DHS through capacity building for health at community level. The essential organizational structures have been established and community based health cadres, paid and voluntary, are active. Although both the structures and cadres are still fledgling, it is the Provincial intent that they should be fully consolidated and the NDCSP has been entrusted with a major role in this process by the DHSMT during Phase II. 5. The DOH has indicated that the NDCSP will be entrusted with the development and introduction of systematic procedures and protocols for the operation of CHCs. The CHCs will be trained to monitor and supervise community workers and community based health activities and to manage community resources , including community funds. It is the evaluator's opinion that the project teamproject team is fully capable of delivering this objective with appropriate technical support. 6. In contacts with the DOH and other principal partners it is evident that the NDCSP is highly regarded and that good interpersonal and professional relationships have been established. The planned on-going contribution of the project to the DHSMT during Phase II, across a spectrum of activities, is universally welcomed. 7. A number of key Provincial DOH developments under-pin the project initiatives and enhance the likelihood of sustainability. These include: a) the Provincial CHWP Policy, b) the recent appointment of a Deputy Director for DHS Support to a post that had been vacant and c) the stated commitment of the Superintendent-General to the DHS initiative. 8. An experienced and dedicated project team is in place and the project is well positioned to expand its role as a principal partner of the DHSMT by helping to consolidate the community base of the DHS; by lending operational support to a number of key Programs – HIV/AIDS; Maternal and Child Health and Youth, Sexuality and Women’s Health in particular and by helping to improve health worker performance through supervision of the PEP and IMCI training initiatives. 9. It is in the area of community empowerment for health that the NDCSP can make a major sustainable contribution through the introduction of community oriented health training. In this initiative training modules developed by the NDCSP can be merged with the Bergville CSP developed IMCI Community Module and the standard HIV/AIDS training module to create a comprehensive community health training package that could become the Provincial standard. Ndwedwe District Child Survival Project Final Evaluation 2001 59 10. The project team includes three highly experienced and dedicated health professionals who are well capable of leading the initiative outlined above. The senior of these has been with the project from the outset and a clearly defined role in health worker training has already been agreed with the DOH Program Principals for this officer during Phase II. The other two officers were recruited as supervisors of HBCVs in the District as a key element of the HIV/AIDS component. However the latter role by itself must be considered an unsustainable extravagance and an extended role needs to be developed to maximize the complementary skills that they bring to the project. They are eminently suited to develop and implement the community training initiative being planned for the District. 11. The two Community Organizers came to the project without any experience (one was identified through an internship placement with the project as a Sociology undergraduate) but have grown in confidence and experience and are now well versed in their roles and are respected for their contribution to the community empowerment process. As CHC mentor and Social Welfare intermediary, respectively, their contribution in Phase II can be substantial. 12. The replacement Project Manager comes with a substantial national program management background and is an experienced epidemiologist in the South African context. The level of engagement with DOH and other partners achieved in the brief time he has been with the NDCSP and the inclusive and responsive interaction with Team members augurs well for the continued successful development of the project. 2. Recommendations Project Location/Orientation 1. As indicated by the DOH the NDCSP should continue to operate for the present within its existing boundaries, even though a part of the southern area no longer falls within the newly established Ndwedwe sub-District of the Ilembe District (KZ29). At the same time it has been requested to extend its field of operations to the newly incorporated area of the sub-District to the northwest, which includes a District Hospital and 3 clinics. 2. The core objectives of the NDCSP should remain the same with prime emphasis on child survival and reproductive health initiatives but with re-defined strategies to take account of the escalating HIV/AIDS epidemic. 3. The principal focus of the NDCSP in Phase II should be on consolidating the District Health System at community level by developing methodologies for specifying, quantifying and certifying the operations of the various structures and cadres operating at community level. In consultation with its partners in the DHSMT, the NDCSP should develop and introduce within the District, protocols and procedures appropriate to this process. 4. The Provincial HIS intends that a community database will be established as a key component of a functional HIS. Such a database will allow the geographic profile of the CHWP to be readily determined and to match the distribution, content and capacity of community and clinic-based health services with the socio-economic, demographic and epidemiological profile of the community. This will allow the DHSMT to monitor the DHS in terms of equity, productivity, efficiency etc. The Provincial HIS expects the Ndwedwe District Child Survival Project Final Evaluation 2001 60 NDCSP to play a lead role in this initiative in partnership with the DHSMT and the Provincial HIS. 5. The Superintendent General has a vision of a “community-data circuit” that would underpin the operations and management of the DHS. This information system would include demographic data (e.g. household size and structure), sociological data (e.g. PLWAs, OVCs), epidemiological data (e.g. sentinel disease surveillance), vital data (e.g. unregistered births and deaths) and other key related data (e.g. maternal deaths) that could be used to generate a community health index . It is recommended that the project, in co-operation with the DHSMT and the Provincial HMIS, should develop and pilot the instrument(s) for this data system in the District, as an essential tool for management, monitoring and evaluation within the DHS. 6. In developing and delivering health messages in the community the NDCSP should use the latest available tools e.g. the IMCI ‘community module’ now being developed by the Bergville CSP and new BCC methods. 7. In its continuing role in strengthening Health Worker performance the NDCSP should foster linkages between the health services and the community in every way possible so that a solid CHC-CHW-Clinic axis is established at the core of the DHS. 8. All future health worker training programs should be accompanied by a parallel program for CHWs, HBCVs and TBAs to ensure that communities can be informed of any intended changes in health worker attitudes and performance as an outcome of the training. 9. All available evidence suggests that the escalating HIV/AIDS epidemic will have a dramatic impact on child survival and quality of life for survivors. The vulnerability of children is further aggravated by reports of very high levels of child abuse in the District. The Project should develop strategies for quantifying and addressing these major negative influences on child survival and well-being. 10. Young women are especially vulnerable to HIV infection, carrying at least five times the risk of infection as their male counterparts. Their vulnerability is further increased by the accelerated disease process associated with pregnancy. The Project should help to address this major problem which further impacts on the status of OVCs through the design and conduct of KPC studies and the design and implementation of appropriate BCC strategies. 11. The NDCSP offices are located in Durban, outside the Ndwedwe District and therefore removed from its sphere of operations, with the newly incorporated areas even further distant. In order to establish a higher community profile for the project it is recommended that an operations base should be established within the District. The newly constructed Ndwedwe Health Center would be an ideal location, given that it is sited at the heart of the District and now incorporates a ‘community facility’. The old Ndwedwe Clinic has been used throughout by the project for meetings and assemblies and the Superintendent General has offered to facilitate the establishment of a dedicated NDCSP office on this site. 12. The DOH should be encouraged to identify local counterparts for each Project staff member in the interest of capacity building and sustainability. Ndwedwe District Child Survival Project Final Evaluation 2001 61 Project Management 1. Responsibility and accountability should be delegated to each Project staff member for project initiatives and activities that fall within their scope of operations and any deviance from agreed action-plans should be cleared with the project Manager or delegated supervisor. 2. All Project staff should be engaged in developing and reviewing work-plans that impact on their areas of activity and responsibility and they should be kept informed of all developments that impact on the project. 3. Home office interventions (e.g. special studies/analyses etc.), which directly impact on Project activities should be devised and implemented in consultation with local Project staff. 4. The Home Office Project Coordinator should have a regular local presence and visit the Project at least once yearly have the opportunity to develop a technical as well as an administrative role in the project. Home Office Use of Lessons Learned The MCDI Home Office uses different strategies to share the lessons learned from its field projects with the broader development community. For example, information on operational research and ethnographic studies is provided to the CORE Group’s database. In addition, MCDI field staff presented their experiences at both the 2000 and 2001 Spring CORE Group meetings. MCDI aims to share its lessons learned with other PVOs implementing child survival or other related health activities as much as possible through presentations and development of materials. Ndwedwe District Child Survival Project Final Evaluation 2001 62 F. RESULTS HIGHLIGHT The NDCSP has had a predominant focus on community mobilization and empowerment for health. Significant gains have been made in establishing the community base of a DHS in the project area of operations. Valuable lessons have been learned along the way that are worthy of mention and may be of interest to other PVOs engaged in similar activities. These include: · That it is essential to determine the principal prevailing authority in the community when new organizational structures with vested authority are being proposed. In the NDCSP area of operations the traditional Tribal Authorities are being challenged by newly created Local Government Councils and valuable time was lost and confusion created by not establishing the hierarchy of authority from the outset. · When multiple agencies are engaging with communities simultaneously it is imperative to define and agree ‘rules of engagement’ from the outset. Otherwise there is great risk of confusion and alienation which is likely to retard or even undermine the process. · Indigenous community volunteers, in this instance Traditional Birth Attendants (TBAs) and Home Based Care Volunteers (HBCVs), who had been selectively trained and/or supervised by the Project proved very useful intermediaries in fostering the creation of Community Health Committees and facilitating Health Action Plan development at community level. · Those volunteers with a tradition of service in the community – the TBAs were more acceptable to the community as purveyors of home care than recently recruited indigenous volunteers like the HBCVs. This is an important consideration when addressing the stigmatization associated with HIV/AIDS. · In focus group discussions the TBAs requested endorsement by the DOH as their reward for service whereas the HBCVs requested monetary rewards, most probably reflection the well established tradition of voluntary service within the TBA cadre. · The TBAs are credited with substantial achievements in promoting health in the communities where they operate – increased ante-natal attendance, increased health facility deliveries, eradication of NNT (secondary to ANC attendance). · Although the TBAs who were trained by the Project in safe home birth practices, they very willingly assumed other roles when their primary role diminished due largely to the risks associated with the escalating HIV epidemic but also due to the progressive decline in home births. Their future role in the Project area will be as promoters of safe motherhood rather than as providers of care during home births. · The use of a 2-step rapid HIV test can be a cost-effective intervention (it was found to be 13.2 times more cost-effective than the 3-step procedure implemented by the DOH). Ndwedwe District Child Survival Project Final Evaluation 2001 63 ATTACHMENTS Ndwedwe District Child Survival Project Final Evaluation 2001 64 1. Final Evaluation Team Members External Evaluator and Team Leader Edmund McGrath, MB BCh FRCPI DCH DTM&H, Evaluation Team Leader Ndwedwe District Child Survival Project Staff Farshid Meidany, MD, MPH, Project Manager Thuli Ngidi, Nurse Training Supervisor Zanele Buthelezi, Social Worker Christopher Mohatsela, Community Outreach Organizer Thoko Radebe, AIDS Nursing Specialist Esme’ Cakata, Home Based Care Training/Counseling Assistant David Patterson, MPH, Former Project Manager, Ndwedwe District Child Survival Project Department of Health Mr K Mungwe, Ndwedwe District Coordinator Dr AKM Hoque, Epidemiologist KZN Department of Health Mrs Dolly Nyasulu, Coordinator Provincial MCH Mrs Mandisa Dlamini, Regional Coordinator HIV/AIDS Medical Care Development International Headquarters Alyssa Wigton, Medical Care Development International, Washington, DC Ndwedwe District Child Survival Project Final Evaluation 2001 65 2. Assessment Methodology The evaluation process used a participatory methodology which involved the project team, representatives of the DHMT and key DOH Programs, local partners, and a MCDI Home Office representative. Relevant documents produced or commissioned by the NDCSP during the lifetime of the project as well as numerous other relevant reports were reviewed. These include: the original DIP, the HIV/AIDS sub-component DIP, KPC study reports, various assessment reports and reports of special studies, as well as DOH policy documents, all of which are listed below. Detailed discussions were held with all Project Team members and site visits were made to all health facilities in the project area in the company of team members. In-depth interviews were held with all the persons named, service operations were reviewed, and relationships with the NDCSP explored. Focus Group Discussions were held with TBAs, HBCVs and one indigenous faith-based group, some of whom were members of CHCs. Opinions expressed and service statistics presented were corroborated where possible in discussion with other opinion leaders and with available data sets. The conclusions and recommendations of the evaluation were discussed with, and guidance sought from the Superintendent General of the Provincial DOH. Documents Reviewed NDCSP - Detailed Implementation Plan, March 1998. NDCSP – Mid-term Evaluation Report, NDCSP – Assessment Reports, March 2001. NDCSP – Detailed Implementation Plan, HIV/AIDS Intervention, Sept. 2000. KwaZulu- Natal, Policy Document on Community Health Workers. DOH 1999. CHWP Phase 1 Evaluation Report, 2000 CHWP Phase II Evaluation Report, 2001 DramAidE Evaluation Report, 2001. IMCI – Health Facility Report, KZN Province, 2001. (DOH, WHO, Unicef.) Third Interim Report, 2000 (National Committee on Confidential Enquiry into Maternal Deaths) KPC 2000 Report. Itinerary and activity schedule Day 1 Travel to Durban. Days 2-3 Briefing with the Project Training Coordinator ( recent Acting Project Manager) Meetings with Project Team. Document review. Days 4-10 Meeting with DramAidE principals. Meeting with TREE principals. Meeting with Valley Trust principals. Meeting with Diakonia principals. Discussions with Project Team and document review. Visit to Osindisweni Hospital. Meeting with District Coordinator and DHMT members. Visit to QADI clinic. Discussions with Project Team and MCDI Home Office representative. Ndwedwe District Child Survival Project Final Evaluation 2001 66 Visit to Montebello Hospital. Visit to Osindesweni Hospital and Oakford Clinic. Visit to Nuyuswa Clinic. Focus group discussion with HBCVs. Visit to Ndwedwe Clinic. Focus group discussion with TBAs. Meeting with IMCI/EPI Program Manager. Meeting with PEP/Maternal Health Program Manager. Meeting with former Project Manager. Meeting with Superintendent General, Provincial DOH. Meeting with HIV/AIDS Program Director. Meeting with Youth and Reproductive Health Program Manager. Visit to Oakford Clinic and Osindisweni Hospital De-briefing with District Coordinator. Visit to Ndwedwe Clinic. Focus group discussion with Faith-based group. Visit to Wosiana and Mafamasi Clinics. Working with Project Team – debriefing and work-plan review. List of Persons Interviewed and Contacted NAME TITLE Prof. Green-Thompson Head: KZN Dept. of Health Mr K Mungwe Ndwedwe District Coordinator Dr Chinasamy Medical Superintendent – Osindisweni Hospital Mrs W. Ndlovu Matron – Osindisweni Hospital Dr AKM Hoque Epidemiologist KZN Department of Health Mr Sipiwo Yose Regional Health Information Manager, DOH Mrs Dolly Nyasulu Coordinator Provincial M.C.H Mrs Esther Snyman Assistant Regional Director Adolescent & Child Health Mrs Phumla Nkosi Assistant Regional Director EPI/MCI Mrs Mandisa Dlamini Regional Coordinator-HIV/AIDS Sister Leona In charge – Oakford Clinic Sr. Virginia Chili Matron in charge-Ndwedwe Clinic Sr. Kubheka Deputy Matron –Qadi Clinic Sr Mthethwa Deputy Matron- Thafamasi Clinic Sr. Thokozani Mbatha Matron in charge- Nyuswa Clinic Sr Nke Ngubane Matron in charge-Wosiyana Clinic Dr Win Zoe Superintendent –Montebello Hospital Sr Busi Coka Matron-Montebello Hospital Dr. K Wimble Director: Valley Trust Mrs Pam Picken Manager-TREE Keith Wimble Director-Valley Trust Mr Mkhonzeni Gumede Manager-DramAIDE Group of TBAs Group of HBCVs Group of Church Volunteers Rev Mike Vorster Diakonia-Programme manager Ndwedwe District Child Survival Project Final Evaluation 2001 67 NAME TITLE Mrs Thuli Thabethe Diakonia- AIDS Network Coordinator Ms Nomusa Faith Based – Ndwedwe Mrs Khumalo Faith Based – Ndwedwe Rev. Ximba Faith Based – Ndwedwe Mr Mkhonzeni Gumede DramAide-Manager Ms Thenjiwe Manana DramAide-Project Coordinator Mr Sthembiso Nzama Valley Trust-Community Developer Dr Farshid Meidany NDCSP Project Manager Ms Thuli Ngidi NDCSP Nurse Training Supervisor Ms Zanele Buthelezi NDCSP Social Worker Mr Christopher Mohatsela NDCSP Community Outreach Organizer Ms Thoko Radebe NDCSP AIDS Nursing Specialist Ms Esme’ Cakata NDCSP Home Based Care Training/Counseling Assistant Mr David Patterson Former Project Manager, NDCSP Ndwedwe District Child Survival Project Final Evaluation 2001 68 4. Special Reports The following reports are included as Attachments 4a, 4b, and 4c: a. Summary of Rapid Test Cost-Effectiveness Study b. Summary of the DramAidE External Evaluation c. Full Report: Investigating HIV/AIDS-impacted Health Beliefs and Behaviors Regarding Pediatric Pneumonia Annex 4a: Cost-Effectiveness of A Decentralized Dipstick/Dot Immunoassay Approach to the Detection of Antibodies to HIV 1 and 2 Versus the Current Centralized 3-Test ELISA Approach Conducted by Christopher Schwabe, PhD March 2000 Study Summary The purpose of this study was to evaluate the cost-effectiveness of the new 2-step rapid HIV testing procedure developed by PATH that could be implemented at a decentralized District Hospital level by existing medical cadres relative to the prevailing 3-step enzyme-linked immunosorbent assay designed for batch testing at a centralized Regional Hospital using a costly and sophisticated immunoassay analyzer. METHODS Due to time limitations, the cost-effectiveness assessment examined only the provider￾side costs of HIV testing. No attempt was made to evaluate the costs to the pregnant women seeking to establish their HIV status. The approach focused exclusively on the short run costs and effectiveness of HIV screening and as such did not consider the much more substantial future flow of benefits and costs that this type of early rapid screening intervention implies. RESULTS The mean cost for the rapid test is US$17.01 per pregnant woman attending antenatal clinics who is correctly tested compared with US$53.96 for the current 3-Step procedure. The cost of correctly testing and counseling pregnant women using the current procedure is roughly 13.2 times greater than it would be under a decentralized rapid 2-test procedure. DISCUSSION Future work will need to focus on the demand for testing, the quality of pre-test and post￾test counseling, and the capacity to efficiently and accurately test patients in a sustainable way. In addition, there is a need to evaluate the potential long term effects of testing and counseling on prevention and clinical assessment and management since these may well influence the decision to choose between these two testing approaches. Annex 4b: Summary of the Evaluation Report of the DramAidE Intervention in the Ndwedwe Child Survival Project Conducted by June Kelly, Child Development Unit for Medical Care Development International (MCDI) South Africa October 2001 In spite of there having been a wealth of information about HIV/AIDS in South Africa for some time, the infection rate of this virus in our country, and in Kwazulu-Natal in particular, continues to climb, especially in the youth. It appears that knowledge alone does not change attitudes.(Leclerc - Madlala 2000; Leclerc-Madlala 1999). The DramAidE AIDS education method goes beyond information, to active engagement of people in a creative, interactive experience of the impact of HIV/AIDS at a personal and community level. This approach was adopted by DramAidE in the MCDI initiated intervention project directed at target schools and the wider community, in the rural Ndwedwe region in Kwazulu-Natal. The intervention used drama, song and dance to communicate messages about AIDS to the youth. It began with introductory plays containing these messages at a number of schools, performed by an out-of￾school youth group. This led to the formation of Health Clubs at these schools. Learners in the Clubs attended workshops at their schools, run by DramAidE trainers, and facilitated by teachers. In these they explored their own attitudes towards sexuality, and deepened their knowledge about HIV/AIDS, and the care of family members who had AIDS. The overall aim of the intervention was to reduce the stigma of AIDS and to lead the youth to adopt sexual practices that would avoid transmission of the virus. In addition the project was intended to be self-sustaining, cascading from the initial Clubs to other schools and into the community. The evaluation process used both quantitative and qualitative evaluative tools. Qualitative techniques included an assessment of background documentation, focus groups with youth and community bodies, and semi-structured interviews with teachers. A quantitative evaluation of the project’s impact on schools with active Health Clubs, in comparison to non-intervention schools in the area, was also conducted. The evaluation concluded that the intervention has successfully achieved its major goal of attitudinal and behaviour change in the target school youth groups. The quantitative assessment showed a significant increase in both knowledge and practice regarding the apppopriate sexual behaviour to avoid transmission of HIV/AIDS, in intervention schools. The out-of-school youth group also benefitted significantly. Teachers gained confidence in their ability to discuss the virus with learners, learned to not discriminate against those had the virus, and about caring for those who were infected and affected by it. The initiative has not being spread as effectively as hoped to other schools and especially, to other community groups. The limited effort made by school principals to inform other relevant community bodies about their Clubs was one reason for this. Another was that Club activities were given low priority on the school time-table, although much of the material covered was compatible with that of the subject ‘Guidance’. Learner time and funding constraints, and the distances between many rural schools were other factor. Teachers and Club members acknowledged these barriers to spreading the AIDS messages, in the evaluation. The limited success of cascading is a reality that occurs in many of rural interventions similar to the one under discussion. Rural communities frequently lack the cohesion and infrastructure to support the cascading idea. Given the obstacles in Clubs spreading the project among other learners at their school (time, peer and staff negative attitudes), the evidence in this evaluation suggests that at the school level, this initiative has been remarkably successful. The following suggestions are made at the conclusion of the evaluation, for a follow-up phase of the project: ß DramAidE’s intended new thrusts in a follow-up phase are in line with the need for more peer leaders to cascade the project, and to counteract the impact of AIDS on the school community. However, a continued effort must be made to take the initial AIDS messages to new schools and the wider community, as there remains a high level of stigma attached to HIV/AIDS, and ignorance about the dangers of contracting the virus by unprotected sex, amongst the youth. ß Principals, senior teachers and school governing bodies, in addition to local education authorities, must be included in future phases of the project to expedite the spread of AIDS messages from Clubs to other schools and community groups. ß School authorities should take responsibility for introducing their school Health Clubs to local community bodies, such as Health Committees. ß Parents need to be included in some Club level workshops, to gain better acceptance of youth ideas about AIDS by their parents and families. ß Greater use should be made of out-of-school youth to spread the AIDS message. ß To sustain the impact of the intervention, the culture of dependence on funders, as revealed by learner and teacher input in the evaluation, should be reduced. Some workshops exploring entrepreneurship by learners, teachers and involved community, where issues such as fund raising, creating teaching tools to pass on messages etc, can be developed. ß Those running the intervention must ensure that regular input from the regional AIDS coordinator’s office and the regional health department is received and supported. Annex 4c: Investigating HIV/AIDS-impacted health beliefs and behaviors of Zulu caregivers of Ndwedwe District, South Africa, regarding pediatric pneumonia. C.A. Moore, MPH1 , 2002, R. Barbour2 , D.W. Patterson2,3, T. Ngidi3 , and N. Groce4 1MPH Program, Department of Epidemiology and Public Health, Yale University School of Medicine, New Haven CT. 2Medical Care Development International, Washington DC. 3Ndwedwe District Child Survival Project, Medical Care Development International, Durban South Africa. 4Division of Global Health, Department of Epidemiology and Public Health, Yale University School of Medicine, New Haven CT. Interim Report Prepared by Christopher Moore, for Medical Care Development International, Washington, DC December, 2001 PROJECT ABSTRACT:In Medical Care Development International’s (MCDI) 2000 Annual Report, the Ndwedwe District Child Survival Project (NDCSP) reported that a core segment of mothers do not seek care at local health care facilities for children with symptoms of pediatric pneumonia, i.e., coughing and rapid, difficult breathing. Given the documented associations between HIV-seropositivity and increased pneumonia-related mortality in children, one may question the social impact of the larger, current HIV/AIDS epidemic on Zulu caregivers’ care-seeking behaviors for children presenting symptoms of pediatric pneumonia. The Behavioral Model of Health Services Use may serve a framework for understanding how the changing social environment may impact caregivers’ choices of treatment, i.e., herbal remedies, traditional healers, and/or modern health facilities. 91 semi-structured interviews, along with one focus group, were conducted with the help of Zulu translators, to ascertain the NDCSP caregivers’ concepts of illness and appropriate care. The project also aims to elucidate how the local HIV/AIDS epidemic has impacted how caregivers seek treatment for children with symptoms of pediatric pneumonia. Results from this research may better enable NDCSP and other health organizations/departments to better serve local communities through tailoring their interventions and campaigns to the changing social environment, impacted by the HIV/AIDS epidemic. A full copy of the Final Report is available upon request from MCDI.