Africare The Gambella Child Survival Project Final Evaluation Report Gambella People’s National Regional State, Ethiopia Regional Strengthening of Child Survival Interventions CA No. FAO-A-00-00-00031-00 October 1, 2000-September 30, 2004 Date of Submission: December 31, 2004 Written and Edited by: Kassie Gebremariam, Evaluator, External Consultant Yilma Tekeleselassie, Former Africare/Ethiopia Public Health Manager Yitayew Mengesha, Former Africare/Ethiopia Field Project Coordinator Haile Wubneh, MPH, Former Africare/Ethiopia Senior Program Manager Mezgebu Chaka, Former Africare/Ethiopia Child Survival Project Manager Peter M. Persell, Africare Country Representative, Ethiopia Kendra K. Blackett-Dibinga, MPH, MPP, Health Program Manager, Africare/Washington ii Acronyms AIDS Acquired Immunodeficiency Syndrome ARI Acute Respiratory Infection BCG Bacillus Calmette-Guerin BCC Behavior Change Communication BHR Bureau of Humanitarian Response CAG Cash Assistance to the Government CSA Central Statistical Authority CS Child Survival CSP Child Survival Project CSTS Child Survival Technical Support Project CRDA Christian Relief & Development Association CHAs Community Health Agents CHWs Community Health Workers CDD Control of Diarrhea Diseases CBR Crude Birth Rate DIP Detailed Implementation Plan DPT Diphtheria, Pertussis, Tetanus DPPO Disaster Prevention and Preparedness Office EFY Ethiopian Fiscal Year EPI Expanded Program on Immunization FMOH Federal Ministry Of Health FPC Field Project Coordinator FY Fiscal Year GCSP Gambella Child Survival Project GRHB Gambella Regional Health Bureau GOE Government of Ethiopia HC Health Center HFA Health Facility Assessment HP Health Post HS Health Station HIV Human Immunodeficiency Virus iii IMR Infant Mortality Rate IEC Information, Education, Communication IMCI Integrated Management of Childhood Illness KPC Knowledge Practices and Coverage MCH Maternal and Child Health MMR Maternal Mortality Rate MOH Ministry Of Health NIDs National Immunization Days NGOs Nongovernmental Organizations NPW Non-Pregnant Women ORS Oral Rehydration Solution ORT Oral Rehydration Therapy/Treatment PA Peasant Association PAs Peasant Associations PEI Polio Eradication Campaign PW Pregnant Women PHC Primary Health Care PVC Private Voluntary Cooperation PVO Private Voluntary Organization RHB Regional Health Bureau STI Sexually Transmitted Infection TT Tetanus Toxoid TBAs Traditional Birth Attendants TNA Training Needs Assessment UFMR Under Five Mortality Rate UNICEF United Nations Children’s Fund UNDP United Nations Development Program UNHCR United Nations High Commission for Refugees USAID United States Agency for International Development VHC Village Health Committee WRA Women of Reproductive Age WAB Women’s Affairs Bureau iv WHO Woreda Health Office WHO World Health Organization ZHD Zonal Health Department v Table of Contents A. Summary.................................................................................................................................... 1 1. Program Description and Objectives ...................................................................................... 1 2. Program Accomplishments..................................................................................................... 1 3. Highlights of Comparison of Baseline to Final Evaluation.................................................... 2 4. Priority Conclusions of this Evaluation.................................................................................. 2 B. Assessment of Results and Impact of the Program.................................................................... 3 1. Results Summary Chart .......................................................................................................... 3 2. Results: Technical Approach.................................................................................................. 3 C. Cross-cutting Approaches ........................................................................................................ 11 1. Capacity Building at Community Level.............................................................................. 11 2. Capacity Building of Local Partner Organizations.............................................................. 14 3. Capacity Building of Health System.................................................................................... 14 4. Strengthening Health Worker Performance ......................................................................... 15 5. Training................................................................................................................................ 15 6. Sustainability Strategy ......................................................................................................... 15 D. Program Management .............................................................................................................. 17 1. Planning ................................................................................................................................ 17 2. Staff Training ........................................................................................................................ 17 3. Supervision of Program Staff................................................................................................ 17 4. Human Resources and Staff Management............................................................................ 17 5. Financial Management .......................................................................................................... 18 6. Logistics................................................................................................................................ 18 7. Information Management...................................................................................................... 18 8. Technical and Administrative Support ................................................................................. 19 9. Management Lessons Learned.............................................................................................. 19 E. Other Issues Identified by the Team......................................................................................... 19 F. Conclusions and Recommendations ......................................................................................... 20 Attachments A. Evaluation Team Members and Their Titles B. Evaluation Assessment Methodology C. List of Persons Interviewed and Contacted D. Special Reports E. Project Data Sheet Form F. Evaluator’s Notes 1 A. Summary 1. Program Description and Objectives Africare’s Gambella Child Survival Project was implemented from October 1, 2000 to September 30, 2004) in 7 woredas (districts) of southwestern Ethiopia’s Gambella Region: Gambella, Itang, Gog, Jor, Abobo, Godere and Dima. Two woredas were not targeted due to security concerns. The goals of the project were: 1) to reduce infant mortality caused by diarrheal and vaccine-preventable diseases, and 2) to reduce the spread of HIV/AIDS/STIs in the region. The objectives were: 1) to reduce vaccine-preventable diseases and vitamin A deficiency through full immunization and vitamin A supplementation; 2) to reduce diarrheal diseases through improved hygiene and sanitation and improved diarrhea treatment practices; and 3) to increase HIV/AIDS/STI prevention and control knowledge among women and secondary school children (aged 14-18). The project targeted 147,000 people, of which 31,200 were women of reproductive age and 26,800 were children under five years. The project’s HIV/AIDS intervention especially targeted 12,500 youth in public schools. 2. Program Accomplishments ß Provision of full immunization and vitamin A supplementation to children through support to the Regional Health Bureau’s 21 static and 83 outreach sites. Support included training, cold chain equipment and transportation/logistics. The coverage rate for DPT3 attained 46.1% in 2003 (Health Bureau and Project Reports; 50% Per Final KPC Survey). ß Construction of 29 water systems (23 springs, 2 covered hand-dug wells, equipped with hand pumps, one public water distribution point, 2 piped water supply systems for health centers and one water supply system for a public slaughter house). ß Establishment of oral re-hydration therapy corners in public health facilities. ß Creation and support of HIV/AIDS/STI high school clubs and women’s groups (commercial sex workers) with 6,272 benefiting from information/education and condom distribution. ß Strengthened capacity of public health workers in training, case management and supervision skills. Continuous education through courses and workshops provided. ß Strengthened grassroots community capacity through training of community health agents, village health committees and water user committees. ß Introduction of Integrated Management of Childhood Illness (IMCI) to the region. ß Design, production and distribution of IEC/BCC materials including posters and flip charts with trainer guides in three local languages. 2 3. Highlights of Comparison of Baseline to Final Evaluation Table I: Key Highlights Key Indicator Baseline KPC Final KPC Children receiving DPT 3 19.4% 50.0% Women with health card with two or more recorded doses of tetanus toxoid vaccine 19.6% 22.3% Children 0-23 months who were ill with diarrhea during the previous two weeks given ORS 15.8% 38.8% Children 0-23 months who were ill with diarrhea during the previous two weeks given more food or breast milk during recovery 15.7 89.4% Households with access to safe drinking water 28.6% 74.0% Households with safe excreta disposal systems 4.3% 20.7% Women who could name two effective means of preventing transmission of HIV 30.0% 54.0% Women who have used condoms 1.0% 12.6% Important Note: Final KPC survey was conducted only in Godere District due to security problems in all other districts. This particularly affected the high percentage of households with access to safe drinking water, as Africare’s water activities were concentrated in Godere. 4. Priority Conclusions of this Evaluation Africare’s intervention areas were appropriate, very supportive of local institutions and addressed the priority problems of the region. Africare employed three main crosscutting approaches - community mobilization, capacity building, and monitoring and evaluation, which improved the community’s organization and increased its involvement in managing its own health concerns. Africare’s support was very much appreciated by all stakeholders. The project contributed greatly to an improved work ethic in the public health system. The strong partnership relationship between the Africare project, the Regional Health Bureau and the community contributed to significant improved capacity of public health workers, community health agents and the general population. The project’s long term benefits will be extended to other parts of Ethiopia, as health workers trained in this project are transferred to other regions. 3 B. Assessment of Results and Impact of the Program 1. Results Summary Chart Table II: Results Summary Chart Program Objective Indicator Baseline Target Final Increase % fully immunized children before their 1st birthday % children 12-23 months fully immunized 21% 50% 38% Increase % children receiving Regular Vitamin A supplementation % children 6-23 months getting at least 1 high dose of Vitamin A in the last 6 months NA 50% 90% (yr 3) Increase tetanus toxoid coverage among WRA % mothers of children 0-23 months receiving at least 2 TT injections before delivery 16% 40% 22.3% Improve proper case management of child diarrhea % children 0-23 months with diarrhea in the past 2 weeks receiving either ORS or increased breast milk of other fluids 20% (ORS) 5% (same or more breast milk) 14% (same or more fluids) 25% 39.8% 49% 41.8% Promote proper hand washing among mothers % of mothers with children 0-23 months who report washing hands before eating, preparing food, feeding children and after toilet use 49% 60% 54.3%* Increase knowledge/awareness of spread/prevention of HIV/AIDS/STI among mothers and high school students % of respondents knowing 2 ways of reducing risk of transmission of HIV/AIDS/STI 36% (mothers) NA (students) 60% NA(students) 54% 46.8% Composite measure of child health and well being % of children 0-23 months underweight NA 30% NA * Before Eating (91%); After Eating (87.7%); Before Preparing Food (89%); Before Feeding Children (57.3%) 2. Results: Technical Approach a. Project Overview In seven of southwestern Ethiopia’s Gambella Region’s nine districts, the project sought to: increase immunization coverage, control diarrheal diseases (including promotion of exclusive breast feeding and appropriate weaning practices), and prevent HIV/AIDS/STI. The project objectives were: 1) to reduce vaccine-preventable diseases and vitamin A deficiency through full immunization and vitamin A supplementation (40%); 2) to reduce diarrheal diseases through improved hygiene and sanitation and improved diarrhea treatment at home and in health facilities (40%) and 3) to increase HIV/AIDS/STI prevention and control knowledge among men, women and secondary school children (20%). The approaches adopted to achieve these objectives were: 1) system-level capacity strengthening, 2) community organization, education and mobilization and 3) inter-sectoral collaboration. 4 b. Progress by Intervention Area Expanded Program on Immunization Expanded Program on Immunization (EPI) was one of the project’s three main interventions. The objective was to reduce vaccine-preventable diseases and vitamin A deficiency through full immunization and vitamin A supplementation. Static and outreach strategies were adopted. By the project’s third year the following 21 static and 83 outreach sites were operational: Table III: Static & Outreach Sites by Woreda in FY 2003 Woreda Number of static sites Number of outreach sites Gog 6 24 Abobo 4 6 Gambella 5 14 Itang 2 21 Godere 3 15 Dima 1 3 Total 21 83 To increase vaccination activities, Africare supported the maintenance and repair of refrigerators in health facilities and secured a supply of replacement batteries for solar refrigerators through UNICEF and the Federal Ministry of Health. Table IV: Children Under One Year of Age Receiving vaccinations October 1, 2002 September 30, 2003 Woreda Population Under 1 BCG Measles DPT1 DPT2 DPT3 PO PO1 PO2 PO3 Gambella 34,949 1,398 853 588 886 725 612 665 886 735 612 Itang 23,552 942 669 633 751 726 616 306 751 636 485 Abobo 17,447 698 410 258 382 287 270 262 382 303 321 Gog 18,972 759 673 388 506 422 327 273 556 438 384 Godere 40,436 1,617 1,554 1,076 1,584 1,436 1,053 378 1,589 1,493 1,096 Jor 10,890 436 63 61 82 58 61 15 82 58 61 Dima 16,250 650 49 53 118 60 48 24 108 53 33 Total 162,496 6,500 3,971 3,057 4,309 3,714 2,996 1,923 4,354 3,716 2,992 Coverage 61.1% 47% 66.3% 57.1% 46.1% 29.6% 67% 57.2% 46% Table V: EPI trends in FY 2003 (selected antigens) (All nine districts) Quarter Planned Achieved BCG Achieved DPT3 Achieved measles First 812 389 (47.9%) 316 (38.9%) 377 (46.4%) Second 1,623 1,261 (77.7%) 835 (51.4%) 1,216 (74.6%) Third 2,435 2,104 (86.4%) 1,360 (55.85%) 1,334 (54.8%) Fourth 3,246 3,971 (122.3%) 2,996 (92.3%) 3,057 (94.2%) TOTAL 8,116 7,725 (95.2%) 5,507 (67.9%) 5,979 (73.7%) 5 Results achieved progressively increased from quarter to quarter both in absolute figures and rates for BCG and DPT3, while for measles there was a tendency to rise and fall. Overall achievement in FY 2003 was remarkably good for the three antigens. Coverage increased from 2002 from 61.1% to 95.2% for BCG; 46.1% to 67.9% for DPT3; and 46% to 73.7% for Polio. Table VI: Under 1 Year Coverage for October 1, 2002-September 30, 2003 by Woreda and Antigen. (Only Africare-targeted districts) As indicated in the above table, BCG vaccination coverage of 96.1% for Godere, 88.7% for Gog and 71.0% for Itang were more than satisfactory, satisfactory and somewhat satisfactory, respectively. Good results were achieved for the other antigens in Itang, Godere, and Gog. In the remaining woredas, especially in Jor and Dima, results achieved were poor due to geographical inaccessibility for Jor and insecurity for Dima, respectively. Table VII. Women of Reproductive Age Who Received Tetanus Toxoid Vaccination (Oct. 1, 2002-Sept. 30, 2003) Women of reproductive age are estimated at 23.7% of the total population. This report does not separately show results of pregnant and non-pregnant women. The overall results achieved were very low, declining sharply from TT1 to TT5, indicating a very high defaulter rate, in spite of defaulter training activities undertaken in many peasant associations. The percentage of mothers of children aged 0-23 months who received at least 2 Tetanus Toxoid Injections (TT2) before the birth of the first child was low. As shown in the table above, TT2 coverage was only 12.41%. Woreda Under 1 Antigens Population BCG Measles DPT3 Polio3 Gambella 1,398 853 (61.0%) 588 (42.1%) 612 (43.8%) 612 Itang 942 669 (71.0%) 633 (67.2%) 616 (65.4%) 485 Abobo 698 410 (58.7%) 258 (37.1%) 279 (40.0%) 321 Gog 759 673 (88.7%) 388 (51.1%) 327 (43.1%) 384 Godere 1,617 1,554 (96.1%) 1,076 (66.5%) 1,053 (65.1%) 1,096 Jor 436 63 (14.4%) 61 (14.0%) 61 (14.0%) 61 Dima 650 49 (7.5%) 53 (8.2%) 48 (7.4%) 33 TOTAL 6,500 4,271 3,057 2,996 2,992 Coverage 65.7% 47% 46.1% 46% Woreda Total Vaccinations population WRA TT1 TT2 TT3 TT4 TT5 Gambella 34,949 8,283 3,810 1,458 411 222 81 Itang 23,552 5,582 853 421 156 25 5 Abobo 17,447 4,135 645 269 47 7 2 Gog 18,972 4,496 972 545 294 132 70 Godere 40,436 9,583 1,946 1,841 340 102 74 Jor 10,890 2,581 263 107 60 0 38 Dima 16,250 3,851 203 138 72 15 1 TOTAL 162,496 38,511 8,692 4,779 1,380 503 271 6 Table VIII. Maternal immunization (Women in reproductive age who received vaccination) October 1, 2003 - September 30, 2004 Non-pregnant Women Pregnant Women Woreda TT1 TT2 TT3 TT4 TT5 TT2+ TT1 TT2 TT3 TT4 TT5 TT2+ Gambella Itang Abobo Gog Godere 764 599 387 50 22 1058 165 26 95 29 14 164 Dimma Jor Total 764 599 387 50 22 1058 165 26 95 29 14 164 The report does not include the performance of September 2004. In 2004, activities were undertaken in Godere Woreda only. Activities performed at static sites by woredas other than Godere were not included in this report. EPI outreach sites of Gambella, Itang, Abobo, Gog, Dima and Jor were all closed during the year. Due to insecurity in all other districts, Godere was the only operational project area where all activities were undertaken smoothly and without interruption. As Africare did for all target woredas in the first three years, in year four Africare supported Godere Woreda Health Office with cash for per diems of health workers participating in EPI outreach, fuel for motorbikes and refrigerators and vehicles for the transport of vaccines from facilities with functioning refrigerators to facilities without functioning refrigerators. Africare supported the maintenance and repair of refrigerators in Pole, Tata and Ginger and Shentawa health stations and the health posts of Itang, Gog, and Jor Woredas. Africare’s inputs to the cold chain system and its management were highly appreciated by Regional Health Bureau. Table IX: Consolidated Results of Achievement October 1, 2000-September 30, 2004 The program objective was to increase the proportion of children aged 12-23 months fully immunized before their first birthday from the baseline of 19% and to increase Tetanus Toxoid coverage among women of reproductive age (WRA) before the birth of the first child from 20%. Activities 2001 2002 2003 2004 P A %A P A %A P A %A P A %A EPI (DPT3) 26,798 - - 26,798 2,285 36 6,500 2,996 46.1 1,613* 324* 20* Vitamin A Supplementation - - - 26,798 26,250 98 6,500 5,850 90 NA NA NA TT2 (PW) 31,265 31,265 4,377 14 31,265 4,779 15.3 31,265* 1,058* 11.04* NB. P = planned A = Achieved %A = %Achieved PW = pregnant women *= Godere District Only 7 Achievements of EPI in rates are only indicated for DPT3 and for years 2002-2004. The results for children with DPT3 vaccinations increased to 46.1% in year three from 36% in year two. This was close to the end of project goal of 50%. The results in 2004 dropped to 20% from 46.1% in 2003. This includes only data for Godere District, does not include September 2004 data, and may be the result of under-reporting. Vitamin A supplementation was carried out in concert with polio vaccination drives. While not always separately recorded, Vitamin A supplementation should mirror polio vaccination coverage. The polio vaccination program was suddenly ended halfway through the campaign by severe communal violence, which reduced coverage and interrupted reporting. The EPI program cost for the first two years was 100% covered by the project. During year three, the Regional Health Bureau covered 66% of the cost of EPI program. Africare participated in the Polio Eradication Initiative (PEI) in the region with financial support from the CORE group. Two EPI campaigns were successfully carried out in priority districts with the exception of Dima, where program activities seemed to lag behind schedule due to logistical problems. Constraints Factors negatively affecting results of EPI included: ß High defaulter rates for women due to mobility of the population and long distances to vaccination sites. ß High staff turnover in the Ministry of Health. ß Inequitable distribution of motorcycles and inadequate motorcycle maintenance. ß Weak or dead batteries sometimes not effectively chilling vaccines. ß Poor security. Above all, EPI outreach activities were paralyzed in the districts of Gambella, Itang, Abobo, Gog, Dima and Jor especially in year four, due to insecurity. Similar security problems in Itang and Gog Districts hampered program interventions in year two. ß Problems in accurate reporting due in some cases to poorly trained health personnel. ß A lack of adequate housing for the Field Program Coordinator in Dima. ß Health workers’ dissatisfaction with the rate of per diem for EPI outreach activities. This problem was ultimately resolved through dialogue between project staff, health workers and senior Regional Health Bureau officials. Control of Diarrhea Diseases (CDD) The major causes of diarrhea in the project area are a) lack of potable water, b) poor hygiene and sanitation, and c) lack of awareness of how to protect food and water from contamination.. The project sought to increase access to potable water, improve case management of childhood diarrhea at household and community levels and to promote proper hand washing among mothers to reduce diarrheal diseases in children. Activities included: 1) purchase and distribution of ORS sachets to health facilities; 2) training of mothers and caretakers on recognition of danger signs in children under five, improved case management of diarrhea at home using ORS, including correct mixing and utilization of ORS using potable water for the re-hydration of children; 3) purchase and distribution of utensils such as measuring jugs to health facilities and 4) construction of wells and springs to increase access to potable water. 8 ORT corners were established in 20 of the 30 health facilities of the region. Ten thousand sachets were purchased locally and distributed by the Africare project. In support of the ORT corners, Africare purchased supplies and utensils for 20 health facilities, including Gambella Hospital. The project educated 9,947 people on diarrhea control. The Government of Ethiopia has adopted Integrated Management of Childhood Illness (IMCI) as a strategy throughout the country. However, this strategy was not introduced in Gambella Region until Africare and its partner (GRHB) advocated for facility-based IMCI to complement the control of diarrheal diseases and ARI. The Federal Ministry of Health offered its support in April 2002. UNICEF provided support for the implementation of IMCI and provided training for health workers on IMCI protocols, provision of equipment to health facilities and supervision related activities. To facilitate the introduction of IMCI, Africare and the GRHB, with the support of WHO, organized a three-day IMCI introductory workshop on IMCI. The facilitators for this workshop were from the MOH and Addis Ababa University. Twenty nine health workers from government health facilities and 3 health workers from NGOs in the region participated. A two-week IMCI case management training for higher-level health workers was also organized by the Federal Ministry of Health in Hovan Region. Two health workers drawn from GRHB and the Africare Country Office participated in this training of trainers on IMCI case management at Jima University. The goal was to prepare health workers to manage diarrhea in the health facilities. The health workers who acquired knowledge of IMCI case management then taught mothers on proper home care and feeding of sick children. Table X: Diarrhea Cases Managed in Selected Health Facilities by Woreda (2003) Type of Diarrhea Status of Dehydration Treatment with Woreda Cases Watery Bloody Persistent None Some Severe ORS Other Godere 1,946 1,501 354 91 1,318 525 75 1,304 1,702 Jor 139 96 41 2 74 40 6 225 49 Dima 94 61 30 3 44 46 6 72 94 Abobo 437 363 67 7 299 114 7 306 0 Gog 91 69 17 5 36 24 7 136 14 Itang - - - - - - - - - Gambella - - - - - - - - - Total 2,707 2,090 509 108 1,771 749 101 2,043 1,859 Source: Africare/Ethiopia Third Annual Report Education to mothers and other caretakers on the causes of diarrhea and how to prevent it reached 4,317 men of 5,630 women (total of 9,947). Africare provided clean drinking water to more than 78,720 people through the construction of 23 protected springs, 2 wells equipped with hand pumps, one urban water distribution point, two health center water supply systems and one slaughterhouse water supply system. 9 Table XI: Water and Sanitation Activities by Woreda Woreda Locality Accomplishment Year Beneficia ries Donor Godere Horra #1 Protected spring 2002 580 Alpha Kappa Alpha Sorority Godere Horra #2 Protected spring 2002 610 Alpha Kappa Alpha Sorority Godere Middle Metti Protected spring 2002 1,000 Alpha Kappa Alpha Sorority Abobo Obaya/Pino 16 meter well with hand pump 2003 1,000 Besser Foundation Gog Obala 22 meter well with hand pump 2003 1,250 Besser Foundation Gog Pugnido Health Center Piped Water Supply System 2002 >20,000 Alpha Kappa Alpha Sorority Godere Metti Kebele 01 Protected spring 2003 2,188 Besser Foundation Godere Metti Kebele 02 Protected spring 2003 1,923 Besser Foundation Godere Dunchy 01 Protected spring 2003 550 Besser Foundation Godere Goshini Protected spring 2003 1,510 Besser Foundation Godere Tileku Metti 01 Protected spring 2003 307 Besser Foundation Godere Chemi 01 Protected spring 2003 405 Besser Foundation Godere Chemi 02 Protected spring 2003 350 Besser Foundation Godere Gelesha Protected spring 2004 1,456 Besser Foundation Godere Metti Town Kebele 01 Water distribution point 2004 8,700 Besser Foundation Godere Tileku Metti 02 Protected spring 2004 3,750 Kite Foundation (African Well Fund) Godere Gengeboz 01 Protected spring 2004 1,250 Kite Foundation (African Well Fund) Godere Birebandy (Middle Metti) Protected spring 2004 750 Flora Family Foundation Godere Tiruneh (Middle Metti) Protected spring 2004 300 Flora Family Foundation Godere Shay (Dushi) Protected spring 2004 600 Flora Family Foundation Godere Gengeboz 02 Protected spring 2004 250 Flora Family Foundation Godere Tetay (Kumi) Protected spring 2004 500 Flora Family Foundation Godere Akashi Protected spring 2004 1,000 Flora Family Foundation Godere Shone Protected spring 2004 362 Phi Delta Kappa Sorority Godere Dunchy 02 Protected spring 2004 643 Phi Delta Kappa Sorority Godere Kabbo Protected spring 2004 2,136 Phi Delta Kappa Sorority Godere Abawondimu (Tolly & Tokally) Protected spring 2004 350 Phi Delta Kappa Sorority Godere Metti Health Center Piped Water Supply 2002 >20,000 Phi Delta Kappa Sorority Godere Metti Town Slaughter House Piped Water Supply 2004 5,000 Phi Delta Kappa Sorority 10 The objectives of increasing proper case management of childhood diarrhea at household and community levels, of promoting proper hand washing among mothers, of increasing the percentage of mothers providing the same amount or more fluids (breast milk) to their children with diarrhea, and of expanding access to safe drinking water throughout the project area appear to be on target. Various activities have been performed in the control and prevention of diarrheal diseases and its case management at household and health institutions jointly by field program coordinators and health workers from district health offices and zonal departments. Real involvement of the community, community leaders, heads of zonal health departments and district health offices in planning, implementation, monitoring and decision-making was evident to the evaluator. This participation is key to ownership and crucial to sustainability. It has created awareness in the community on the differences between contaminated water sources and safe/potable water. A slaughterhouse in Metti Town, Godere District, which formerly obtained water from an unprotected source, was provided with a piped water supply system by Africare. This has enabled the slaughterhouse to wash the floor, to clean all equipment such as knives and hooks and to keep the persons who slaughter animals clean. Committees were established in all villages where springs were developed. Male and female committee members were officially registered with local government. They were trained in spring maintenance and keeping the area around the springs clean and free of standing water. A committee at the Metti Town water distribution point was created and is collecting water user fees for future maintenance. Local government was encouraged by Africare to extend this fee system to the protected springs. The skills and organizational experience gained by local government in these cost-effective solutions to clean water provision are likely to result in the implementation of similar projects throughout the district in the future. Africare was able to demonstrate how communities, local government and NGOs can pool their resources to realize clean water at very low cost compared to similar government sponsored projects in the past. HIV/AIDS/STI Prevention and Control The program objective was to increase knowledge and awareness regarding the spread and prevention of HIV/AIDS/STI among mothers and secondary school students. The project staff conducted a baseline assessment of HIV/AIDS/STI knowledge among secondary school students in 2002 with the data analyzed and the report finalized in 2003. High school HIV/AIDS clubs were established in seven high schools. Africare field coordinators provided technical support to all high school activities. In 2003, Africare purchased and distributed mini media supplies including tape recorders, video decks and megaphones to all high schools in the project area. An estimated 6,272 students, of which 1,062 females, were involved in club activities. The project trained high school teachers in HIV/AIDS prevention. Students took anti-HIV/AIDS messages into the general population through drama and music. 11 Africare initiated an activity named “Love Life” under its Africa-wide HIV/AIDS Voluntary Service Corps. Love Life focused on women commercial sex workers with multiple sexual partners. This activity was funded by Africare headquarters. In 2002, over 70 “bar ladies” voluntarily participated in this activity. In 2003, the number of participants increased to 176 women. They were organized into 17 groups of ten women per group. Each group has its own leader, elected by the members. The leaders were trained in group facilitation skills and HIV/AIDS prevention and control themes. Each of the groups met bi-weekly and discussed issues pertinent to the transmission of HIV/ AIDS/STI. They were trained in HIV/AIDS/STI prevention, testing and control, negotiation skills and confidence building. All women received free condoms and the group leaders distributed condoms to bars, hotels and restaurants. The knowledge of commercial sex workers on HIV/AIDS/STI has greatly improved. At least one leader ceased being a commercial sex worker and with the project’s assistance obtained employment as a mechanic in a local garage. The GCSP baseline KPC survey also informed the project as to priority HIV/AIDS intervention areas and set program objectives. Regarding HIV/AIDS prevention and control, women were asked a series of questions on knowledge and practice. There were significant increases in knowledge of the modes of transmission and means of prevention of HIV/AIDS and where to access condoms from baseline to final. As for knowledge of modes of transmission, 56.3% cited sexual contact at the baseline, compared to 89% at final; unsterile needle/blade 28.6% at baseline compared to 85.5% at final; contact with infected blood 9.6% at baseline compared to 38.6% at final; and mother-to-child 7.5% at baseline compared to 17.9% at final. Methods of prevention cited included having one partner 33.6% at baseline compared to 69.3% at final; abstention from sex 30.2% at baseline compared to 74.2% at final; avoidance of dirty needles/blades 19.3% at baseline compared to 19.3% at final; and use of condoms 15% at baseline compared to 54.5% at final. Utilization of condoms was as low as 2% at baseline compared to 13% at final. The high school club activities are clearly replicable, where there are knowledgeable health workers who can train and support teachers and students. Having students conduct HIV/AIDS education in the community on a voluntary basis is extremely cost-effective. The work with commercial sex workers appears to be worthy of replication. It empowers the women and gets other community people such as bar and hotel owners involved. Having health professionals interact with commercial sex workers can “open the eyes” of the women to other, safer careers. C. Cross-cutting Approaches 1. Capacity Building at Community Level Training of village health committees and CHAs helped to bridge the gap between the community and health facilities for health information dissemination to increase communication awareness on health service utilization for immunization and other services and early care seeking at health facilities. 12 Forty CHAs who were originally trained by UNICEF in Abobo and Godere Districts received a one-week refresher training on the three project intervention areas as well as on malaria control, water/sanitation and harmful traditional practices. It was financially supported by Africare and technically supported by project staff in collaboration with senior health workers from the GRHB. To use them as assistants in project implementation in areas were there are no CHWs or enable them to play a role in social mobilization, 48 members of village health committees were trained for five days in all districts except Jor. The members of the village health committees (VHC) contributed greatly to the success of the National Immunization Days (NIDS), HIV/AIDS/STI prevention and control and defaulter tracing. Peripheral EPI modular training improved the quality of immunization services and enhanced the cold chain system. This one￾week training organized by GRHB and Africare in Gambella Town involved twenty-five staff from selected health facilities and one project field coordinator. Participatory project design and health management training was jointly conducted at Jima University in collaboration with the GRHB. Eleven persons, two of whom were Africare staff, participated in the training. A public health expert from Jima University provided the training in a very participatory way. The training was supported by a field visit which enhanced the participants’ understanding of project monitoring and evaluation. The participants designed two projects and presented them in class. The trainers reportedly provided constructive feedback to the trainees on their proposals. Training strengthened the region’s capacity in project design, monitoring and following up. The table below details all of the project’s community health education training sessions. 13 Summary of Health Education Sessions Conducted by Year (Oct. 1, 2002 - Sept. 30, 2004) S/N Subject Number of health education attendants by year Oct. 1, 2002 -Sept. 30, 2002 Oct. 1, 2002-Sept. 30, 2003 Oct. 1, 2003 - Sept. 30, 2004 No. of Session Male Female Total No. of Session Male Female Total No. of Session Male Female Total 1Immunization (EPI) 64 540 728 1,268 207 4,393 5,701 10,094 154 4,064 5,234 9,298 2HIV/AIDS/STI 69 1,993 1,648 3,641 352 10,990 8,918 19,908 121 3,882 2,549 6,431 3 Harmful traditional practices drug abuse 3 17 33 50 0 0 0 0 0 0 0 0 4 Communicable & vector born diseases 5 199 323 522 0 0 0 0 38 688 573 1,261 5 Environmental sanitation & personal hygiene 30 305 280 585 92 1,923 1,483 3,406 61 1,249 1,259 2,508 6 Family planning and use of condom 19 82 226 308 76 3,632 3,559 7,191 53 1,388 746 2,134 7Acute respiratory Infections (ARI) 4 47 28 75 0 0 0 0 0 0 0 0 8Measles 4 29 66 95 0 0 0 0 0 0 0 0 9Malaria 37 416 341 757 88 787 775 1,562 79 1,404 1,179 2,583 10Breast feeding 5 56 73 129 18 294 259 553 13 387 753 1,140 11 Malnutrition and Intestinal parasites 14 182 212 394 0 0 0 0 19 364 284 648 12Control of diarrheal diseases 14 108 92 200 70 3,423 4,779 8,202 53 786 759 1,545 13 Others (TB, Nutrition, Home delivery Amobiasis) 0 0 0 0 133 3,728 3,003 6,731 119 14,212 7,417 21,629 Total 268 3,974 4,050 8,024 1,036 29,170 28,477 57,647 710 28,424 20,753 49,177 14 Training for water resource management committees at village level involved water committee members from all developed springs and taps. They learned how to handle developed water sources in a proper way and use them safely for consumption. They were trained how to take care of protected springs and water points, and how to repair them whenever necessary. 2. Capacity Building of Local Partner Organizations Africare emphasized improved organizational capacity, information management systems and health service management as a whole. Health planning exercises were conducted at the district level for the first time so as to decentralize micro health planning. District health managers, members of woreda social service sectors, planners and heads of health facilities were involved in the training. Trainees were provided with basic concepts of planning, identification of health problems, priority setting, objective setting, monitoring, evaluation and feedback This improved the knowledge of health workers in recording and data processing for planning and decision￾making. Training was conducted in six of the seven project districts with 13-20 health officials from each district participating in the 2-3 day workshops. The project trained health workers were in IMCI, immunization, and water/sanitation. The Government of Ethiopia adopted IMCI throughout the country but it had not been introduced in Gambella until Africare convinced the Federal Ministry of Health (FMOH) to do so. To launch the IMCI strategy in the region, the project provided refresher training on IMCI case management to enable them to train others. The Federal Ministry of Health organized the two￾week IMCI case management for high-level health workers in Harar Region. In 2003, Federal Ministry of Health senior staff conducted a three-day IMCI orientation workshop for health managers and other local government officials. Africare, in collaboration with the regional health bureau, sponsored a 15-day IMCI case management training for 25 health workers at Jima University’s Pediatric and Child Health Department. Trainees visited Gambella Hospital and Itang, Pugnido and Metti Health Centers to observe the application to IMCI techniques. The extensive project intervention in capacity building changed partner organizations. Improvements were reported in management (utilization and reporting on donor funds such as UNICEF’s), better cold chain implementation and work habits. 3. Capacity Building of Health System Africare was effective in strengthening the health management information system and providing technical and logistic support (per diems for health workers participating in EPI outreach, fuel, ORS and ORT material purchase, cold chain maintenance and motor bike maintenance) The baseline KPC survey and health facility assessment helped the GRBH to identify problems in the health institutions (i.e. low EPI coverage, high incidence of diarrhea, low awareness on HIV/AIDS prevention, and low clinical performance of health workers). Key points are noted below: ß The plans/programs that were supported by the project are parts of the MOH activities built in already existing structures. These plans will be continued by the MOH. The 15 capacity building conducted at community and institution level is significant for project sustainability after the project phase out. ß The sustainability plans are realistic as they are within the MOH plans and as they are built with active participation of the community. ß The capacity building activities at the community level will help to link the facility and the community and vice versa. 4. Strengthening Health Worker Performance Key results are noted below: ß Health worker performance was strengthened through training (seminars, workshops) practical work, supervision and review meetings. ß Supportive supervision activities motivated health workers to keep up their ethics and morale, resulting in better performance. ß The plans for sustaining health workers performance once the program closes are the current government policy that launches decentralization, capacity building for workers to build their knowledge and skills at the periphery level. 5. Training Key points are noted below: ß The training strategy was effective as it was conducted using participatory methods in which each participant contributed his or her valuable experience during training. ß Training objectives were met as reflected by health workers in health facilities and village health committee members’ performances during post training and supportive supervision. ß While a final health facility assessment was not done, improved practices at health institutions and at EPI outreach sites were noted during supportive supervision. ß The best practices and lessons learned were those things observed as health workers showed good morale in doing their work more effectively than before. The health workers have started to monitor the cold chain, check vaccine stocks and to request supplies in a timely manner. ß The plans for sustaining these training activities once the program closes are practices of conducting trainings like seminars, workshops, using the printed and distributed IEC/BCC materials, health workers who attended TOT and concerned higher officials at district and regional level. ß Sustainability plans for training may not be as realistic as those of EPI, CDD and HIV/AIDS prevention. 6. Sustainability Strategy There are potential problems, but there is also considerable hope. Staff turnover and irregular supply of vaccines, materials etc. of other donors due to e.g. CAG BAN policy of UNICEF can be cited among the problems the Regional Health Bureau would be expected to face. Complaints of lack of resources and of difficulties in obtaining equipment, drugs, cash allocation etc. may continue. In the January 31st 2002 Revised Detailed Implementation Plan for the Gambella Child 16 Survival Project, sustainability is defined from a financial perspective, in terms of community competence and increased capacity and motivation of health staff. In the same document improvement of sustainability as it pertains to the GSP encompassed three components: a) improvement in the financial and logistics management of local institutions (GRHB, WAB, District Health Offices, etc), b) improvements in the morale, technical and managerial capacity of health workers, and c) increased community organization and community involvement in managing its own health. One of the strategies to improve the health services financing system is the use of fee for service. Though this is a government policy; the region has not introduced the system. Ability and willingness to pay for services, and ways and means of improving exemption rules and regulations was under study during the GCSP implementation period. But, cost sharing strategy, e.g. 65% of the costs of EPI outreach programs were covered during year three of the implementation period. Regarding community competence, Africare (GCSP) facilitated household and community action with respect to health which had the potential for helping people take care of them selves better, look after surroundings and remove hazards to health, and be able to make better use of the health care system. Refresher training of Community Health Agents (CHAs), establishment of voluntary village committees, school clubs, and women commercial sex workers, involvement of the community in planning implementation, monitoring and decision making which were real for GCSP would be essential to ownership, and ownership would be crucial to sustainability. However, established committees, groups etc. which functioned for a short period, e.g. ‘Love Life Project’ women sexual workers groups not continued to exist due to most of the members not living in Gambella Town. Provision of logistical and other kinds of supports strengthened the health institutions serving the project areas. Institutions such as the GRHB and the various district facilities as well as community-based groups were expected to be sustainable. Transfer/turning over of Africare properties like vehicles and fuel will contribute to sustainability. Other materials transferred include properties of the Food Security Initiative comprising agricultural tools, a generator, vehicles, motorcycles, computers and office furniture. All these properties were transported and stored in the DPPO store at Gambella Town. Organizational capacity development focused on developing structures, processes, management system were well undertaken; participatory techniques were employed, improving training programs, running frequent refresher courses, continuous education were all positively done. The training or capacity building on health program management and implementation of activities would enable the RHB, woreda health offices better manage the resources available to them and identify other resources for years. There has been improvement of GRHB capacity through participation in Child Survival activities which were not in place before the introduction of the GCSP including training, surveys, outreach services, educational materials development, social mobilization; woreda- based planning, supervision etc. All of these and the like would contribute a lot to sustainability. 17 D. Program Management 1. Planning a. Project planning was very inclusive. The project was designed and the DIP was developed on site in Gambella with the involvement of Africare headquarters health staff, consultants, Africare/Ethiopia country office and project staff, the Gambella Regional Health Bureau, the Gambella Disaster Prevention and Preparedness Office, the Gambella Education Bureau and the Gambella Women’s Association. This inclusive approach was key to the successful implementation of the project. All government stakeholders were fully knowledgeable about the project. This led to their active involvement in project coordination meetings, project progress reviews and evaluation as well as in project implementation. The strong partnerships were crucial to the project’s successes in a very difficult climate, with important security and logistical constraints. b. The original DIP was overly ambitious. The revised DIP, approved by USAID, was more realistic, but still underestimated the difficulties of working in remote Gambella Region. c. There were no apparent gaps in the DIP. 2. Staff Training a. The high turnover of Bureau of Health staff resulted in repeated trainings and orientation for government health partners. The training helped the project, but the rapid turnover makes it difficult to measure. b. Staff training resources were adequate. c. Flexibility is the key to staff training. New staff arrivals must be trained, whenever they arrive. 3. Supervision of Program Staff a. The supervisory system worked well. Africare headquarters regional management provided adequate supervision of senior country office staff who supervised senior project staff. The field project coordinators at district level worked hand-in-hand with Ministry of Health district officials. This facilitated supervision and made it more likely that project and Ministry of Health staff would work responsibly in remote settings. Long distances between staff, poor telecommunications and poor security were supervisory challenges, which were for the most part overcome. b. The supervisory system is not institutionalized, as there is no more project. c. The Ministry of Health’s improved reporting systems and the logistics support of the project regarding vehicles and motorcycles, bode well for improved supervision within the Ministry of Health. 4. Human Resources and Staff Management a. The project, as such, is over. The project does not continue. However, the project’s capacity building and infrastructural support will help the Ministry of Health continue activities such as EPI and IMCI which were carried out during the project. b. Morale and cohesion of project and partner staff were excellent. In the remote and harsh environment, professional and social relationships were essential. Project and partner staff shared the benefits of training and the burdens of work. 18 c. Staff turnover was low during the first three years. During the final year, turnover and early departures of staff were high due to extreme insecurity and the need for staff to seek other employment as the project neared its end. The impact on the project was not great, since the real problem was lack of access to most project districts during the year four. In fact, the concentration of the remaining staff in Godere District resulted in very high levels of accomplishment in water resources development and water user committee training in Godere District in year four. d. Departing staff were provided with employment certificates and letters of recommendation to assist them in finding employment in other projects or to continue their education. 5. Financial Management a. Africare’s financial management system was fully developed, documented and in use throughout the project. The project’s budgetary needs were forecasted and approved on a quarterly basis. There was sufficient decentralization to the country and project levels to enable timely procurement of goods and services. There was an instance of embezzlement by the project accountant in year three, which led to the dismissal of the project manager and a tightening of internal controls. The country office took over spending decisions. The regional Ministry of Health has gradually taken over financial responsibility for the EPI program. While the project funded 100% of EPI activities in the first two years, by year three the regional Ministry of Health had assumed 66% of the costs. b. Africare donated all project vehicles, motorcycles, project computers, office equipment and remaining supplies to the Gambella Regional Government. It is expected that these items will help local government pursue the goals of the project and help it to attract other technical and financial assistance. c. Africare, internally, had sufficient expertise to promote, within reason, the sustainability of the project. 6. Logistics a. The long distance from Addis Ababa to Gambella and the long distances and poor roads within Gambella made procurement and distribution time-consuming and costly. The fact that the child survival project was managed jointly by Africare with its food security, nutrition, water and HIV/AIDS project made logistics more manageable. A considerable amount of procurement, accomplished under the food security project, was of benefit to the child survival project. b. As mentioned above, the donation of transport and office assets to the local government greatly enhances the local government’s capacity to sustain project activities into the future. 7. Information Management a. The data collection systems were generally effective except when interrupted by insecurity, forcing staff to be relocated from their assigned posts. b. Monthly, quarterly and annual reports helped project, country office and headquarters staff to monitor the project’s progress and problems in a timely way. Frequent telephone, 19 fax and email correspondence generally worked well. The decision to consolidate and then safely evacuate staff resulted from telephone calls from the project manager to the country representative. c. The Ministry of Health in Gambella has improved data collection skills and instruments and is expected to continue its reporting within the Ministry and to partners such as UNICEF. d. The project conducted a high school student HIV/AIDS survey to inform its design of the high school HIV/AIDS component of the project. The project also conducted a baseline survey, health facility & training needs assessment study, community nutrition study and the final KPC survey. All studies assisted project staff to better understand the problems in the region and promoted the partnerships necessary for successful project implementation. e. The project directly improved the Ministry of Health’s reporting through training and deliberate collection of reports, especially on EPI. f. Staff and local government partners worked together on project implementation and evaluation. Local government has an excellent understanding of overall project accomplishments, although this varies depending on how long officials and technicians have been in place and associated with the project. Community members understand the interventions that took place directly in their communities or at the health facility serving their community. As far as the local community is concerned, their understanding comes from first-hand knowledge rather than written reports. g. The CORE Group can take advantage of polio vaccination data to design its interventions in the future. 8. Technical and Administrative Support a. The project benefited from at least four headquarters health professionals visiting Gambella, some more than once. There were also outside consultants in IEC production and assistance from a nutritionist in one of Africare’s projects in Uganda. An IFESH volunteer also assisted the project. The technical assistance was particularly useful given the remoteness of Gambella. The technical assistance was appreciated by project technical staff, as it kept them tied into the wider international public health community. b. There was no unmet expression for outside technical assistance. c. See “a” above. 9. Management Lessons Learned a. Joint planning with local partners is critical to success and sustainability. b. Frequent visits to the project by country office staff can prevent small problems from becoming large problems. E. Other Issues Identified by the Team a. Training of additional community health agents was inhibited by the local government’s inability to pay for their services. This was partially compensated for by training of village health committees. 20 b. Local government senses a need to identify other external partners, with Africare’s departure. F. Conclusions and Recommendations General Africare’s contribution to the reduction of child morbidity and mortality through regional strengthening in the areas of child survival intervention was greatly appreciated by its local partners. The child survival program comprising EPI plus, prevention/control of HIV/AIDS/STI and control/management of diarrheal diseases was very supportive and addressed the priority problems of the region. In addition, Africare worked collaboratively with the GRHB, the project’s main implementing partner in the selection of intervention areas and strategies and worked through partnerships with local providers. Africare helped communities to organize themselves in support of community health systems and trained health workers on management were appropriate to better serve client needs. The approaches of the project (community mobilization, capacity building, inter-sectoral collaboration, as well as monitoring and evaluation) were appropriate and essential. They enabled beneficiaries to be aware of their needs and available resources and to make appropriate demands, which, in the long run, increases community organization, ownership and involvement in managing its own health. The project enabled health workers, mainly mid and upper managerial level, to be aware of, interested, and enthusiastic about their work and more confident in the management of childhood illnesses. These people, if not leaving the region soon or before they are replaced by others who are knowledgeable and skilled in management, will be of great use for the region. This approach, in the long run, will have the greatest impact, if and only if lessons learnt and knowledge acquired are applied in a sustainable way. Activities such as the training of commercial sex workers also proves that people can carry out important simple tasks without sophisticated training. To be able to meet the goal at the end of the project period, a lot of effort was made to improve service delivery and quality of care, develop skills, enthusiasm and understanding of what is to be done, develop confidence, and enable people to devote their time, energy and knowledge acquired for the work. The first year concentrated on organizing and arranging for project implementation and the final year was hampered by security problems. In years 2002-2003, program implementation was especially effective. Hence, overall achievement of service delivery was satisfactory. High involvement of communities in the implementation, making services accessible and scaling up of service provision as well as indications of smooth relationships among partners were good lessons to be learnt. Specific Despite serious security constraints, the project came very close or exceeded most objectives. As noted earlier, security concerns in year four made it impossible to evaluate project outcomes outside of Godere District after year three. 21 The most important achievements were: a. Strengthened capacity of regional Ministry of Health; improved services for under-fives and women institutionalized. b. Decentralized health micro-planning at district level strengthened. This is especially important in such a vast remote area. c. Introduction of IMCI to the region. d. Strengthened community capacity through village health committees, community health agents and water use committee training in health education, sanitation, HIV/AIDS. e. Significantly higher vaccination rates. f. Provision of clean water to more than 78,000 people. Best practices are collaborative planning and implementation with local health ministry and system-wide and community capacity building. Recommendations ß To assure sustainability of EPI plus programs, the region should immediately dialogue with UNICEF regarding the CAG BAN policy so as to release its cash assistance in a timely manner. The EPI program, which was highly dependent on Africare support, will suffer greatly if not given immediate attention. ß Regarding staff turnover, the region should keep trained staff on board until others are trained and ready to step in. ß District government should closely monitor the developed springs to ensure their sustainability. ß Regarding sustainability, the project’s emphasis on training and capacity building in the health system and community bode well for the future. Lessons learned will be shared through Africare headquarters Office of Health and HIV/AIDS newsletters and participation in meetings and workshops. Staff will carry lessons learned with them to other Africare-assisted countries. The project will scale-up and expand to the extent that the Government of Ethiopia can attract resources to Gambella Region. The development of oil resources in the region should attract corporate and governmental resources to further address child health in the region.