g~ - /+AS- E: 6S- 4 /'Kbz5 PEOPLE-TO-PEOPLE HEALTH FOUNDATION, IN'C. (Project HOPE) Grant No. 656-0229-G-00-7007-00 FINAL EVALUATION FOR IMPROVING MOTHER AND CHILD HEALTH IN ILE DISTRICT, ZAMBEZIA , MOZAMBIQUE JUNE ISTH Through JUNE 30m, 2000 Submitted to: Project HOPE Headquarters Carter Center Millwood, VA 22646 Authors: Ms. Maria Valencia-Gutierrez, M . A., Team Leader Elise Jensen, MPH, Project HOPE, Dir. Africa program Dr. Arturo Sanabria, MD, MPH, Project HOPE'S Country Representative Mr. Carlos Domingos, Field Coordinator Ms. Claudia Goveia, MCH Health Nurse Ms. Celia Sendela, MCH Health Nurse Ms. Tania Lucinda, MCH Health Nurse Mr. Cardoso Albino, Field Trainer Mr. Benjamin Dovel, Field Supervisor Mr. Placido Eduardo Health Information Systems Secretary TABLE OF CONTENTS List of Acronyms EXECUTIVE SUMMARY - Introduction - Findings - First Phase - Findings - Second Phase - Institutional Sustainability - Meeting with the Zambezia Province Governor PART I. 1. 2. 3. 4. 5. 6. 7. 8. PART 11. 1. 2. PART 111. 1. 2. 3. INTRODUCTION Project Goal and Objectives Geographic Coverage Country's Population Data Country's Economic Data Public Health Services Beneficiaries Strategies and Partnerships Surveys EVALUATION POINTS Sources of Information Personal Interviews DISCUSSION OF PROJECT ACHIEVEMENTS Project's First Phase - May 1997 through December 1998 Technical Activities - Immunization Activities During this Period - TTV Activities - MCH Activities - Family Planning Services - HIVIAIDSlSTDs Prevention Activities Conclusions PART IV. 1. PART V. 1. 2. 3. 4. PART VI. PART VII. DISCUSSION OF PROJECT ACHIEVEMENTS Project's Second Phase - January 1999 through June 2000 - Immunization/- Findings - Diarrheal Disease Controll- Findings - HIVIAIDSlSTDs Prevention/- Findings - Family Planning/- Findings - Monitoring the Quality of Interventions SUSTAINABILITY Community Participation Ability and Willingness of Government Institutions Health Information System Sustainability Plan LESSONS LEARNED LIST OF PERSONS CONTACTED ACRIS AIDS CHW CS DIP DDS DPT GNP HC HCW HIV HOPE IUD KPC MPH MCH MOH MVB NGO OPV ORSIT PC PH PVO STD SOW TTV UNICEF URC US AID LIST OF ACRONYMS Ajuda Cristao para Saude (Christian Health Aids) Acquired Immune Deficiency Syndrome Community Health Workers Child Survival Detailed Implementation Plan Derek Distrital de Saude (District Health Directorate) Diphtheria, Pertussis and Tetanus Gross National Product (total value of goods & services produced by the economy in a year .) Health Center Health Care Worker Human Immunodeficiency Virus Project HOPE Intra-uterine Device Knowledge, Practice and Coverage Survey Masters in Public Health Maternal Child Health Ministry of Health Mobile Vaccination Brigades Non-governmental Organization Oral Polio Vaccine Oral Re-hydration SolutionITherapy Prevention Committees Public Health Private Voluntary Organization Sexually Transmitted Disease Scope of Work Tetanus Toxoid Vaccine United Nations Children's Fund University Research Corporation United States Agency for International Development EXECUTIVE SUMMARY Introduction "Improving Mother and Child Health" implemented by the People-to People Health Foundation, Inc. (Project HOPE) started on May 1997 in the Ile District, upper Zambezia, Northeast Mozambique. Ile is the country's most populated district, and Zambezia the most populated province. The major economic activity in Ile is family agriculture and there is very little commercialization of agriculture produce. There are no banking facilities or telephone lines and electricity is severely rationed. Funds for the project were provided by USAID/Mozambique. The USAID $1,100,000 was matched by $366,667' of Project HOPE'S private funds. Pipeline analysis was beyond the Scope of Work (SOW). The in-country portion of the final evaluation of HOPE's Child Survival project was conducted from June 18" through July 2nd, 2000, by an eight (8) member multi-disciplinary team. Team members included three project Maternal and Child Health (MCH) nurses, a field coordinator, a field trainer, a field supervisor, the health information systems manager, HOPE's Regional Director for Africa, the Country Director and an external evaluator who served as team leader. Eight communities and health centers were chosen to conduct group interviews with Health Care Workers (HCWs) and HIV/AIDS/Prevention Committees (PCs) and community members for the final evaluation. The first evaluation meeting took place the day after arrival in Ile with the District Health Directorate's (DDS) Director and his staff. Four days were then spent conducting interviews in the following communities: Socone, Muliquela, Phalane, Mugulama, Chiraco, Mulevala, Nampevo, and Narnanda. Approximately eighty community members and fourteen Community Health Workers and HIVIAIDSISTD Prevention Committees participated in these meetings. On a daily basis, upon concluding community meetings, team members regrouped to discuss and synthesize the information into the main areas of concern specified in the SOW. Among the team members, consensus was reached regarding the needs expressed by the various community groups. On the last day of the evaluation, a meeting with the DDS staff was to take place. Unfortunately, the DDS' Director had been called the prior day to another part of the Province so this scheduled meeting didn't take place. Both the Field Coordinator and the Country Representative were requested to meet with the DDS' Director to inform him of the evaluation results. ' Project's Detailed Implementation Plan Findings First Phase HOPE's "Improving Mother and Child Health in Ile District" suffered from ineffectual project management during its first eighteen months. According to project staff and colleagues, the first Project Director who also served as HOPE's Country Director, was technically familiar with child survival concepts, but not strong in conceptualizing, long term planning and day-to-day organizational needs of the project. The lack of these skills hampered project progress. There were also difficulties in recruiting staff to fill project posts. In addition to these internal difficulties, some of the inputs that the project partners were expected to provide were not forthcoming. The project experienced an out-stock of vaccines which ended in October 1997 and health centers were not equipped with scales and functioning refrigerators due to lack of fuel. Nevertheless, during this first phase of the project a logistical base was established in Mocuba the nearest big town next to Errego (HOPE's base of operations in Ile). In at least two initial quarterly reports the Project Director complained about the choice of Mocuba as opposed to Quelimane for HOPE's logistical base due to some major problems, such as poor communications and poor electrical supply which interfered with the good hctioning of the office equipment. Ile was mentioned in the reports as a problematic area because of its isolation, which was an obstacle for the recruitment of staff. The technical activities that took place between January 1997 and December 1998 seemed to be either ad hoc, or not linked to the activities planned and presented in the Detailed Implementation Plan (DIP). A lesson learned from the first phase of the project is that HOPE should not have waited eighteen months to dismiss an ineffective Project Director. While HOPE attempted to remediate the situation by hiring a expatriot deputy coordinator; conducting 5 senior level technicaVadministrative visits within the first year; and hiring two external consultants to assess and provide a project improvement plan, none of these strategies were successful in addressing the fundamental problem. One can not help wondering what the final impact of the project would have been if field activities would have been initiated when they were planned. Not surprisingly, a review of the project's achievements for this first phase shows less than expected vaccination results. Vaccination campaigns started during the September-December 1998 period. A total of 18,734 children 0-1 1 and +12 months received BCG, Polio l,2,3 and Measles vaccines during the fourth quarter 1998. Out this total figure 9,042 were immunized by the project's Mobile Brigades with the support of the health committees' increased mobilization. This represented a project contribution to the campaign of 48.3%. A total of 867 women received Tetanus Toxoid Vaccine (TTV). 44% got second doses as opposed to the 6,000 that were supposed to be vaccinated the first year. These figures fall below the project's first year target. A total of 2,270 pregnant women received la, 2nd, 3rd, and 4" doses during the fourth quarter 1998. A total of 900 were immunized by the projects Mobile Brigades with the support of the health committees' increased mobilization. This represented a project contribution to the campaign of 39.6%. Pre-and Postnatal consultation in the District amounted to 6,069 during the 4' quarter 1998. Only two users accepted the IUD the rest opted for Depo Provera and the pill which are the most popular and widely accepted contraceptives. HOPE, through the mobile brigades, distributed pills to 454 women and Depo Provera to 456 during the 4' quarter 1998. Health talks were conducted in sixteen communities. A total of 2,850 participants received information on child spacing, FP, vaccination, nutrition and sanitation, and 668 women received pre-natal examinations. The health committees made 989 home visits in total that included 558 children and 256 pregnant women. No HIVIAIDSlSTDs prevention activities took place during the first phase of the project. The project's accomplishments noted for this period are: the establishment of the health committees which are still hctioning; the establishment of a relationship with the Ile DDS through the support the project provided to the DDS in their vaccination campaigns and the logistical support the project provided during a cholera outbreak; training of DDS staff; the establishment of regular planning and problem solving meetings with the DDS staff; establishment of the Project's physical presence in Ile; resolution of the communications problems; and the initiation of the Health Information System. In response to the above mentioned obstacles, HOPE gave notice at the end of 1998 to the first Project Director. A new team composed of the new Country Representative and a Program Manger, both physicians, arrived in the country on March 14', 1999. The new Country Representative was a Public Health specialist with extensive experience working with Non￾Governmental Organizations (NGOs) and government institutions, and the Program Manager was a Child Health specialist with many years of experience managing primary health care and Child Survival (CS) programs in rural Afiica. The new team initiated an accelerated strategy in collaboration with the DDS and Ajuda Cristao para Saude (ACRIS) the project's partners. With the new team in place, the project started its second phase. A total of 142 Mobile Brigades were carried out between January 5" and June 16h, 2000, which provided services to women and children from 40 communities2 in the Ile District. The following is a summary of the various services provided by the HOPE during the above mentioned period: Vaccinated Children: 14,442; Pre-natal Consultations: 3,168; Post-natal Consultations: 669; FP - Pill 1,394; FP - Depo-Provera: 2,214; TTV- Pregnant lst doses: 2,481; TTV- Pregnant 2"* doses 871; WFA - 1" doses 6,084; WFA - yd doses: 2,895; WFA - 31d doses: 544; TTV - Other: 563; TOTAL SERVED: 35,3253. Only 14 pregnant women returned for the 3'd; 4'; and 5' TTV doses, and 24 WFA returned for the 4' and 5' doses. HOPE'S focused on providing immunization delivery by training Community Health Workers (CHWs) and DDS health center staff. The CHWs serve as health promoters in the localities were the project works. The job of this groups is to motivate women to get their children vaccinated * Resumo das actividades de PAV e SMI das Brigadas Moveis prepared during the evaluation by HOPE'S staff. Resurno das actividades de PAV e SMI~das Brigadas Moveis prepared during the evaluation by HOPE'S staff. 3 either by attending the health center vaccination day or the mobile vaccination clinic held at the village level. A final Knowledge, Practice and Coverage (KPC) Survey Report has been prepared by HOPE and is attached. The report contains a complete analysis of the final survey results. The evaluation report will only be presenting a summary of relevant findings made during the final KPC. Findings - Second Phase The project surpassed its vaccination objectives. The per cent of children who received some type of vaccine ever was recorded at 8 1.4% of 30 1 surveyed mothers. Children with immunization cards increased to 72% at final survey from 44% at baseline. Almost half of children with immunization cards are vaccinated with DPT3 and OPV3. Complete coverage rates for BCG, DPT3. OPV3, and measles increased significantly from 15% at base line to 41% at final KPC for children with immunization cards. DPT dropout rates among children 12-23 months of age was calculated at 3 1.1 %, and at 33.6% for OPV. TTV coverage rates, although low at the final survey, reflect a significant increase from baseline: 46.7% of women of reproductive age at final survey as opposed to only 9% at baseline had one TTV doses. 26.3% of women had received two doses , a significant increase from the 4% at baseline. These achievements are particularly significant given the stock out of vaccines experienced in the Province during the first part of the project which creates distrust among the population, and the short period of time (1 8 months) in the second phase, the project had to reach its vaccination targets. The final KPC survey found that 59.8% of 301 mothers interviewed stated that their children had had diarrhea within the two weeks prior to the survey. 49.2% of mothers interviewed recognized weakness and tiredness as a symptom to seek advice or treatment for diarrhea. 37.9% mentioned signs of dehydration as a symptom for a diarrhea episode. These figures clearly indicate that mother's knowledge of diarrhea case management did improve as a result of the project's interventions. Of the three hundred adults interviewed during the final KPC, 287.1, or 95.7%, declared that they had heard about HIVIAIDS, an increase from 15.7% at baseline. HOPE'S response to the AIDS epidemic was primarily to attempt to reduce the number of HIV infections by focusing on increasing HIVIAIDSlSTDs awareness among Mozambicans in the Ile Province through providing information on how the disease is spread, and how it can be avoided. Posters, lectures and increasing the availability of condoms were the means used. The project started its HIVIAIDSlSTDs prevention campaign late in August 1999. Rather late into the project, however, meetings with community members did confirmed their level of knowledge about HIVIAIDSlSTDs reflected in the final KPC. Mozambique's adult HIV infection rate at the end of 1997 was estimated at 14.17% occupying the 7th place among African countries. The estimated number of AIDS cases in the country is 290,000 adults and children, the estimated number of deaths (cumulative) is 250,000. AIDS cases by age and sex are : Males 2O-29=3 1%; 30-39=34%; for females: 20-29=34%; and 30- 39=36%. During the evaluation meetings with the communities the evaluation team was informed that there were condom stock-outs or significant reductions at the Health Center (HCs). Of the 300 adults surveyed 37.7% mentioned that they got condoms from the hospital, 18% did so at baseline; 25% obtain them from health centerslposts 11% at baseline; 21.3% stated that they purchase them at the market , only 4% did at baseline. 3 1.3% did not know where to get condoms. Having to obtain condoms from health facilities is problematic for those who have to walk the approximately 13 mile average to get to the nearest center. The availability of Health Centers at present is 1 outlet per 3,500 people (1 5-49 years). Nevertheless, 44.59% reported using a condom in the final survey as opposed to 9.1 % at baseline. Percents for contraceptive prevalence vary depending on the source. The UNAIDS/WHO Fact sheet for Mozambique indicates that contraceptive prevalence in Mozambique is 4%. The International Planned Parenthood Federation (IPPF) presents a slightly higher figure of 5%. Nonetheless, contraceptive prevalence in Mozambique is extremely low when compared with other South African countries. The most widely accepted methods of contraception in the Province are the pill and Depo Provera. Some 84% women of reproductive age, when surveyed, had heard about FP methods. 85.3% of those interviewed in the final survey mentioned the pill as opposed to 57% at baseline. 74.2 mentioned injectables as opposed to 30% at baseline. Institutional Sustainability During this final evaluation it became apparent how very much involved the communities have been in this project, with the Community Health Workers (CHWs) and the HIV/AIDS/STDs PC playing key roles in project implementation and future sustainability. These health committees, according to the health centers' chiefs, played an instrumental role in helping to manage the health centers through which many of the project's services are provided. Again the center's chiefs emphasized that without the active participation of the health committees the health centers might not have perform as well or have been frequented as much. Likewise, the health committees helped to motivate village women to participate in and support health center services such as immunizations and DDS' activities. Similarly, these health outreach groups participated in training sessions and also promoted healthy child survival behaviors. These groups served voluntarily. A major achievement of this project, though not measurable in terms of CS indicators but in sustainability terms, is the commitment of the DDS' to continue to maintain and build on the project's accomplishments. The DDS Director in Ile indicated the DDS' commitment to continue Source: UNAIDS Epidemiological Fact Sheets. 5 providing financial and material resources to support the work of the health centers. The health outreach groups and the members of the community, although very committed to continue the work of the project, were also aware of the DDS' financial constraints so they expressed apprehension about the possibility of the work not continuing beyond the project's end. As of the final evaluation, the outreach community groups were still very motivated and when asked, they responded that they will continue their work. What they didn't know was until when for the reasons explained above. Training curricula, educational materials and pamphlets were some of the items requested by these groups to continue their work. These developed materials will also become a standard part of the DDS' work that their agents will continue to use after HOPE's funding ends. Another request made by the outreach groups was that of support from an organization like HOPE so that they could keep up-to-date with developments in the PWCS area. Our interpretation of this request was that they feared the MOH wouldn't be able to provide the kind of guidance, or the validation they perceived they would need. No predetermined sustainability plan was drawn-up by the DDS per se. However, during the first meeting with the DDS' Chief and his staff it was expressed clearly that: (1) the DDS will continue implementing project-initiated child survival activities in their corresponding areas; (2) that the DDS both at headquarters and at the Health Center level will continue meeting quarterly to both oversee and ensure continued provision of CS services by HC staff and CHWs; and (3) community health groups including CHWs, traditional birth attendants and traditional healers will continue to promote healthful behaviors among target populations (women of CBA, mothers and children.) Meeting with the Zambezia Province Governor The day prior to departure from Quelimane, the District's capital, HOPE's Regional Director for Africa, the Team Leader and the Project's Field Coordinator attended a meeting with the governor of the Province, Honorable Senhor Orlando Candua who had requested that we meet with him before departure. During this meeting Sehnor Candua informed us that he has been kept informed on all the activities carried out by Project HOPE in Ile, and expressed his thanks and his satisfaction with the work the project had done in his Province. During this very friendly discussion, Senhor Candua also explained to us in some detail the vast differences among ethnic groups within the same Province. These cultural differences andfor cultural barriers might impinge in implementing both FP and HIVIAIDS prevention projects. These differences, as he recommended, are to be taken into account when designing any future health project to be implemented in Zambezia. We were also informed that some funds had been earmarked by the United Nations to carry out health activities in the Zambezia Province and was very pleased to informed us that among the US-PVOs he had recommended as possible implementers was Project HOPE. The time frame for release of this funding and any other specifics about it were not disclosed. PART I. INTRODUCTION A. Project Goal and Objectives "Improving Mother and Child Health", a Child Survival project implemented by the People - to People Health Foundation, Inc. (Project HOPE) started May 1997 in the Ile District. Ile is situated in upper Zambezia, Northeast Mozambique. The project sought to decrease the mortality and morbidity in children under two and women of reproductive age, and to increase the involvement of local communities in developing solutions to local health problems5. To achieve this goal, a number of objectives were established and presented in a Detailed Implementation Plan (DIP.) Some of these objectives were later modified in preparation for the project's closing June 30th, 2000. Funds for the project were provided by USAID/Mozambique. The USAID $1,100,000 was matched by $366,6676 of Project HOPE'S private funds. Pipeline analysis was beyond the SOW. B. Geographiccoverage The Province of Zambezia is divided into fourteen (14) Districts. Each district is divided both along traditional lines and government administrative boundaries, with twenty-nine (29) chiefs spread over three administrative government posts comprised of fourteen (14) localities. The project has worked with a target population from 8 of the 14 localities and a population of approximately 103,000 inhabitants in the localities chosen. These include: Phalane, Mugulama, Socone, Muliquela, Carica, Mulevala, Nampevo and Namanda. The population distribution is as follows: 19,500 children under two years of age; 61,000 women of reproductive age and their partners; and 22,500 adolescents (pre-census data)7. The selection of the project sites was made in consultation with the local health authorities based on levels of need and community response. Project HOPE (HOPE) trained health centerslpost DDS staff in the eight areas described above in immunization, HIVIAIDSlSTDs prevention, child spacing and diarrheal diseases. Each locality serves the population of an average of 7.6 smaller villages in the surrounding area. Mozambique is a southern Afican country slightly less than twice the size of California, with a population of 19,124,335 (July 1999 est.). Ile is the country's most populated district, and Zambezia the most populated province. The major economic activity in Ile is family agriculture and there is very little commercialization of agriculture produce. Because very little commercial activity, there are no banking facilities in the District nor there are telephone lines that extend beyond the town of Mocuba. Electricity is severely rationed. 'Addendum to program description, preparation for close-out Project's Detailed Implementation Plan. ' Target population taken from the DIP 7 C. Country's Population data Data for Mozambique is not readily available. The data found in the research done during the writing of this report varies from source to source, in some cases significantly . Because of these variations, the data chosen for the report is the most recent according to the sources used. In some cases data from the World Fact Book(WFB); the International Planned Parenthood￾Mozambique (IPPF) ; UNAIDS-Mozambique; and the Congressional Research Service data were used: 9 Population growth rate 2.54% 9 Population under 15 years of age 46% 9 Birth rate 42.7511000 P Death rate 17.3 111000 > Infant mortality rate 1 17.5611 000 live birthss 9 Maternal mortality rate N1A9 9 Total fertility rate 5.88 children per woman'0 9 Women aged 15-49 using contraception (modern methods) 5% (IPPF) > ~vera~e lif expectancy for the total population in years 45.89; males 44.73 ; females 47.09 years. > Literacy 57.7% males and 23.3% females > 1 %-3% of the population have access to potable water in the Ile District; 63% countrywide. D. Country's Economic Data The civil war, which lasted thirteen years, devastated Mozambique's economy. People were left with little access to safe water, health services and education. In 1994 Mozambique ranked as one of the poorest countries in the world. Privatization of a large number of state enterprises has formed part of a series of economic reforms. Despite Mozambique's achievement of one of the highest growth rates in the world in 1997-98 (1 I%), it still depends on foreign assistance to balance the budget and pay for a trade imbalance in which imports outnumber exports by three to one. Mozambique is an economic aid recipient of US$1 .I15 billion (1 995 data.) Gross National Product (GNP) per capita is US$80. Media with the exception of radio, is limited. In 1992 it was estimated that there are 700,000 radios and 44,000 television sets in Mozambique. Data on unemployment, percent of labor force, and population below poverty line are not available. It is important to note that IPPF indicates a much higher rate of 13411000 IPPF indicates 15 12 number of deaths per 100,000 live births in a given year. 'O UNAIDS-Mozambique Epidemiological Fact Sheets set TFR at 6.3 8 E. Public Health Services Ministry of Health (MOH) facilities are sparely dispersed over the country, and the Ile District is no exception. A major newspaper article dated July 1"' 2000, stated that the MOH will be constructing an undisclosed number of health facilities and will be rehabilitating approximately three dozen others in the Zambezia Province. This capital investment is to be financed by the European Union with an estimated cost of US38 million. The health centerlinhabitants ratio should be one health post for every 20,000 inhabitants and one health center per 60,000 inhabitants by 2002.". F. Beneficiaries The distribution of the target population is as follows: 61,000 women of reproductive age and their partners, 19,500 children under the age of two (2), and 22,000 adolescents (pre-census data.) HOPE was the only US-PVO implementing a project in the Ile District. The estimated total population in the District is 2,000,000. The DIP estimated that 102,500 persons were to benefit from the project's services. G. Strategies and Partnerships The project's main partner was the DDS with whom the project worked very closely through out the project's life. Another large and very important partner were the Community Health Workers (CHWs). To achieve the stated objectives, the project provided training to its own staff, 130 Health Center staff, to CHWs, and to HIVIAIDSlSTDs Prevention Committees (HIVIAIDSlPCs). Training strengthened the provision of health services and enabled these groups to promote within the communities the primary MCH messages of the project. The project also provided equipment and supplies to the health centers to enhance service delivery, and facilitated transport for community outreach workers by providing bicycles. H. Surveys Two surveys were conducted during the project's life. A baseline survey, which took place in September 1997, and a final KPC survey in May 2000. The sample for both surveys consisted of 300 women with children 0-23 , months of age in 16 rural communities, and 300 adults of reproductive age in the same communities. Both Project staff and DDS staff participated in the collection and analysis of data. Two questionnaires were used to conduct the survey: one for mothers with children less than 24 months of age, and a second questionnaire for adults of reproductive age. Both surveys used the standard USAID thirty (30) cluster survey. According to the DIP "the baseline survey revealed very low levels of vaccination, with a DPT dropout rate of 45 % . Likewise knowledge IPPF Country Profile for Mozambique 9 of immunization calendars and effects was mostly lacking, e.g. only 4% knew that a child should receive measles vaccination at nine months"I2. In addition to covering the child survival topics of direct interest to the project, during the interviews for the final KPC both partners were asked questions related to the prevalence of use of modern contraceptives among the women of reproductive age. Both partners were interviewed. It is suspected that the presence of a male in the interview might have prevented the woman from expressing her own opinion freely, rendering the results to these FP questions ineffective. This might be the case particularly with those women who choose Depo-Provera as the preferred contraceptive. No mid-term evaluation was conducted on this project or prior to the Project Manager's departure. Such a survey could have provided worthwhile insights into the status of the project and enabled project management to develop strategies to more effectively attain the set objectives, providing better guidance in the implementation of the DIP for the new team. PART 11. EVALUATION POINTS The in-country portion of the final evaluation of HOPE'S Child Survival project was conducted from June through July 2"*, 2000, by an eight (8) member multi-disciplinary team. Team members included three project MCH nurses, a field coordinator, a field trainer, a field supervisor, the health information systems manager, HOPE'S Regional Director for Africa, the Country Director and an external evaluator who served as team leader. Upon arrival of the team leader, both HOPE Regional Director for Africa and Country Director discussed their suggested plan and time-table with the team leader. The evaluation timetable, which identified the clients to be interviewed, was agreed upon and the three team members headed towards Ile District to join the rest of the team. Once the team was assembled the scope of work was reviewed and a joint evaluation purpose statement was developed. The larger team was then divided into two, with each team surveying a sample of the project's partners and beneficiaries. A Child Survival KPC survey had taken place a month prior to the initiation of the final evaluation. A. Sources of Information The evaluation team reviewed a number of documents from the HOPE'S field office. A list of the documents reviewed includes: 1. Detailed Implementation Plan (DIP) 2. KPC Baseline Survey 3. Quarterly Reports 1 997-2000 - - '' Detailed Implementation Plan: Executive Summary 10 Project Personnel Training Schedule DDS Staff Training Schedule CHWs Training Schedule CHWs Training Curriculum Summary of Activities of the Health Brigades Socio-Economic Indicators, Zambezia Addendum to Program Description, Preparation for Close-out Project Design Revision Rapid Knowledge, Practice & Coverage (KPC) questionnaire Final KPC Survey Report (Draft) Personal Interviews Eight communities and health centers were chosen to conduct group interviews with HCWs and HIVIAIDSlPCs and community members. One initial meeting took place the day after arrival in Ile with the DDS' Director and his staff. Four days were spent conducting interviews in Socone, Muliquela, Phalane, Mugulama, Chiraco, Mulevala, Nampevo, and Namanda. Approximately eighty community members and fourteen outreach health workers and HIVIAIDSISTD PCs participated in these meetings. (See Annex B for a schedule of the evaluation. The purpose of selecting a number of key people (Community Health Workers; HIVIAIDSlSTDs Prevention Groups and members of eight (8) communities) to interview during this final evaluation was to obtain qualitative information and their perspectives to help the evaluation with the interpretation of the "quantitative" data the project collected in its Final Survey; obtain descriptive information; and generate recommendations to help the project understand its service users as well as those that received training through the project's training program. Those interviewed were selected for their first hand knowledge of the project. A simple questionnaire was developed with five (5) questions relating to the training, the service and the quality of the service during the implementation of the project. On a daily basis, upon concluding interviews, team members regrouped to discuss and synthesize the information into the main areas of concern specified in the SOW. Among the team members, consensus was reached regarding the needs expressed by the various community groups. On the last day of the evaluation, a meeting with the DDS staff was to take place. Unfortunately, the DDS' Director had been called the prior day to another part of the Province so this scheduled meeting didn't take place. Both the Field Coordinator and the Country Representative were requested to meet with the DDS' Director to inform him of the evaluation results. The day prior to departure from Quelimane, the District's capital, HOPE'S Regional Director for Africa, the Team Leader and the Project's Field Coordinator attended a meeting with the governor of the Province, Honorable Senhor Orlando Candua who had requested that we meet with him before departure. During this meeting Sehnor Candua informed us that he has been kept informed on all the activities carried out by Project HOPE in Ile, and expressed his thanks and his satisfaction with the work the project had done in his Province. During this very friendly discussion, Senhor Candua also explained to us, in some detail, the vast differences among ethnic groups within the same Province. These differences andor cultural barriers might impinge on the implementation of both FP and HIVIAIDS prevention projects. These differences, as he recommended, are to be taken into account when designing any future health project to be implemented in Zambezia. We were also informed that some hds had been earmarked by the United Nations to carry out health activities in the Zambezia Province and the governor was very pleased to informed us that among the US-PVOs he had recommended as possible implementers was Project HOPE. The time frame for release of this funding and any other specifics about it were not disclosed. PART 111. DISCUSSION OF PROJECT ACHIEVEMENTS A. Project's First Phase - May 1997 through December 1998 The country agreement with the MOH and the registration of HOPE as an NGO took place during 1996. The project, which was due to begin in May 1997, got off to a slow start with significant child survival activities being initiated rather late in to the first phase of the project. The first seven months of the project's life were spent finding suitable quarters for the project both in Mocuba and in Errego Ile Town); staffing; conducting the baseline KPC survey; procurement of equipment such as computers and two vehicles, four motorbikes and a electricity generator. Unfortunately, the vehicles were omitted from the planned MOH duty exemptions causing delays in obtaining the proper documentation for circulation of the vehicles, and forcing the Project to utilize the vehicles on a per month permit basis, though the lack of permanent circulation permits didn't seem to have affected the vehicles circulation. It is important to note, that the project's initiation was hampered by a series of unfavorable factors including less than aggressive project management and serious difficulties recruiting staff to fill the project's posts. A good number of candidates interviewed to fill the project's available posts were either dismissed due to lack of skills, or they resigned. The then Project Manager did not want to recruit personnel that wanted to leave the MOH to work for the project. As of the December 1997 recruitment of technical staff had not fully materialized, only an MCH nurse had been recruited. Nevertheless, during the first phase of the project, a logistical base was established in Mocuba the nearest big town next to Errego (Ile) where the project was based. At least in two initial quarterly reports the Project Director complained about the choice of Mocuba as opposed to Quelimane as HOPE'S logistical base. Apparently the choice of Errego as the town where the project was to launch its activities impinged on personnel recruitment because of its isolation. There are no banking facilities in the area due to lack of economic activity and communications and poor electrical supply interfered with the good hctioning of the office equipment. Until December 1997, HOPE's office in Mocuba was still having communications problems due to a faulty telephone system. It wasn't until the quarter July- September 1998 that the communications problem was solved by purchasing four VHF radios. Other less than favorable arrangements were made at the project's start. For instance, an attempt made to accommodate HOPE's base within the Ile District Hospital compound was unsuccessful. Just prior to the conclusion of two adjacent rooms in the district hospital, the District Administrator decided to halt HOPE's building rehabilitation work because the rehabilitated rooms were to become the police station. After extensive negotiations, these rooms were finally made available. B. Technical Activities The technical activities that took place between January 1997 and December 1998 seemed to be either ad hoc, or not linked to the activities planned and presented in the DIP. At times, one has the impression that the project did not take a pro-active stance but a reactive one. Some of the activities the Project Director participated in were piggy backed on activities planned by other organizations but whose timing may not have been appropriate. An example of this is the project's participation in training of elementary nurses and midwives offered by University Research Corporation (URC). The project did not have any nurses or midwives employed as yet, so the training was attended by the Project Director and two administrative staff. Such a training would have been better suited for nursing staff, once hired. When reading, one has the impression that there were a series of activities in which the Project Director felt an obligation to participate. These took a good per cent of the Project Director's time, although they did not form part of HOPE's work plan. Lack of focus on the actual planned activities seem to have been a major problem. It is worth noting though that many discussions and management plans were made during this period which did bear fruit much later in the life of the project. Another lesson learned from the first phase of the project is that a project with a busy agenda like HOPE's project should not wait eighteen months to dismiss an ineffective Project Director. While HOPE attempted to remediate the situation by hiring a expatriot deputy coordinator; conducting 5 senior level technicdadministrative visits within the first year; and hiring two external consultants to assess and provide a project improvement plan, none of these strategies were successful in addressing the fundamental problem. The project could have made significantly more progress under better leadership and if field activities would have taken place when they should. A detailed List of Activities covering the first phase of the project from May 1997-December 1998 is found on a table on "Annex C". Comments in relation to the activity carried out are in bold letters. - Immunization Activities during this period A review of the Table of achievements for this first phase, shows less than expected vaccination results for the reasons already explained above. However, some important accomplishments of the project's first phase were: the establishment of the health committees which are still fimctioning; the establishment of a good relationship with the Ile DDS through the support of their vaccination campaigns and logistical support during the cholera outbreak; training of DDS staff; the establishment of regular planning and problem solving meetings with the DDS staff; establishment of the Project's quarters in Ile; resolution of the communications problems; and initiating the Health Information System (HIS.) These achievements were mostly administrative rather than technical. A total of seventeen (1 7) Mobile Vaccination Brigades (MVB)s were operating between July￾September. Although the number of mobile vaccination brigades exceeded those planned in the DIP for FY 1 and FY2, vaccination figures fell short of goals in part due to stock-outs at the Provincial level of DPT, TTV and BCG during August and September. Other factors played a role in this short fail, such as inadequate mobilization prior to the health committees being established, and a disinformation campaign in some guerrilla influenced areas. HOPE did take some steps to minimize the problem of stock-outs as it did with the cold chain problem of shortage of fuel. The actual vaccination campaigns started in September-December 1998. A total of 18,734 children 0-1 1 and +12 months received BCG, Polio 1,2,3 and Measles vaccines during the fourth quarter 1998. Out these total figure 9,042 were immunized by the project's Mobile Brigades with the support of the health committees' increased mobilization. This represented a project contribution to the campaign of 48.3%. A total of 867 women received TTV. 44% got a second doses as opposed to the 6,000 that were supposed to be vaccinated the first year. The widespread stock-out of key vaccines at national level ended towards the end of October. These figures fall below the project's first year target. - TTV activities The project's contribution to TTV coverage during this first phase was also limited due to the unavailability of the vaccine. A total of 2,270 pregnant women received 1"' Td, 3rd, and 4' doses during the fourth quarter 1998. A total of 900 were immunized by the project's Mobile Brigades with the support of the health committees' increased mobilization. This represented a project contribution to the campaign of 39.6%. - MCH Activities Pre-and Post natal consultation in the District amounted to 6,069 during the 4' quarter 1998. - Family Planning Services The District provided services to 3,745 women. Only two users accepted the IUD the rest opted for Depo-Provera and the pill, which are the most popular and widely accepted contraceptives. HOPE, through the mobile brigades distributed pills to 454 women and Depo-Provera to 456. During the 4" quarter 1998. Health talks were conducted in sixteen communities. A total of 2,850 participants received information on child spacing, FP, vaccination, nutrition and sanitation, and 668 women received pre-natal examinations. It is important to note that nutrition, sanitation, growth monitoring, children consultations were conducted by MOH members of the MVBs. The health committees made 989 home visits in total to 558 children and 256 pregnant women. At the end of these visits the committees concluded that the majority of houses have neither a latrine nor kitchenware stand, and many children did not have a health card. In most communities people did not have access to potable water, and inadequate care was taken of the traditional wells. According to the quarterly report there was a 5% reduction in trained members of the health committees due to the agricultural season which made their regular contribution difficult. This poses the question of weather working with health committees composed only of volunteers is as productive as working with paid staff. Polio eradication campaign took place in Ile between August 24" and October 2"d, 1998. During this campaign a total of 64,975 children were vaccinated. Comparable figures were obtained for the second dose. - HIVtAIDSISTDs Activities No activities took place in this area of health during the first phase of the project. C. CONCLUSIONS HOPE'S "Improving Mother and Child Health in Ile District" project suffered from ineffectual project management during its first eighteen months. According to project staff and colleagues, the first Project Director who also served as HOPE'S Country Director, was technically familiar with child survival concepts. However, it was reported that he wasn't strong in conceptualizing, long term planning, and day-to-day organizational needs of the project. The lack of these skills hampered project progress. In addition to these internal difficulties, some of the inputs that the project partners were expected to provide were not forthcoming. The project experienced vaccines stock-outs which ended in October 1997. In addition, health centers were not equipped with scales and functioning refrigerators. In synthesis, looking at the project's activities during those eighteen months in a positive light one can conclude that those initial months were spent preparing the ground so that the technical activities described in the DIP could be implemented. However, the time frame utilized for preparation purposes was too long; few technical activities were carried out; and the majority 15 where either ad hoc activities or extremely delayed due to the persistent difficulties the then project Director had in finding and recruiting qualified staff. In response to these obstacles, HOPE gave notice at the end of 1998 to the first Project Manager. A second Project Manager was appointed. A new team composed of the new Country Representative and a Program Manger, both physicians, arrived in the country on March 14~, 1999. The new Country Representative was a Public Health specialist with an extensive experience working with NGOs and government institutions, and the Program Manager was a Child Health specialist with many years of experience managing PHC and CS programs in rural Africa. The new team initiated an accelerated strategy in collaboration with the DDS and ACRE the project's partner. PART IV. DISCUSSION OF PROJECT ACHIEVEMENTS A. Project Second Phase - January 1999 through June 2000 A total of 142 Mobile Brigades were carried out between January 5' and June 16", 2000, which provided services to women and children from 4013 communities in the Ile District. The following is a Table showing a summary of the various services provided by HOPE during the above mentioned period: l~accination Data Jan-June, 2000: Vaccinated Children: 14,442 Pre-natal Consultations: 3,168 Post-natal Consultations: 669 FP - Pill 1,394 FP - Depo-provera 2,214 ITTV- Pregnant 1" doses 2,481 TTV- Pregnant 2"* doses 871 WFA - 1" doses 6,084 WFA - 2"d doses 2,895 WFA - 3rd doses 544 TTV - Other 563 TOTAL SERVED: 35,325 Data Source: Resumo das actividades de PAV e SMI das Brigadas Moveis prepared during the evaluation by HOPE's staff. Only 14 pregnant women returned for the 3rd; 4'; and 5th TTV doses, and 24 WFA returned for the 4' and 5' doses. " Resumo das actividades e PAV e SMI das Brigadas Moveis prepared during the final evaluation by HOPE's Ile staff. 16 - Immunization HOPE'S focused on providing immunization delivery by training CHWs and DDS' health center staff. The CHWs serve as health promoters in the localities where the project works. The job of this group is to motivate women to get their children vaccinated either by attending the health center vaccination day or the mobile vaccination clinic held monthly at the village level. - Findings A final KPC Survey Report has been prepared by HOPE and is attached. This reports contains a complete analysis of the final survey results. The evaluation report will only be presenting a summary of relevant finding made during the final KPC. The project surpassed its project vaccination objective. The per cent of children who ever received some type of vaccine ever was recorded at 8 1.4% of 301 surveyed mothers. Children with immunization cards increased to 72% at final survey from 44% at baseline. Almost half of children with immunization cards are vaccinated with DPT3 and OPV3. Complete coverage rates for BCG, DPT3,OPV3, and measles increased significantly from 15% at base line to 41% at final KPC for children with immunization cards. DPT dropout rates among children 12-23 months of age was calculated at 3 1. 1%' and at 33.6% for OPV. TTV coverage rates although low at the final survey, they reflect a significant increase from baseline: 46.7% of women of reproductive age at final survey as opposed to only 9% at baseline had one TTV doses. 26.3% of women had received two doses , a significant increase from the 4% at baseline. These achievements are particularly significant given the stock out of vaccines experienced in the Province during the first part of the project, and the short period of time (1 8 months) the project had to reach its vaccination targets. - Diarrheal Disease Control The final KPC survey found that 59.8% of the 301 mothers interviewed stated that their children had had diarrhea within the two weeks prior to the survey. 49.2% of mothers interviewed recognized weakness and tiredness as a symptom to seek advice or treatment for diarrhea. 37.9% mentioned signs of dehydration as a symptom for a diarrhea episode. These figures clearly indicate that mother's knowledge of diarrhea case management did improve as a result of the project's interventions. The strategy used to achieve this objective focused on training DDS staff and community outreach groups to promote behavioral changes among mothers, and to promote the use of Oral Rehydration Therapy (ORT) and sugar salt solution SSS. Exclusive breast feeding as well as appropriate feeding practices during diarrhea episodes were also important messages conveyed to the health outreach groups and the DDS staff with the expectation that these groups would teach women in the communities. 17 All community members interviewed by the evaluation team expressed their gratitude to HOPE for the training received, emphasizing that they had learned a lot from them. Employing volunteers has its advantages especially fiom the financial standpoint. However, project staff are unable to properly supervise the CHWs due to lack of authority, making quality control of training practically impossible. This is a factor that should be taken into account in a future design of a CS project. Some of the CHWs complained of not having sufficient educational materials. As a result, awareness raising activities were limited to informal talks. Findings Changing feeding habits especially among traditional ethnic groups, is a formidable task, and therefore it is not surprising that achieving this objective was difficult. The final survey indicated that 38.33% of women used Oral Rehydration Solution (ORS) sachets as opposed to 23% at baseline. Women who were able to correctly describe ORT preparation increased from 5% to 17.27% in the final KPC survey. The members of the community interviewed by the evaluator said that more women do prepare the SSS when their children have diarrhea, and when asked, all respondents were able to explain how to prepare the SSS. It was agreed among those groups that due to the project interventions more mothers were using SSS than before. Of the three hundred adults interviewed during the final KPC 287.1 or 95.7% declared that they had heard about HIVIAIDS, an increase of 15.7% fiom baseline. - Findings HOPE'S response to the AIDS epidemic was primarily to attempt to reduce the number of HIV infections by focusing on increasing HIVIAIDSlSTDs awareness and by providing information on how the disease is spread and how it can be avoided. Posters, lectures and increasing the availability of condoms were the means used. The project started its HIVIAIDSlSTDs prevention campaign late in August 1999. Rather late into the project, however, meetings with community members did confirmed their level of knowledge about HIVIAIDSlSTDs reflected in the final KPC. A follow-on project should continue with a much more aggressive campaign in this important public health area. Of the eight communities visited, one did not have an HIVIAIDSlSTDs committee, and another community had a committee compassed of very young people, who most likely could only transmit their knowledge to their peers, very doubtfully to adults. Among many of the comments made by both community members and health outreach groups was that people don't seem to believe that AIDS actually kills people, although the knowledge about the disease is quite high among these communities. On various occasions it was suggested that a way be found to show these communities people suffering with the decease, they felt that this was the only way the communities would believe it. Statistical data on the number of persons that have been decimated by the decease or the number of persons infected with the virus are scant. UNAIDS data from 1997 was the most recent data found by the evaluator. In Mozambique the adult HIV infection per cent of the total population was estimated at the end of 1997 to be 14.17% l4 occupying the 7th place among African countries. The estimated number of AIDS cases in the country is 290,000 adults and children, the estimated number of deaths (cumulative) is 250,000. AIDS cases by age and sex are : Males 20- 29=3 1%; 30-39=34%; for females: 20-29=34%; 30-39=36%. During the evaluation meetings with the communities we were informed that there were stock￾outs or significant reductions at the HCS in the amount of condoms readily available. Of the 300 adults surveyed 37.7% mentioned that they got condoms from the hospital, 18% did so at baseline; 25% obtain them from health centerslposts 11% at baseline; 21.3% stated that they purchase them at the market only 4% did at baseline. 3 1.3% did not know where to get condoms. Having to obtain condoms from health facilities represent a problem for those who live at a long distance from the center. It is estimated that some users have to walk approximately 13 miles to get to the nearest center. The availability of Health Centers at present is 1 outlet per 3,500 people (1 5-49 years). Nevertheless, 44.59% reported using a condom in the final survey as opposed to 9.1 % at baseline. Like other African countries, Mozambique suffers from a variety of economic and social factors that could only exacerbate the epidemic. Poverty for example deprives Mozambique of an effective health information system, health education, and health care. No record was kept of how many people were treated for STDs during the project's life. However, It is possible that a high rate suffer from untreated sexually transmitted infections (STIs) other than AIDS, and these are believed to open the way to infection by HIV. Cultural and behavior patterns may also be affecting the epidemic. The proportion of people presenting with a STD in health facilities assessed and treated in an appropriate way (according to national standards) is 13 1,115" Church groups and other humanitarian organizations were approached to request collaboration in spreading the word about the project's prevention program. A new strategy would require an aggressive program that would continue with the prevention campaign already started by HOPE and many other organizations operating in Mozambique. Poverty, lack of accessibility to health care, and lack of prevention programs are quite a combustible mixture that could, if not checked, bring fatal consequences. l4 Source: UNAIDS Epidemiological Fact Sheets. l5 UNAIDS/WHO Epidemiological Fact Sheet for Mozambique. 19 - Family Planning (FP) Per cents for contraceptive prevalence vary depending on the source. The UNAIDS/WHO Fact sheet for Mozambique indicates that contraceptive prevalence in Mozambique is 4%. The International Planned Parenthood Federation (IPPF) presents a slightly higher figure of 5%. Nonetheless, contraceptive prevalence in Mozambique is extremely low when compared with other South African countries. Findings The most widely accepted methods of contraception in the Province are the pill and Depo￾Provera. Some 84% of women of reproductive age had heard about FP methods, and 85.3% of those interviewed in the final survey mentioned the pill as opposed to 57% at baseline. 74.2 mentioned injectables as opposed to 30% at baseline. - Monitoring the Quality of Interventions Although HOPE trained the DDS staff and worked very closely with them, there wasn't a monitoring plan per se. HOPE and the DDS divided the territory so each group went its own way on the Mobile Brigades. The Country Director stated that he personally went with the DDS staff in many Brigades and Vaccination campaigns. The idea is that the DDS staff have gone in all Mobile Brigades together with the HOPE staff. For the purpose of determining what lessons could be learned from a joint outreach work (HOPEIDDS), and to ensure both Institutional sustainability and sustainability of Quality of Services, joint work between this governmental body and an PVO could have been useful for future projects. PART V. SUSTAINABILITY A. Community Participation During this final evaluation it became apparent how very much involved the communities have been in this project, with the CHWs and the HIV/AIDS/STDs PC playing key roles in project implementation. These health committees, according to the health centers' chiefs, played an instrumental role in helping to manage the health centers through which many of the project's services are provided. Again the center's chiefs emphasized that without the active participation of the health committees the health centers might not have performed as well or frequented as much. Likewise, the health committees helped to motivate village women to participate in and support health center services such as immunizations and DDS activities. Similarly, these health outreach groups participated in training sessions and also promoted healthy child survival behaviors. These groups served voluntarily. It is worth noting that many members of these communities were under the impression that HOPE was responsible for building infrastructure. There were many requests from community members and community outreach groups alike for building of new health centers closer to their own communities, as well as transportation and equipment. HOPE staff clarified to these community members that the responsibility for building and equipping centers/posts was that of the MOH. B. Ability and Willingness of Government Institutions A major achievement of this project, though not measurable in terms of CS indicators, but in sustainability terms, is the commitment of the DDS' to continue to maintain and build on the project's accomplishments. The DDS Director in Ile indicated the DDS' commitment to continue providing financial and material resources to support the work of the health centers. The health outreach groups and the members of the community, although very committed to continue the work of the project, were also aware of the DDS' financial constraints so they expressed apprehension about the possibility of the work not continuing beyond the project's end. For instance, one of the major complaints received by the evaluation group was that of lack of transport. As of the final evaluation, the outreach community groups were still very motivated and when asked, they responded that they will continue their work. What they didn't know was until when for the reasons explained above. Training curricula, educational materials and pamphlets were some of the items needed for these groups to continue their work. These developed materials will also become a standard part of the DDS' work, which their agents will continue to use after HOPE'S funding ends. Another request made by the outreach groups was that of support from an organization like HOPE so that they could keep up-to-date with developments in the PWCS area. Our interpretation of this request was that they feared the MOH wouldn't be able to provide the kind of guidance, or the validation they perceived they would need. C. Health Information System (HIS) The Health Information Systems whose development started during the first phase of the HOPE project, was completed and was fully operational and ready to be turned over to the DDS at the time of the final evaluation. As far as the usefulness of the system to the project itself, the only observation made about the HIS is that the system was modeled exclusively to be DDS driven, that is, the system compiles data based on the needs of the DDS which only require that the system provides information on a global basis. For the project, this kind of information which lacks detailed -- e.g. by intervention, population served by community, etc.-- is not very useful for managerial purposes. Both the Regional Director for Africa and the Team leader were assured by the Project Manager that the information generated by the HIS had been sufficient. D. Sustainability Plan No predetermined sustainability plan was drawn-up by the DDS per se. However, during the first meeting with the DDS' Chief and his staff it was expressed clearly that: (1) the DDS will continue implementing project-initiated child survival activities in their corresponding areas; (2) that the DDS both at headquarters and at the Health Center level will continue meeting quarterly to both oversee and ensure continued provision of CS services by HC staff and CHWs; (3) community health groups including CHWs, traditional birth attendants and traditional healers will continue to promote healthful behaviors among target populations (women of CBA, mothers and children.) PART VI. LESSONS LEARNED The following lessons learned have been gleaned from this evaluation effort upon reflection of the entire project. Only lessons that are deemed relevant to PVO Child Survival and/or to USAIDys support of these projects have been included here: 1. HOPE's training and continued support of the community outreach teams was definitely one of the best strategies this project had. The outreach teams themselves were the biggest asset to the communities and the project. The outreach teams ensured the mobilization of the communities so they were readily assembled and awaiting the vaccination and Mobile Teams. Although, there are communities that are quite populated like Socone, the population in the District is quite dispersed. Without these community outreach teams it would have been impossible to have had the success in delivering the services the project delivered. 2. A lesson learned from the less than productive first phase of HOPE's project is that those in charge of a field project at Headquarters should be aware that a project with a busy agenda like HOPE'S Ile District project should not wait a lengthy period of time -- e.g. eighteen months - to dismiss an ineffective Project Manager. Judging by the tremendous success of the project once a new team took over one could reckon that a much greater impact would have been made by this project if field activities would have started with the same drive as when the new team took over. 3. Sustainability of a Child Survival project depends a great ideal on a strategy that focuses on strengthening the project's partners institutionally and in the provision of quality services. The creation of a parallel service delivery system prevents a project's partner(s) fiom having the opportunity to experience together ways in which a service delivery mechanism can be implemented. The project trained staff fiom the MOH but did not work together in delivering services. Instead each group took its own area of activity. With blended teams, HOPE could have handed over the leadership of the project to the DDS well before the end of the project's final cycle of USAID finding. A phase out plan and timetable should have been developed working closely with the DDS, the Health Centers and the Health Committees. 4. Obtaining the information necessary for each of the key players of a health system is the main purpose of the HIS. The systems provides information that serves management in measuring the success and areas of need within the project and even within the communities. The HIS developed by HOPE was driven by the DDS' information needs exclusively which focuses on reporting on health services to the provincial level. Obviously, the concerns are whether or not the HIS will be sustainable now that the project has ended and the system is being turned over to local leadership. Unless, there's a modification to the system, the DDS will continue to generate, global data as opposed to detailed data for managerial purposes. 5. Making capital investments within government structures is not good business because an investment such as this does not guarantee the organization that the capital investment will be recuperated during the life of the project. Once the project is concluded and the premises have to be vacated, the US-PVO does not have any control of what the government decides to do with the vacated premises. This is true particularly if there is a change in the government. PART VII. RECOMMENDATIONS The following are the evaluation's recommendations for future Child Survival projects in the Ile District, Zambezia Province: 1. The donor must ensure that the gap left by this project in the Ile District is filled by a new project to start as soon as possible to ensure that the investment already made in both financial and human resources is not lost. 2. Increase the HIVlAIDSISTDs volunteer base to include teen age peer groups and adults. 3. Any HIVIAIDSlSTDs prevention program designed for Mozambique should be aggressive and should ensure a greater availability of condoms so that the population can obtain them from sources other than the Health Center. There are three theater groups operating at present. These theater groups should be supported and expanded to other communities. The use of multi-media technology and the coordination of activities with organization that support HIVIAIDS prevention programs is of outmost importance. 4. From a qualitative perspective there was an articulated need for family planning that is not being met. Additional up-front assessment of cultural barriers should be done to identify barriers that could help in updating family planning service delivery that include a FP service delivery design that takes into account the constrains imposed by the believes of the ethnic groups in the Ile district. 23 5. A means of transport such as bicycles must be arranged for the members of volunteer groups. The average distance between a health center and a community is 13krn. If bicycles are the means of transport chosen, these should be sold to the volunteers at a subsidized price to make them accessible to as many volunteers as possible. 6. A future project should maintain a HIS that incorporates detailed data by intervention on the population served. 7. During the final evaluation interviews, a limited participation of women in health Committees was observed. Again cultural barriers should be identified if these are the reason for the absence of women in the outreach committees. 8. A future project should ensure that joint (ProjectDDS) community work is done, and that the DDS staff is included in the project's Health Brigades. Project staff should be able to train and supervise the activities of the DDS staff while they're providing service delivery and prevention activities so that capacity in the provision of services is installed within the DDS. 9. While infrastructure is upgraded by the MOH the very serious need for services that exist in the District's rural communities could be fulfilled by NGOs like HOPE. PERSONS CONTACTED District Health Management Board Senhor Eduardo Arce Director Distrital (District Director) Ms. Acia Rafique MCH Nurse Zelia Raibo Marrune Responsable NEP Lourenzo Mamudy Chiel Nurse Joao Ernesto Vafinde Pharmacist Luis Alfandega Laboratory Ricardo Male HIVIAIDS Clinic HOPE's Ile Staff Dr. Arturo Sanabria, MPH Dr. Carlos Domingos Claudia Goveia Celia Sendela Tania Lucinda Benjamin Dove1 Placido Eduardo Cardoso Albino Ismael Ibraimo Joao Victoriano AMETRAMO, Representative HOPE's Country Representative Field Coordinator MCH Nurse MCH Nurse MCH Nurse Field Supervisor HIS Secretary Field Trainer Driver Driver Association of Traditional Healers Health Centers' Chiefs. Communitv Health Committees and Members of the following communities: Socone Muliquela Phalane Mugulama Chiraco Mulevala Nampevo Namanda Honorable Senhor Orlando Candua Governor, Province of Zarnbezia