CS-16 Mozambique Final Evaluation Cooperative Agreement No.: FAO-A-00-00-00036 The Strength Project Strengthening Districts and Community Teams for Mother and Child Health in Northern Mozambique 30 September 2000 – 30 September 2003 Submitted By: Save The Children Federation, Inc. 54 Wilton Road Westport, CT 06880 Telephone: 203/221-4000 Fax: 203/221-4056 Report Prepared By: Armand L. Utshudi, External Consultant and Team Leader Salazar Portugal, District Director of Health, Navala-a-Velha Ellen Warming, SC, Health Program Manager Greta Stina, Field Program Coordinator Contact Person: Eric Swedberg, MPH, Child Survival Specialist December 31, 2003 CS-16, Mozambique, Final Evaluation Save the Children 2 ACKNOWLEDGEMENTS The FE team congratulates the efforts of the provincial health officials and those of the District Health Offices (DHO), and the local partners for their active participation in the planning, implementation, and supervision of project activities that have contributed to the improved quality of life in Nacala-a-Velha, and Memba Districts of Nampula Province. The team expresses sincere appreciation for the cooperation and support received from the MOH/Maputo, USAID/Mozambique, and the Provincial Director of Health/Nampula during the course of the evaluation process. The team extends its appreciation for the collaboration and support received from Eric Swedberg, Child Survival Specialist, SC/HO; Ellen Warming, Health Program Manager, SC/MZ, Greta Stina, Field Program Coordinator, SC/MZ, and the District Health Directors of Memba and Nacala who participated actively in the planning and implementation of the evaluation activities in the districts. In addition, the team wishes success to all those who participated directly and indirectly in the achievement of the objectives and the goals of the Northern Nampula Child Survival Strength Project. CS-16, Mozambique, Final Evaluation Save the Children 3 ACRONYMS AND TERMS ANC Ante-natal Care APE Basic Level Nurse ARI Acute Respiratory Infection A&T Systems Alarm & Transport Systems BCC Behavior Change Communication BCG Tuberculosis Vaccine CDD Control of Diarrheal Disease CHT Community Health Team CLC Community Leaders Council CS Child Survival CS-12 Child Survival-12 CS-16 Child Survival-16 CY Calendar Year DAP Development Area Program DCHA Bureau for Democracy, Conflict, and Humanitarian Assistance of USAID DDS District Health Director DHO District Health Office DPS Provincial Health Director DIP Detailed Implementation Plan DPT Diptheria/Pertussis/Tetanus Vaccine EGP Provincial Management Team EMOC Emergency Obstetrical Care EPI Expanded Program of Immunization FE Final Evaluation FP Family Planning FS Food Security HC Health Center HFA Health Facilities Assessment HIS Health Information System HIV/AIDS Human Immuno-Deficiency Virus/Acquired Immune Deficiency Syndrome HMIS Health Management Information System HO Home Office of Save the Children HP Health Post HPN Health, Population and Nutrition IEC Information, Education and Communication IMCI Integrated Management of Childhood Illnesses ITN Insecticide Treated Nets KPC Knowledge, Practice and Coverage Survey MCH Maternal and Child Health MOH Ministry of Health NGO Non-Governmental Organization ORS Oral Rehydration Solution ORT Oral Rehydration Therapy CS-16, Mozambique, Final Evaluation Save the Children 4 Palestras Presentations or discussion groups, educational “talks” PLG Program Learning Group PMT Program Management Team PVC Office of Private and Voluntary Cooperation of USAID PVO Private Voluntary Organization QOC Quality of Care Regulado Domain of a Régulo (also known as village) SC Save the Children Federation, Inc. SC/MZ Save the Children/Mozambique Field Office Soccorista Volunteer, curative care community worker/first aid supplier, trained by MOH-lower level than APE STI Sexually Transmitted Infection TA Technical Assistance TB Tuberculosis TBA Traditional Birth Attendant TT Tetanus Toxoid UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USAID United States Agency for International Development WHO World Health Organization CS-16, Mozambique, Final Evaluation Save the Children 5 TABLE OF CONTENTS Page EXECUTIVE SUMMARY - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 7 I. INTRODUCTION - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 10 A. Purpose of the Evaluation B. Scope of Work for the Evaluation C. Methodology D. Results: Summary Table of Baseline and Final KPC II. TECHNICAL APPROACH - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 13 A. Brief Overview of the Project B. Program Goals/Objectives C. Progress Towards Achievement of Objectives by Intervention Area III. CROSS CUTTING APPROACHES - - - - - - - - - - - - - - - - - - - - - - - - - - - - 24 A. Community Mobilization B. Communications for Behavior Change C. Capacity Building Approach D. Sustainability Strategy IV. PROGRAM MANAGEMENT - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 31 A. Planning B. Staff Training C. Staff Supervision D. Human Resources and Staff Management E. Financial Management F. Logistics G. Information Management H. Technical and Administrative Support V. OTHER ISSUES IDENTIFIED BY THE TEAM - - - - - - - - - - - - - - - - - - 35 VI. CONCLUSIONS AND RECOMMENDATIONS - - - - - - - - - - - - - - - - - - 36 A. Overall Achievement of Objectives and Project Status B. Important Achievements and Constraints C. Lessons Learned D. Recommendations for USAID/DCHA/PVC E. Recommendations for Partners F. Plan for Dissemination of Information by PVO/HO VII. RESULTS HIGHLIGHT - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 40 CS-16, Mozambique, Final Evaluation Save the Children 6 VIII. ATTACHMENTS A. Team Members and Their Titles B. Final Evaluation Schedule C. List of Persons Contacted and Interviewed D. Final Evaluation Questionnaires Table 1: Progress in Achieving Objectives in Nacala-a-Velha District Table 2: Progress in Achieving Objectives in Memba District Table 3: Summary of Accomplishments to Date Table 4: Summary of Accomplishments to Date Under Immunization Table 5: Summary of Planned Vaccination Activities Table 6: Summary of Accomplishments to Date Table 7: Summary of Accomplishments to Date Table 8: Summary of Accomplishments to Date Table 9: Summary of Accomplishments to Date Table 10: Summary Table of Progress with Capacity Building Related Activities Table 11: Summary of Accomplishments to Date Table 12: Summary of Accomplishments to Date Table 13: Summary Table of Short-Term TA Support to the Project Table 14: Summary Table of Project Status and Impact Figure 1: Map of the Project areas of Nacala-a-Velha and Memba Figure 2: Bicycle Ambulance with Sunshade CS-16, Mozambique, Final Evaluation Save the Children 7 EXECUTIVE SUMMARY During the past three years, Save the Children Federation, Inc. (SC) has been implementing a follow-on, Child Survival (CS-16) Project for Nacala-a-Velha and Memba Districts that are among the most poor and inaccessible districts in Mozambique. The project responds directly to the needs identified by the target population as well as the MOH and USAID’s commitment to reducing under five and maternal morbidity and mortality caused by the most prevalent public health problems including malaria, anemia, Acute Respiratory Infections (Pneumonia, and T.B.), HIV/AIDS, malnutrition, and diarrhea. The three-year project goal is to sustainably reduce under-five mortality and maternal mortality in the two target districts of Memba and Nacala-a-Velha through the following: (1) Improving the capacity of the health districts to implement CS approaches and support community structures; (2) Improving the capacity of the communities to identify and respond to their health needs; (3) Increasing the use of key health services and improving CS practices at the household levels; (4) Increasing the capacity of SC to achieve large scale innovative CS programs in the Southern Africa setting; and (5) Informing Nampula Provincial Health Office of innovative CS strategies. The project has made excellent progress in establishing and expanding vaccine coverage for children under-five years old (fully immunized children aged 12-23 months increased from an estimated 32% at baseline to 56% at final), and in providing prenatal and postnatal care in the focus districts. To date, most of the participating health facilities have trained health workers (elementary nurses, APEs, and MCH nurses). Mobile Brigade activities that include EPI and provide prenatal and postnatal care have been enhanced through improved transportation to facilitate service delivery in areas that cannot be adequately covered by functioning health centers and health posts. Trained APEs are providing limited curative care and are referring patients to appropriate primary referral facilities in the districts, and trained TBAs are assisting mothers during delivery (births attended by trained personnel increased from 34% in Nacala-a￾Velha and 38% in Memba at baseline to 65%). Mothers’ knowledge with regard to case management of diarrhea at home has increased. There is also a remarkable increase in the mothers’ knowledge about child spacing (increased from 24% to 53%), danger signs of pregnancy, malaria, pneumonia, and dehydration caused by diarrhea. The project strategy of using Community Leader Councils (that include elementary nurses, community leaders, TBAs, Activistas, and community members) has been successful in establishing and expanding community-based activities. These include the dissemination of health and hygiene information in rural communities, community mobilization for participation in program activities, promotion of EPI and CDD related activities, and the establishment of EMOC plans to transfer pregnant women with obstetric emergencies to primary referral facilities where appropriate care can be provided. Although considerable progress in project implementation has been made, not all the targets or objectives could be achieved by the completion date of September 30, 2003 due to the constraints discussed in the following FE report. (Please see Section II (c) of this report entitled “Progress Towards Achievement of Objectives by Intervention Area.”) CS-16, Mozambique, Final Evaluation Save the Children 8 Priority conclusions resulting from the final evaluation (1) General Conclusions · The FE team concluded that the project has made considerable progress towards the achievement of goals and objectives. The project is also achieving objectives and outcomes of the USAID Country Strategic Plan. · The inputs provided to the project were adequate and appropriate for the expected outputs, but the number and capacity of the district-based health workers to implement and sustain program activities remains low. · The overall knowledge of mothers interviewed in Nacala-a-Velha villages was considered very high with regard to disease prevention, hygiene and sanitation, family planning, and newborn care. · Some of the mothers volunteered to serve as models for others who have not acquired the knowledge and skills to care for themselves and their small children (positive deviance approach to educate other mothers in the community). · More inputs should have been gathered and provided to strengthen the DDS/DHO, and to upgrade the emergency obstetrics at the health center and the referral hospital levels of the districts. · In parts of Memba and Nacala-a-Velha Districts that continue to suffer from a low level of health personnel, training alone, without the support from MOH that carries out regular monitoring and follow up supervision of trained health workers, does not contribute to sustained effective delivery of quality services. (2) Maternal and Newborn Care · Mothers continue to favor TBA-assisted deliveries in their homes compared to assisted deliveries at health facilities; because district-based health facilities are not easily accessible to the public, (they are located far away from the villages (regulados). · Mothers knowledge regarding clean and safe delivery are very high and all interviewed mothers confirmed that they would not give birth without previously purchasing a clean birth kit. · A large number of Community Leaders Council (CLC) members have been trained to facilitate the dissemination of information and the expansion of IEC activities in the communities, especially about the promotion of prenatal care and the vaccination of mothers and children. · TBAs’ efforts to promote clean and safe deliveries were also recognized as being high in the communities, however collaborative efforts must continue with the CLCs to ensure the establishment of functional councils that can plan for emergency evacuation of women with complications at delivery, as well as ensure the maintenance and repair of bicycle ambulances. CS-16, Mozambique, Final Evaluation Save the Children 9 (3) Child Spacing · Knowledge about the use of modern methods for child spacing is high in the rural communities and continues to increase. · Due to the prevailing shortage of contraceptives in rural areas, and barriers such as tradition and the prevailing desire by men to have more children in rural areas, the team concluded that the actual use of contraceptives by women of child bearing age remains low and it will take time to show a remarkable increase. · Men should be trained about the need for child spacing so that they can actively contribute to the improved health of the mothers and children in the communities. (4) Immunization Activities · The target for fully immunized children aged 12-23 months was exceeded by the project due to the improved logistics of vaccines and the increased involvement of Mobile Brigades in EPI activities at outreach sites. · Efforts should be made to establish affordable approaches to support and improve vaccination coverage in the districts, as the use of vehicles to support Mobile Brigades’ vaccination efforts in the districts is too expensive for the MOH. · CLCs played a major role in increasing community participation in vaccination activities. Mothers’ knowledge about disease prevention was higher in areas where mothers had more exposure to CLC activities than in villages where CLCs were less active. CS-16, Mozambique, Final Evaluation Save the Children 10 I. INTRODUCTION A. Purpose of the Evaluation This final evaluation (FE) focused on the review of the six interventions that have been described in the preceding sections of the report. The purpose of the evaluation was to assess progress toward achieving the goals and objectives of the project as stated in the approved DIP and the Program Description of the Cooperative Agreement. B. Scope of Work for the Final Evaluation The FE of the Strength Project was planned for early September 2003. The focus of the evaluation activities was on: (a) assessing if the program has met the stated goals and objectives, (b) assessing the effectiveness of the technical approaches, (c) development of the lessons learned from the project, and (d) a strategy for use or communication of these lessons, both within the organization and to partners. To facilitate the evaluation process, a questionnaire was designed to provide valuable qualitative information and recommendations for inclusion in the evaluation report. The objectives that the FE team used to guide the development of the report include: · Assess progress made toward the accomplishment of objectives. · Determine if interventions were sufficient to reach desired outcomes. · Identify strengths and weaknesses in the implementation of the project activities including maternal and newborn care, immunization, diarrheal disease control, malaria control, and case management of ARI. · Identify external factors that may have adversely affected project implementation. · Determine the acceptability of the project by the communities and other collaborating groups, including the provincial and district health authorities and NGOs. C. Methodology The principal methodologies used for this evaluation include: · The final KPC survey was carried out in the two districts prior to the start of the evaluation activity; · Document review, including previous quarterly progress and annual reports; · Interviews of key project and district health personnel and officials · Interviews of participant organizations (USAID/Mozambique, UNICEF, provincial and district MOH officials, and local authorities); and · Interviews of community members, including CLCs and mothers who attend MCH clinics in the communities. The FE activities began on September 3, 2003 with the team’s participation in a two-day workshop on provincial planning and review of progress made under the Mission-funded CS-16, Mozambique, Final Evaluation Save the Children 11 Bridges-to-Health Project in Nacala-Porto. The sessions included participants from the MOH provincial and district levels, USAID/Mozambique representatives, Helen Keller International representative, District Health Directors, districts MCH nurses, Save the Children/Mozambique Bridges-to-Health Project Coordinator, and other representatives of the Strength Project. Upon completion of the two-day session in Nacala Porto, the team met to plan and carry out FE activities in Nacala-a-Velha and Memba Districts. The evaluation activities included interviews with health personnel, CLC members, community leaders, and with the field level project staff to observe and verify progress, and obtain qualitative information for the evaluation report. Approximately 100 people and community members were interviewed during the evaluation process. D. Results: Summary Tables of Baseline and Final KPC The Nacala-a-Velha and Memba (CS-16 project) areas made good progress toward the achievement of the program objectives as summarized in Tables 1 and 2 below. Bolded figures under Final KPC survey indicate areas where planned targets were achieved or surpassed. Please note that targets were not set for all of the indicators in the two tables below. Some of the indicators were to be measured by a final health facility assessment, however due to budgetary constraints in Year 3 a HFA was not conducted. Table 1: Progress in Achieving Objectives in Nacala-a-Velha District Objectives and Indicators Baseline 2000 Midterm 08/02 Final KPC 2003 Immunization (15%) 1. 55% of children aged 12-23 months will be fully immunized by the age 12 months 32% 11% 55% 2. 90% of health facilities will have the equipment and supplies to support full vaccination services 58% N/A N/A Maternal and Newborn Care (30%) 1. 85% of women will have at least 2 antenatal care visits with trained health personnel during their last pregnancy 77% 74% 80% 2. 80% of women will receive 2 doses of tetanus toxoid during their last pregnancy 30% 74% 80% 3. 50% of women will be attended by trained personnel during their last delivery 34% 51% 52% 4. 10% of women will have had birth plans (3 of 5 components) during their last pregnancy 0% 3% 2% 5. 80% of women with children less than 24 months will know at least 3 pregnancy related danger signs 18% 11% 55% Malaria Control (15%) 1. 80% of mothers with children < 24 months with fever in previous two weeks sought care within 48 hours 57% 30% 40% 2. 40% of children under five who presented at health facilities will have two or more fever examination tasks completed N/A N/A N/A Diarrhea Case Management (10%) 1. 80% of mothers with children < 24 months with diarrhea in previous two weeks who managed the diarrhea with oral rehydration therapy (oral rehydration solution or community-based oral rehydration fluids such as watery porridge) 66% 67% 73% 2. Percent of women with children less than 24 months who reported giving colostrums to their children 83% 79% 75% 3. Percent of women with children less than 24 months who reported initiating breastfeeding for their last child within 8 hours of birth 73% 86% 85% 4. Percent of women with children < 6 months who reported giving their CS-16, Mozambique, Final Evaluation Save the Children 12 Objectives and Indicators Baseline 2000 Midterm 08/02 Final KPC 2003 children only breast milk N/A 30% 9% 5. Percent of women with children < 24 months who reported knowing at least two ways of preventing diarrhea 46% 56% 59% Pneumonia Case Management (15%) 1. 85% of mothers with children < 24 months with cough and difficult/rapid breathing in previous 2 weeks sought care within 48 Hr. 64% 54% 61% 2 Percent of women with children < 24 months who reported knowing 2 IMCI danger signs 45% 86% 84% Child Spacing (15%) 1. 25% of mothers with children < 24 months, who do not desire to have a child in the next 2 years, will use modern FP method 18% 18% 22% 2. 50% of women with children < 24 months, will know at least 2 modern methods for child spacing 24% 28% 53% Table 2: Progress in Achieving Objectives in Memba District Objectives and Indicators Baseline 2000 Midterm 08/02 Final KPC 2003 Immunization (15%) 1. 55% of children aged 12-23 months will be fully immunized by the age 12 months 34% 13% 57% 2. 90% of health facilities will have the equipment and supplies to support full vaccination services N/A N/A N/A Maternal and Newborn Care (30%) 1. 85% of women will have at least 2 antenatal care visits with trained health personnel during their last pregnancy 74% 55% 85% 2. 80% of women will receive 2 doses of tetanus toxoid during their last pregnancy 62% 42% 70% 3. 50% of women will be attended by trained personnel during their last delivery 38% 28% 66% 4. 10% of women will have had birth plans (3 of 5 components) during their last pregnancy 0% 5% 2% 5. 80% of women with children less than 24 months will know at least 3 pregnancy related danger signs 44% 57% 38% Malaria Control (15%) 1. 80% of mothers with children < 24 months with fever in previous two weeks sought care within 48 hours 66% 34% 80% 2. 40% of children under five who presented at health facilities will have two or more examination tasks completed N/A N/A N/A Diarrhea Case Management (10%) 1. 80% of mothers with children < 24 months with diarrhea in previous two weeks who managed the diarrhea with oral rehydration therapy (oral rehydration solution or community-based oral rehydration fluids such as watery porridge) 67% 62% 75% 2. Percent of women with children less than 24 months who reported giving colostrums to their children 74% 70% 76% 3. Percent of women with children < 24 months who reported initiating breastfeeding for their last child within 8 hours of birth 73% 75% 82% 4. Percent of women with children < 6 months who reported giving their children only breast milk 14% 43% 9% 5. Percent of women with children < 24 months who reported knowing at least two ways of preventing diarrhea 64% 50% 53% Pneumonia Case Management (15%) 1. 85% of mothers with children < 24 months with cough and CS-16, Mozambique, Final Evaluation Save the Children 13 Objectives and Indicators Baseline 2000 Midterm 08/02 Final KPC 2003 difficult/rapid breathing in previous 2 weeks sought care within 48 Hr. 73% 67% 69% 2 Percent of women with children < 24 months who reported knowing 2 IMCI danger signs 82% 73% 71% Child Spacing (15%) 1. 25% of mothers with children < 24 months, who do not desire to have a child in the next 2 years, will use modern FP method 15% 12% 10% 2. 50% of women with children < 24 months, will know at least 2 modern methods for child spacing 37% 23% 37% II. TECHNICAL APPROACH A. Brief Overview of the Project The Memba and Nacala-a-Velha Districts of Nampula Province were the two focus areas for this follow-on grant. The districts are located in the northeastern part of Nampula Province and most of the districts’ population is rural with about 25 percent urban. These districts are the two (of the province’s 21 districts) poorest and most inaccessible in Nampula and in Mozambique in general. Their estimated total population is 286,814. The under-five mortality rate of 315/1000 is reported to be almost 50% higher than the national average of 219/1000 live births, and maternal mortality is recognized to be high due obstructed labor, anemia (hemorrhage), and infections (including puerperal and post-abortion sepsis). The public sector is the main provider of health services in the two districts and Save the Children is the only NGO partner that is active in these Nampula Districts (see Figure 1 for a map of the project site). The project was working through the existing district-based health facilities to strengthen the capacity of personnel, and promote and deliver child survival and maternal care services. The major public health problems that contribute to high infant and maternal morbidity and mortality include malaria, anemia, infections (ARI, TB, STI/HIV/AIDS), malnutrition, and diarrhea. B. Program Goals/Objectives, and Strategies The Strength Project was designed in response to the needs and priorities of the communities, and the goals of the Ministry of Health (MOH) and USAID/Mozambique, to reduce the under￾five and maternal mortality and morbidity through increased access to essential health services, and through strengthened district and community-based health systems. The District Health Offices were the main implementers of the project interventions and SC provided technical assistance, training/capacity building, management and financial support. The ultimate goals of the project were to: (1) contribute to the sustainable reduction of the under￾five mortality rate; (2) improve the capacity of the Health District Directorate (DDS) and communities to respond to their health needs, and (3) develop innovative approaches to inform policy and practices of Save the Children (SC) and its development partners. CS-16, Mozambique, Final Evaluation Save the Children 14 Figure 1: Map of the Project areas of Nacala-a-Velha and Memba The two major strategies that were used to implement program activities in the two districts are: (1) Partnership and Institutional Development for District Health System Strengthening through the: (a) Program Management Team (PMT) to enhance SC-DHO partnership for program planning, monitoring and evaluation, capacity-building and sustainability of program activities. (b) Support of additional staff for DHO to address the inadequate number of nursing staff in the two districts. (c) Upgrade emergency obstetric health facilities and services. (2) Strengthening the Community Component of MCH Through: (a) Support for Community Health Teams (CHTs) that would monitor and support activities of community health workers in the villages (regulados). (b) Support the outreach services by Mobile Brigades. (c) Facilitation of Community Alarm and Transport (A&T) system. (d) Provision of bicycles ambulances as part of the A&T system. (e) Production and promotion of clean birth kits. (f) Training of community-based providers (APEs and Soccoristas). (g) Training of Activistas and TBAs. (h) Training and dialogue with Traditional Healers. (i) Support of community mobilization for responsive and appropriate behavior change communication (BCC) for caregivers, community leaders, and male and female decision makers. CS-16, Mozambique, Final Evaluation Save the Children 15 C. Progress Towards Achievement of Objectives by Intervention Area (a) Project inputs The primary inputs of the Strength Project consisted of financing (the total grant amount includes $700,000 from USAID, matched by $233,300 from SC). Most of these funds were planned to support the implementation of the six project components that support the MOH’s delivery of essential health package that include Maternal and Newborn Care (30%), Immunization (15%), Control of Malaria (15%), Pneumonia Case Management (15%), Control of Diarrheal Disease (10%), and Child Spacing (15%). Some of the commodities that have been delivered by the project include bicycle ambulances and radio communication equipment to assist with the transfer of the critically ill patients to primary referral facilities in the districts. Other donors continue to collaborate in the provision of essential drugs to the country’s health care system, and UNICEF/Mozambique is providing vaccines and cold chain equipment to improve the logistics of vaccine and cold chain systems. In addition, funds that have been provided to CS-16, have contributed to the training of health care workers (APEs, and TBAs), and have also contributed to the training-of-trainers, decentralization activities, and an improved referral system. (b) Summary of Overall Progress The project has made excellent progress in supporting vaccination activities at fixed posts and at outreach sites in collaboration with health workers. Progress has also been made with increased vaccine coverage for children under five and with women of child-bearing age, through the support provided by the Mobile Brigades at the outreach sites. To date, all project assisted health facilities (health centers and health posts) have trained MCH nurses, APEs, and TBAs on site, to support program activities at health facilities and community levels. Both the APEs and TBAs are promoting disease prevention and the utilization of health services. The TBAs are also assisting mothers during deliveries and promoting child spacing. Trained CLC members and the theatre groups are also contributing to the promotion of hygiene, disease prevention, and maternal and child health activities in rural communities. (c) Major Accomplishments to Date per Program Intervention A summary of major achievements to date under the six program components is as follows: (1) Maternal and Newborn Care (30% of Effort) This intervention has been very successful because most of the nurses and CLC members trained by CS-16 have been collaborating to support and promote prenatal, postnatal and vaccination program activities at the health facility and community levels. Funds reserved for this component, which include TA support from SC, are adequate but additional time and efforts are still needed to facilitate the establishment and the expansion of this component in the districts. Achievements to date under this component include: CS-16, Mozambique, Final Evaluation Save the Children 16 · A remarkable increase in the proportion of women who seek prenatal care and utilize TBAs. There is also a very significant increase in the number of women who seriously prepare themselves for safe delivery through the use of locally available clean birth kits. · The clean birth kits promoted by TBAs include soap, a sterile razor blade, cloth for wrapping the baby after birth, and some funds that could be used in case of emergency. · The project has supported the training and development of local theater groups in the two districts to expand the presentations that contain health information designed to improve community knowledge about disease prevention and utilization of available health services (prenatal and postnatal care). · A large number of CLC members have been trained and are working to facilitate the dissemination of information and the expansion of IEC activities in the communities. The CLC members are also promoting prenatal and postnatal care, and malaria prevention activities in the communities. · 219 TBAs and 322 CLC members have already taken part in the refresher courses. Table 3: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 85% of women will have at least 2 ANC visits with trained health personnel during their pregnancy (Baseline: 77% Nacala-a-Velha, 74% Memba). 83% In progress. Health facilities are too far away from villages. (2) 80% of women will receive 2 doses of TT during their last pregnancy (Baseline: 30% Nacala-a-Velha, 62% Memba). 73% Progress has been made but target has not been achieved due to stockouts of vaccines. Mobile Brigades are increasing accessibility to immunizations. (3) 50% of women will be attended by trained personnel during their last pregnancy (Baseline: 34% Nacala-a￾Velha, 38% Memba). 65% This target has been exceeded. Trained health workers include TBAs. (4) 10% of women will have birth plans (3 of 5 components) during their most recent pregnancy. (Baseline: 0%). 2% Final KPC reveals higher achievement. Interviews with mothers indicate that a higher proportion is planning births than before. (5) 90% of regulados (villages) will have formal alarm and transport plans established and functioning. 95% (283 villages have a plan, while 49 have bicycle ambulances; 19/49 villages are CS-16 assisted communities - There are 120 communities in Nacala and 179 communities in Memba. - Maintenance of bicycles and ambulances has not been adequate and more needs to be done in this area to ensure that this equipment is well maintained and functioning. (6) 50% of women 15-49 will know at least 3 pregnancy￾related danger signs (Baseline: 18% Nacala-a-Velha, 44% Memba). 35% In progress. Mothers are increasingly becoming knowledgeable about the danger signs. CS-16, Mozambique, Final Evaluation Save the Children 17 Figure 2: Bicycle Ambulance with Sunshade (i) Results as Measured by Comparison of Baseline and Final Evaluation Document review and interviews with health personnel in Nacala-a-Velha and Memba confirmed that more women attend prenatal and postnatal services now than three years ago. From interviews carried out with more than 30 women in the communities, the FE team was able to gather the following information: · About half of the women targeted by this intervention come to established facilities for prenatal consultation before the 6th month of pregnancy. · Some of the reasons women gave for coming early to prenatal care include: (a) availability of prenatal services now, (b) inability to hide pregnancy at five to six months, and (c) feeling of the baby movement. (ii) Factors Affecting Achievement or Non-Achievement of Program Objectives · The inadequate number of district-based health facilities in Memba and Nacala-a-Velha continues to negatively impact program goals. · Due to existing traditions, many women prefer to participate in a prenatal care program during the second trimester of pregnancy. Reasons given for this preference included taboos around revealing pregnancy before “showing”, waiting until the sensation of fetal movement is felt and hearing that nurses at health posts will not see you until later in the pregnancy. (iii) Lessons Learned · Increasing the number of trained health workers at district-based health facilities to provide counseling and health education, and deliver curative and preventive health services can contribute to improved health of mothers and children in the communities. · Community traditions that are a deterrent to improved child health and survival can be changed through training and motivation of community members, and through good dialogue CS-16, Mozambique, Final Evaluation Save the Children 18 between trained TBAs, traditional healers, and the CLC members to support behavior change. · Bicycle ambulances are an acceptable and sustainable means of transportation for transporting people (including women) with medical emergencies to nearby facilities for appropriate medical care. (iv) Unexpected Successes or Constraints Although progress has been made with the maternal and newborn care component, efforts are still needed to address the following constraints: · Long distances between homes and health facilities in rural areas of Memba and Nacala-a￾Velha Districts continue to reduce the access of the rural population to existing health services. This constraint could be addressed by establishing more health posts or outreach sites near villages where the majority of people live. The use of the Mobile Brigade teams is an option that can be considered but the issue of its long-term financial sustainability should be addressed so that appropriate means of transportation for Mobile Brigades can be used to ensure sustainability of the outreach activities in the districts. · The traditions identified as barriers that impede progress with program objectives need to be more adequately addressed by behavior change strategies. (2) Immunization (15% of Effort) Project activities focused on strengthening the capacity of the districts and the local EPI teams to plan, manage, and implement vaccination programs at fixed facilities and at outreach sites. The ultimate goal of this component is to increase vaccine coverage of children aged 0-23 months (focus was on fully immunizing children before their first birthday) and women of child bearing age through efficient use of available vaccines and human resources. There are five functioning facilities in Nacala-a-Velha, and 11 in Memba District that are involved in EPI activities with logistical support from the district teams (Mobile Brigades). In addition, the project has been providing logistical support to 118 Mobile Brigades team members (52 in Nacala-a-Velha, and 66 Memba) in the districts. Table 4: Summary of Accomplishments to Date Under Immunization PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 55% of children aged 12-23 months will be fully immunized by the age 12 months (Baseline: 32% Nacala-a-Velha, 34% Memba) 56% Considerable progress has been made with the immunization activities in the two focus districts due to improved organization of Mobile Brigades and improved logistics of vaccines and cold chain system. (2) 90% of health facilities will have the equipment and supplies to support full vaccination services (Baseline: 58%) Not available UNICEF is the provider of vaccines and cold chain equipment. Hospitals, health centers/posts reported stockouts of vaccines and they often do not have enough kerosene to support their cold chain system CS-16, Mozambique, Final Evaluation Save the Children 19 Table 5: Summary of Planned Vaccination Activities BENEFICIARIES (# Vaccinated between 01/03 - 05/03) SUMMARY OF PLANNED VACCINATION ACTIVITIES/OBJECTIVES Nacala-a-Velha Memba BCG 0 – 1 years (@ 9 months) 3533 7643 POLIO 0 1 2 3 0 – 3 years 2484 2636 2351 2343 4224 5698 4380 4007 DPT 1 2 3 2 months – 3 years 2628 2306 2378 5396 4171 4343 6 – 8 months MEASLES 9 months – 5 years 5 – 15 years 2526 2575 5420 5877 TETANUS 1 2 15 – 45 years (women) 2093 1410 4700 2896 Note: The above table contains information for January – May 2003. Previous year’s data are not available and should be requested from the Provincial Health and District offices. These are also data that have not yet been officially reported by the DPS. (i) Results as Measured by Comparison of Baseline and Final Evaluation · 56% of children aged 12-23 months are now reported to be fully immunized as compared with baseline (32% Nacala and 34% for Memba). Progress has been made with vaccination activities in the districts due to the increased involvement of Mobile Brigades, CLC members, and the Activistas (community volunteers). (ii) Factors Affecting Achievement or Non-Achievement of Program Objectives · Save the Children supported transportation logistics for the Mobile Brigade outreach activities, and this effort contributed significantly to increasing vaccine coverage in the district. However, the overall low achievement of program objectives is attributed to frequent stockouts of vaccines at district-based health facilities, including the unreliable supply of kerosene necessary to maintain the functioning of the district cold chain system. (iii) Lessons Learned · Enhancements made to planning, supervision and vaccine and cold chain system logistics can considerably improve vaccine coverage of the target population. · Mobile Brigades are an effective approach to extending the provision of health services to areas outside the reach of health facilities. Their services must be regularly scheduled and coordinated with the CLCs, and there should be at least 4-5 visits per year to ensure increased vaccine coverage of the target population. · Assigning one new vehicle to Memba and one vehicle to Nacala-a-Velha to support Mobile Brigades’ community-based activities would have improved transportation logistics in the CS-16, Mozambique, Final Evaluation Save the Children 20 two target districts. The MOH continues to rely on SC for logistical support for the mobile brigades. (iv) Unexpected Successes or Constraints · Frequent stock outs of vaccines and lack of kerosene had a negative impact on vaccination activities in the districts. This constraint is currently being addressed by SC and the district health officials through increased supervision of the cold chain system and vaccine supply to the districts. The possibility of establishing and maintaining an effective solar cold chain system at the district level to support the district-based health facilities with EPI-related activities, is being explored. (3) Control of Malaria (15% of Effort) Through CS-16, SC and its partners at the district and community levels have been supporting and promoting an effective Malaria Control program for children under five years of age and pregnant women. Malaria control activities are also included in the provincial’s essential service package that is supported by the MOH and its partners (USAID, WHO, and UNICEF). Program activities have been focusing on increasing awareness about malaria transmission and what can be done at the community and household levels to prevent malaria. Mothers (care takers) have also been trained to recognize danger signs that require prompt attention and treatment of children by trained health workers. Table 6: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 80% of mothers with children < 24 months with fever in previous two weeks will seek care within 48 hours (Baseline: Nacala 57%, Memba 66%) 67% - Awareness about danger signs of malaria has increased among mothers (care takers). - The team concluded also that early care seeking for malaria related illness in children has increased in the communities. (2) 40% of children under five who presented at health facilities will have two or more examination tasks completed (Baseline: 7%). N/A Final HFA not conducted. (i) Results as Measured by Comparison of Baseline and Final Evaluation · Awareness about dangerous signs of malaria has increased among mothers (care takers) in rural communities as compared to baseline. · Early care seeking for malaria related illness in children has also slightly increased in the communities as compared to baseline. (ii) Factors Affecting Achievement or Non-Achievement of Program Objectives · Mothers and children living far away from areas that can be covered by health facilities continue to suffer from lack of information about malaria prevention and lack of appropriate malaria case management. CS-16, Mozambique, Final Evaluation Save the Children 21 (iii) Lessons Learned · Knowledge about malaria case management and prevention can be increased in the population by increasing group counseling activities and the involvement of the theater groups and trained CLC members in IEC/BCC activities. · Case management of malaria at home and at health facilities can be assured by improved logistics of essential drugs (Chloroquine, Fansidar (Sulfadoxine and Pyrimethamine, and Paracetamol) at district-based health facilities. (iv) Unexpected Successes or Constraints · Frequent stock outs and unreliable supply of essential drugs at community-based health facilities contributed to difficulties in ensuring timely and effective case management of malaria at home and at the health facilities. (4) Pneumonia Case Management (15% of Effort) Acute respiratory infections are among the leading cause of illness and death in children under five years old. Program activities in this area have focused on increasing the knowledge of the community members (especially mothers and care takers) about the danger signs of pneumonia and on the need to seek early treatment from trained health workers at an established facility within 48 hours of recognition of symptoms. Table 7: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 85% of mothers with children < 24 months with cough and difficult/rapid breathing in previous 2 weeks sought care within 48 Hr (Baseline: Nacala 64%, Memba 73%) 67% - The target is too high to be achieved by the districts. - Some progress was made with this intervention. (2) Percent of women with children < 24 months who reported knowing 2 IMCI danger signs (Baseline: Nacala 45%, Memba 82%). 77% - Progress has been made with this intervention. - It should be noted also that IMCI strategy and approaches have not yet been fully introduced in the project areas in the districts. - Training of health workers in IMCI strategy and approaches is still needed. (i) Results as Measured by Comparison of Baseline and Final Evaluation · Although data that are summarized above do not show clear progress with this activity as compared with the baseline, the perception of the FE team members following document review and interviews with health workers is that progress has been made in this area. More attention needs to be given to refresher training and follow up of health workers to ensure effective case management of pneumonia at the health facility level. (ii) Factors Affecting Achievement or Non-Achievement of Program Objectives · The continued low capacity and insufficient number of trained health workers in the districts that can deliver curative and preventive services has limited progress of this intervention. CS-16, Mozambique, Final Evaluation Save the Children 22 (iii) Lessons Learned · The involving other partners (UNICEF, the Dutch Embassy, NGOs, etc.,) in the support of IMCI program activities that complement the partners’ goals and objectives in the province and the districts (especially when there are budgetary shortfalls) would have contributed to the achievement of IMCI related targets. (iv) Unexpected Successes or Constraints · The shortage of trained health workers in IMCI related activities is still a problem that needs to be addressed by the MOH in the two districts. · The unreliable supply of essential drugs (antibiotics) at health centers/health posts had a negative impact on the success of this intervention at health facilities and at the community level. (5) Control of Diarrheal Diseases (10% of Effort) Diarrhea is one of the leading causes of illness and death in children less than five years of age. It can exacerbate an existing malnutrition condition in children, and if it goes unchecked, it can cause death due to severe dehydration in small children. Program activities have been focused on improving case management of diarrhea at home and on early recognition of danger signs by mothers/care takers that require the attention of trained health workers. Table 8: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 80% of mothers with children < 24 months with diarrhea in previous two weeks who managed the diarrhea with oral rehydration therapy (oral rehydration solution or community￾based oral rehydration fluids such as watery porridge). Baseline: Nacala 66%, Memba 67%. 67% - The reported 67% achievement is not reflective of true accomplishment under this intervention. - Mothers’ knowledge is very high with regard to how they prepare ORS and homemade solutions for kids with diarrhea. (2) Percent of women with children less than 24 months who reported giving colostrum to their children (Baseline: Nacala 83%, Memba 74%). 76% Progress is being made with this objective. Mothers are progressively learning to provide colostrums to their newborn babies. (3) Percent of women with children less than 24 months who reported initiating breastfeeding for their last child within 8 hours of birth (Baseline: Memba 73%, Nacala73%). 83% In progress. Breastfeeding is quite high in the two districts but exclusive breastfeeding during the first 6 months is still low in some areas. (4) Percent of women with children < 6 months who reported giving their children only breast milk (Baseline: Memba 14%, Nacala N/A). 9% In progress. Some people still give water and sugar to their infants in addition to breast milk. (5) Percent of women with children < 24 months who reported knowing at least two ways of preventing diarrhea (Baseline: N-46% M-64%). 55% Knowledge about diarrhea prevention is quite high among mothers (care takers). Work still needs to be done with IEC/BCC to promote diarrhea prevention and increased consumption of potable water as well as home made solutions during diarrheal episodes. CS-16, Mozambique, Final Evaluation Save the Children 23 (i) Results as Measured by Comparison of Baseline and Final Evaluation · Despite little change in the KPC indicators, the FE team members observed a remarkable level of knowledge by mothers regarding diarrheal disease treatment and prevention. (ii) Factors Affecting Achievement or Non-Achievement of Program Objectives · Traditional beliefs and barriers continue to prevent considerable progress with diarrheal disease control program activities. For example, exclusive breastfeeding is not yet fully practiced by all mothers in the target districts because some of them continue to discard the colostrum and give sugar water to their babies. (iii) Lessons Learned · It would have been helpful to further emphasize the following three key behavior change practices as part of the primary prevention of diarrhea: hand washing at critical times with appropriate technique, sanitary disposal of feces, and protection of drinking water from fecal contamination. (iv) Unexpected Successes or Constraints · Regardless of the constraints and traditional barriers that impede progress with the breastfeeding objective, the project worked well through the CLC members and TBAs to increase breastfeeding of babies within eight hours of birth. (6) Child Spacing (15% of Effort) Child spacing is an essential component of reproductive health services that has been supported and promoted by the project. The focus of the program activities has been on training and capacity building of nurses and TBAs to promote child spacing through increased counseling and availability of supplies and services at health centers and health posts. This is in accordance with the MOH policy regarding the provision of family planning services by trained health workers. Table 9: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 25% of mothers with children < 24 months, who do not desire to have a child in the next 2 years, will use modern FP method (Baseline: method (Nacala18%, Memba 15%). 16% The 25% target is believed to be too high and unrealistic to accomplish within three years. (2) 50% of women with children < 24 months, will know at least 2 modern methods for child spacing (Baseline: Nacala 24%, Memba 37%). 63% The knowledge about family planning is believed to be high in the districts but the actual use of modern methods is still low. (i) Results as Measured by Comparison of Baseline and Final Evaluation From the interviews carried out during the FE, the evaluation team learned about the progress made in this component. More women who were interviewed know about the need for child spacing today than when the project was introduced in the districts. Efforts have also been made CS-16, Mozambique, Final Evaluation Save the Children 24 to increase the availability of condoms and pills in the project areas, but more still needs to be done. (ii) Factors Affecting Achievement or Non-Achievement of Program Objectives Traditional beliefs and barriers continue to prevent considerable progress with child spacing intervention because most men believe that having many children enhances their standing in the community. In addition, the persistent high rates of infant and child mortality encourage families to have more children to compensate for those who die from childhood diseases. The FE team believes that child spacing has a good chance for success because communities and health facilities are working on the interventions that will reduce child mortality in the project area. It is believed that as more children survive beyond the age of five, more families will start to use available family planning services that will contribute to achieving the desired family size in rural areas of Mozambique. (iii) Lessons Learned · Behavior change is effective when an enabling environment is created. This is best accomplished through the training of the community to increase awareness about the need for child spacing, including the increased involvement of men in child health related activities. (iv) Unexpected Successes or Constraints · Continued occasional stockouts of pills, condoms, and Depo Provera occur as a result of sporadic problems in procurement and distribution to the district-based health facilities. · Because of a shortage of personnel and high staff turnover at the DDS, many district level staffs were not involved in the design of the original project activities that included child spacing. III. CROSS CUTTING APPROACHES A. Community Mobilization (1) The FE team noted that the community mobilization strategy of the project has been successful and included the involvement of the Community Health Team comprised of CLC members, TBAs, Activistas, and Traditional Healers. The Community Health Team’s community mobilization plans include IEC and BCC activities for the promotion of key behaviors related to the project interventions. (2) Most of the objectives for community mobilization have been met. The FE team also learned that the community members would like to see program activities continue beyond the life of the project. (3) One of the options that is being explored with the MOH and USAID/Mozambique to continue program activities beyond September 30, 2003 is the need to integrate CS-16 activities in the ongoing USAID-assisted Bridges-to-Health Project that covers six districts in Nampula. CS-16, Mozambique, Final Evaluation Save the Children 25 (4) The demand for continued program activities was measured through focus group discussion that took place in the districts with community members and their leaders. The district health officials are also requesting a cost extension of the grant for an additional year so that planned activities can be completed. Findings and Observations: It was discovered that in some cases Activistas from the former CS-12 project were unclear about their status, as SC has changed the focus from Activistas to CLC involvement in community-based health activities. Some Activistas have not been fully integrated into the CLCs as members and therefore they have not effectively contributed to the promotion of community￾based activities. This matter is being addressed by the district health officials to ensure that Activistas participate fully in community-based health activities as integral members of the CLCs. (5) The FE team made the following observations: (a) CLC Local Plans for Obstetric Emergencies Although these activities have been implemented in the communities, the team was unable to assess the degree to which the community has responded to them. Barriers are not well known at this time, but are expected to relate to the limitations that CLCs have to influence behavior change. Further, the status of roads and the availability of vehicles will strongly influence the effectiveness of community-developed obstetric emergency plans that include the use of bicycle ambulances for health emergencies in the community. (b) Insecticide Treated Nets Promotion Through CLC Activists In Nacala-a-Velha and Memba Districts, mothers who were interviewed preferred using bednets because in the long-run, it is more cost effective for malaria prevention in children and pregnant women. Unfortunately, due to the prevailing poverty in the project area, the cost to acquire insecticide treated nets (ITN) is quite prohibitive. Nevertheless, the CHT has been able to make progress in the area of training mothers and care takers on the early recognition of danger signs of malaria, and the need for early referral and treatment of suspected cases of malaria in children and pregnant women. (c) Care Seeking Behavior Among Mothers/Care Takers Similarly, early recognition, referral and treatment of pneumonia and diarrhea is also being supported and promoted by the CHT members in combination with the other program interventions. For example, in Nacala-a-Velha, TBAs have referred 771 expectant mothers to community health agents for prenatal consultation. In Memba, TBAs have referred a much higher number (1,601) of pregnant women to appropriate facilities for prenatal consultation during the first trimester of pregnancy. Conclusion and Recommendation: (a) It is difficult for the TBAs to make regular household visits to areas that are located far away from their homes in rural communities; (b) Regular CS-16, Mozambique, Final Evaluation Save the Children 26 supervision of TBAs and CLC members is important to ensure the quality of service delivery and the motivation of TBAs. B. Communication for Behavior Change Communication for behavior change is at the pillar of the six project interventions that have been carried out under the Strength Project in Nacala-a-Velha and Memba Districts. Hence, communication objectives were closely linked to the program interventions. Some of the BCC￾related specific objectives under the project interventions include: · Immunization: Mothers/care takers will get their children fully immunized before their 12 month birthday; · Malaria Control: Mothers/care takers with children < 24 months old with fever in previous two weeks will seek care within 48 hours of noticing symptoms; · Maternal and Newborn Care: 85% of women have at least two antenatal visits with trained health personnel during their last pregnancy; and · Diarrheal Disease Control: Mothers/care takers practice immediate breastfeeding of babies at birth and use ORT when their children have diarrhea. Unfortunately, due to time constraints, the team could not observe theater presentations but was able to carry out focus group discussions and interviews with mothers in selected villages of the two districts. To achieve the CS-16 program objectives, the project used the multi-channel approach for BCC to support project implementation. Some of those approaches include: (a) Group counseling at both health facilities and community levels; (b) One-on-one counseling provided at both facility and community levels by a trained health worker; (c) Songs at both the facility and community levels; and (d) Interactive drama at the community level by the theater groups. The following observations were made of these approaches: (i) Effectiveness: No specifically targeted studies have been carried out to measure the effectiveness of IEC and BCC messages that support changes in knowledge, attitudes, or practices. The only measure of effectiveness that has been observed is the presence of condoms and oral contraceptives that are being provided to clients at health centers and health posts, and the perceived increase in the knowledge of mothers regarding pregnancy related danger signs, modern methods of contraceptives, and improved care-seeking behavior at the appearance of danger signs for malaria and pneumonia. (ii) Achievement of BCC Objectives: Based on the final KPC findings and results, progress has been made in the area of BCC objectives. As indicated above, the team was not able to observe the performance of a theater group but was informed that efforts are made to verify audience knowledge before and after a presentation by the theater groups. Health workers indicated that most of the time, the results have been quite positive with clear messages and measurable learning by the target groups. The messages contained the required standard key practices and used the standard approaches used by UNICEF, WHO, and the MOH. CS-16, Mozambique, Final Evaluation Save the Children 27 (iii) Conclusion: From interviews with mothers in the villages, and from the recently completed final KPC survey, the FE team concluded that considerable progress can be made with IEC and BCC related activities when barriers to behavior change are addressed during the design and implementation of the IEC/BCC strategy and approach. (iv) Sustainability: The IEC/BCC approaches that were used to support project implementation are sustainable because they use the existing channels of communication in the communities and include the involvement of the CLCs. As result of progress made to date with BCC, during the focus group discussions, some of the most knowledgeable mothers indicated that they would be willing to serve as models to support the learning of those mothers who need to learn more about malaria control, immunization, CDD, and pneumonia case management. This finding was quite indicative of the progress made with the project’s IEC/BCC approaches to support project interventions. The enthusiasm that was demonstrated by mothers during the focus group interviews indicates how much they have learned and to what extent they would be willing to share their experience with other mothers so that they can save lives. This proves also that by training CLC members who actively promote IEC/BCC related activities, a community can support the learning of other community members through peer education and positive deviance approaches at low cost. (v) Factors Contributing to Achievement of IEC/BCC Objectives · APEs and TBAs made regular home visits to educate families; · Mothers continue to express the need for more presentations (palestras) by the theater groups and home visits by the TBAs and APEs; · Promotional IEC/BCC activities by the Mobile Brigades during vaccination activities at outreach sites; and · One-on-one and group counseling by trained health workers. C. Capacity Building Approach and Objectives The project has successfully provided critical training and capacity building of PVO partners including nurses, CLC members, TBAs, APEs, and the SC field level staff. Table 10: Summary Table of Progress with Capacity Building Related Activities Category of Personnel Number Trained and Supported Participants in Refresher Technical Courses Total Trained/Retrained CLC Members 322 322 322 APEs 14 14 14 Mobile Brigades 118 - 118 Socorristas (Village Heath Agents) 7 - 7 Traditional Birth Attendants (TBAs) 144 219 219 Traditional Healers 60 60 60 Note: Community health agents = CLC members (TBAs, Activistas, Régulos, and Traditional Healers) Capacity building within the project includes building the capacity of SC staff, local partners, health facilities, and health workers. Activities have included training (including participation in CS-16, Mozambique, Final Evaluation Save the Children 28 workshops), supervision, and follow-up of health workers, CLC members, TBAs, theater groups, and school children. Findings and Observations: The FE team observed that during the last year of project implementation, follow-up and refresher training for health workers was not carried out effectively due to difficulties with transportation to and from remote areas and staff turnover within the project and DHOs. (i) Strengthening SC as an Organization: The Strength Project was designed and implemented as a community and district level development project. The role of SC has been to improve the capacity of the DHOs and the communities to respond to their identified health needs. To achieve the project goals and objectives, SC staff has been working directly with the DHOs and with the community leaders and community-based health workers to promote and to provide selected child survival components. Through the implementation of CS-16, SC has demonstrated a commitment to strengthening its capacity for health systems development in addition to direct service delivery. It is also important to note that regular SC technical and administrative staff meetings took place at country and home office levels build staff capacity. At the provincial and district levels, SC country level staff met with their counterparts on a regular basis to plan and review progress with program activities. Those program review meetings contributed to the learning and information exchange about program performance. SC staff participated in country and regional level seminars that also contributed to the learning and capacity building. In addition, senior technical advisors from SC/HO conducted supervisory visits to Mozambique that also contributed to staff training and capacity building. (ii) Strengthening PVO Partners’ Performance: Under CS-16, the capacity of partners such as CLCs, APEs, and Activistas were strengthened through technical support and the provision of in￾service training that followed workshops in order to reinforce learning and improve service delivery. The strengthening of health worker performance through regular supervision has also taken place in combination with in-service training. The continued increase in the number of trained health workers, including TBAs, will further increase the capacity to plan, manage and implement activities, and thus contribute to the long￾term sustainability of program activities in the districts. Training and in-service approaches that have been used to increase the capacity of partners has worked well to increase knowledge, improve practices, and contribute to the positive impact of this program, as demonstrated during interviews with mothers and from the final KPC data. CS-16, Mozambique, Final Evaluation Save the Children 29 Below is a summary of accomplishments to date under Capacity Building. Table 11: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) SC/HO: At least 2 SC field offices and/or other organizations will develop a plan adopting successful CS-16 approaches Done and Ongoing The DHOs have been assisted to develop plans for implementation of the key interventions (EPI, malaria, CDD, etc.). The implementation of plans is dependent on having qualified staff at HC and HP levels as well as continued collaboration with Communities. (2) SC/MZ: SC/MZ Staff will participate in policy forums at the provincial level advocating for changes in policy based upon CS-16 strategies In progress Quarterly and semi-annual PMT meetings are taking place. (3) DHOs: Presence of annual work plan specifying activities, persons responsible, date and resources available for monitoring and supervisory tools for each capacity In progress More support is still required with budget and logistical support to implement monitoring, supervision and training/capacity building activities. (4) Presentation of HIS reports during quarterly PMT meetings Done and In progress Data collection, analysis and reporting still need to be improved to ensure quality. (5) Communities: (1) 75% of regulados will be able to identify 2 or more priority maternal and child health problems to be addressed through an Action Plan; (2) 70% of regulados will have formed CHTs and members, will know the roles and responsibilities; (3) 80% of CHT members will be able to cite 5 approaches to increase community participation in health activities; (4) 75% of CHT members will understand numerical trends of HIS and use of health planning. Done but not systematically measured to assess progress More support is still required to make CHTs more functional and able to carry out at their program responsibilities at the local level. Status of each of the targets/objectives is not known. A special study is required to determine the level of achievement for each of the objectives. (iii) Health Facilities Strengthening The project inputs were not directed toward the equipment of health facilities but rather toward the strengthening of health worker knowledge and skills to ensure effective delivery of essential health services. Most of the project efforts have also been directly related to improving the capacity Memba and Nacala-a-Velha Districts to implement child survival approaches and support community structures. To accomplish this, some joint planning, monitoring and supervision with the DDS staffs has been carried out. The focus of the district level support has been on training/capacity building of health workers so that they can use their technical and management skills to perform monitoring and supervision activities as well as collect and analyze data for the country’s HIS. This will then contribute to a more informed decision-making process. D. Sustainability Strategy and Objectives The anticipated increase in the number of trained health workers including TBAs, under the partnership between the Bridges-to-Health Project in Nampula and the MOH, will further increase health worker capacity to plan, manage and implement activities that will contribute to program objectives beyond the life of CS-16. CS-16, Mozambique, Final Evaluation Save the Children 30 The strategy that SC and its partners adopted to work through existing health facilities and the DHO personnel has worked well but needs reinforcing because there is a continued need for additional, qualified MOH staff at primary facilities level (health posts/health centers) in the districts. During field visits, the FE team observed that the groundwork for program sustainability has already been laid by the existing structure of the country’s primary health care system. What the project has provided through TA, support of the logistics of drugs and vaccines, and financial support, has enabled the DHO to establish a Program Management Team (PMT) designed to promote program planning, monitoring and evaluation, capacity building and sustainability of program activities. Due to MOH ownership of provincial and district-based health program activities (i.e. EPI), and the active participation of the DDS personnel in the implementation process (without additional compensation), it is believed that program sustainability of existing and future activities will continue to improve. Some of the program activities that appear sustainable include prenatal and postnatal care, and immunization programs. These program activities are already reflected in the existing MCH program of the MOH. What is needed is the provision of additional support in the areas of management and distribution of essential drugs, IEC materials for child spacing and disease prevention, and increased nutrition and malaria prevention activities to expand the scope of the DHO and the PMT program responsibilities. Table 12: Summary of Accomplishments to Date PLANNED ACTIVITIES/OBJECTIVES STATUS COMMENTS (1) 2 DHOs will prepare an annual health work plan indicating efficient use of resources. Done and in progress Due to a change in leadership at DHO level, very little has been accomplished with this objective but efforts are being made for its implementation. (2) 75% of CHTs at regulado level will attend meetings regularly. Done and In progress Essential CHT members attend the PMT meetings regularly. (3) 2 DHOs will have supervisory and monitoring systems for QOC and peer review. Not fully completed In progress Supervisory and monitoring system exists but related activities are not being carried out on regular basis due to lack of funds to support regular supervision activities in the districts. (4) DHOs will regularly share lessons learned at provincial and national MOH levels. In progress Lessons learned under the CS-16 program will be documented and shared with the provincial and national level MOH officials and partners (5) DHOs commit to sustaining CHT training and formation by inclusion into yearly DHO plans. In progress The CHT approach will be maintained under the ongoing DHO plans. Its activities will be sustained through the implementation of current and future programs in the districts. CS-16, Mozambique, Final Evaluation Save the Children 31 IV. PROGRAM MANAGEMENT A. Planning Findings and Observations: From document review and interviews with SC and district level staffs, the team learned that the original program planning for the project was done collaboratively in a workshop that included twenty people. Participants in the planning meeting included district and provincial health office representatives, community leaders, and the consultants. During project implementation, planning and review meetings were scheduled on a quarterly and monthly basis with the DHOs, to allow for the review and development of micro plans designed to facilitate project implementation at the community level. In addition to the overall work plan of activities that is contained in the DIP, annual work plans are developed in collaboration with the PMT and CHT. These annual work plans are further simplified by having quarterly review and planning meetings where key district health workers and officials participate to review accomplishments during the review period and to plan new activities that can be carried out during the following period. It is also important to note that SC is the NGO responsible for management and implementation of the USAID-funded Bridges-to-Health Project activities in Nampula. This Bridges-to-Health Project has similar program objectives, especially in the areas of training, capacity building, monitoring and supervision of program activities, and the overall provision of maternal and child health services. Because of this, it is practical to include Strength Project staff in the quarterly review and planning meetings that include representatives from six districts that are supported by the Bridges-to-Health Project. This joint review and planning of the provincial health program activities has facilitated the collaboration between the neighboring districts and has also enabled other lower performing districts to learn from one another about how to carry out program activities. The FE team learned also that: · An Action Plan of program activities that was developed in December 2002 by the Health Program Manager in collaboration with the DHOs to respond to the midterm evaluation recommendations could not be carried out fully because of CS-16 budgetary constraints. · The Strength Project Coordinator and Health Program Manager worked closely with their counterparts in the districts to prepare for the quarterly PMT/EGP meetings. These joint planning exercises held before the quarterly PMT/EGP meetings have contributed to the learning of the district level management and technical staff. These meetings should be continued because the quality of reports, including HIS data that are produced, contain valuable information that contribute to the achievement of program objectives in the two districts. · Due to recent change at the DDS for Nacala-a-Velha, the new Director who did not participate in the planning of the original project has made considerable efforts to learn and understand the project goals and objectives. He has actively participated in the development of quarterly and annual work plans to support project implementation. · Monitoring and supervision activities were planned jointly with the DHOs and the Community Health Teams (CHT), but due to transportation constraints, only a few joint supervision activities have been carried out. CS-16, Mozambique, Final Evaluation Save the Children 32 · During meetings with the project staff, the team also learned that although adequate planning had been carried out to ensure timely procurement and delivery of essential project commodities, the actual procurement and delivery of some items, such as bicycles ambulances and a new project vehicle, was not done in a timely fashion. These items were procured and delivered to the project areas during the last year of project implementation. · The FE team observed that although most of the communities have functioning CLCs, not all of them have established effective BA maintenance and repair plans. Conclusion: A more effective planning for the procurement of project commodities should have been done during the first year of project implementation. Without reliable transportation, very little could be done to support monitoring and supervision activities in the districts. B. Staff Training Findings and Observations: Adequate resources have been used to train SC and the district￾based health staff to carry out project activities. SC program staff also received frequent opportunities to improve their knowledge, skills, and competencies through district, provincial or country level workshops, on-the-job training, and supervision from country coordinators and HO staff. Program staff did mention that they would like to participate in more domestic and regional conferences in their respective professional areas. DPS partners participate in the above trainings as well as opportunities for additional workshops and seminars. From the team’s participation in the recent PMT meeting in Nacala-Porto, the FE team noted that SC and the district-based staffs have strong management and planning skills that enable them to discuss planning and reporting of field level activities. Some of the field level training that has been provided to health workers and the community￾based providers (APEs, TBAs, socorristas, etc.) include: (1) Refresher training of APEs and socorristas to diagnose and refer children with pneumonia to the nearest health post or health center for treatment. (2) Train traditional healers to recognize pneumonia and refer clients to the nearest health post or health center for treatment. (3) Train socorristas to recognize children with fever and refer them to the nearest health post or health center for treatment. (4) Train health post/health center nurses in the MOH IMCI protocol. (5) Training of CLC members on case management of diarrhea so that they can educate mothers and community members on the importance of increasing children’s fluid intake during diarrheal episodes. (6) Refresher training for APEs and socorristas on the case management of diarrhea to prevent dehydration. (7) Promote the use of ORT by mothers and the community service providers for management of diarrhea. CS-16, Mozambique, Final Evaluation Save the Children 33 C. Staff Supervision Findings and Observations: Directing and supporting staff has been performed by the Child Survival Field Coordinator and Program Manager, and by the principal DDS or DPS officials (i.e., Chief Medical Officer, District Director, and Chief of MCH services). Frequent joint meetings with SC staff and DPS/DDS staff provided additional support. From document review and interviews with DHO personnel, the FE team learned that the number of trained personnel that is currently assigned to district-based facilities is very low. Not enough qualified nurses are available in the province and in the country in general, to assume program responsibility in rural areas. Because of this, SC has been required to train and work with the available personnel including basic level nurses (APEs), TBAs, and Activistas. All these categories of personnel require further supervision and training by SC and the DHOs. Conclusion: As indicated in earlier sections, joint supervision activities by SC staff and their counterparts at DHOs have not been as regular as expected in the districts due to a lack of adequate means of transportation. The need to increase supervision activities is continuously being addressed by SC and the DHOs. D. Human Resources and Staff Management Findings and Observations: CS-16 personnel needs were well planned for, however the actual number of MCH nurses recruited and assigned to the district health centers and offices, has been insufficient to support the Strength Project activities at the district and community levels. In addition to assigning three MCH nurses to DHOs, SC should have assigned its own field coordinators to be based in the DHOs to ensure closer collaboration in planning, joint supervision and increased logistical support. This would have facilitated project implementation and the transfer of skills and experience to the local DDS staff during the life of the project. Conclusion: The vacancy of the Project Field Program Coordinator position from July to December 2002 created a vacuum in program management and implementation of field level activities in the two districts. Fortunately, SC/MZ managed to recruit and assign two valuable replacement staffs (one Field Program Coordinator, and one Health Program Manager) who have been working extensively with their counterparts since January 2003 to support project implementation. E. Financial Management Findings and Observations: Due to time constraints, the team did not fully review the project budget and expenditures, however efforts were made to look at the issues of adequacy of funds to support planned project activities. From the team’s interview with the accountant and bookkeeper, it is clear that management and finances are in accordance with approved line-item budgets, Save the Children’s policies and procedures, and USAID’s policies regarding financial management. Financial management is being ensured at three levels (SC/HO, Country Office, and the Field Project Office). CS-16, Mozambique, Final Evaluation Save the Children 34 The FE team learned that due to a cost overrun that occurred with some line items during the second year of the project, funds that remained in December 2002 were not enough to cover all of the planned activities for the third project year. Some of the CY 2003 activities had to be reduced or cancelled in order to support remaining program activities in the districts. Conclusion and Recommendation: A more careful review of the project finances during the midterm evaluation held in August 2002 would have helped to verify the availability of funds for program activities through September 30, 2003. F. Logistics Findings and Observations: Both Nacala-a-Velha and Memba Districts are isolated and have poor road conditions. Well-equipped vehicles are necessary to support field level activities including supervision and outreach activities by MCH nurses and the Mobile Brigades. Lack of sufficient transportation created significant difficulties because the vehicles that were acquired from the former project were old and were inadequately maintained to support the demand for transportation in the districts. Due to this lack of vehicles, some of the planned activities were canceled or delayed. Without a reliable means of transportation, the joint supervision activities of APEs, TBAs, and CLC members in the communities, could not be carried out. Conclusion and Lessons Learned: · To ensure adequate support of field level activities in the districts, one new vehicle and a driver should be assigned to both the DHO of Memba and Nacala-a-Velha. · SC and the DHOs could have used other alternative means of transport such as motorcycles to support and promote supervision activities by the DDS and the MCH nurses. · Due to the prevailing transportation difficulties, Mobile Brigades visits should only be made to areas that are not easily covered by health centers/health posts. G. Information Management The FE team observed that the districts have a functioning health information system (HIS) that needs strengthening. One of the roles of SC staff has been to work with their counterparts at the DHO and community health team (CHT) levels to improve routinely collected data and information to: (1) assess whether current activities are effective in achieving program objectives, and (2) in instances when there is not evidence of progress, determine how program activities can be modified to attain the desired beneficiary-level results. The team also noted that the Strength Project results have been monitored and evaluated using community data, service statistics, qualitative studies, supervisory checklists, KPC surveys, and Health Facility Assessments. Prior to conducting the FE, SC and its counterparts carried out the final KPC survey that generated data and information that is being shared with partners. CS-16, Mozambique, Final Evaluation Save the Children 35 Beyond the CS-16 program activities, data and information collected from the project areas have been consistently used for region-wide review, planning and implementation of the Bridges-to￾Health Project for Nampula Province. This regional approach to program activity planning contributes to the integration of projects and the promotion of program sustainability by USAID and the MOH. Conclusion and Recommendation: Progress has been made with HMIS but a great deal remains to be done in the area of data collection, analysis, and the sharing of HIS at the community level to improve planning and decision making. H. Technical and Administrative Support During the life of the project, some key technical and administrative support was planned and provided to the project in timely fashion. This includes: SC/HO Support: At SC/HO, a Child Survival Specialist (Eric Swedberg) has been available to support the planning of program activities including the provision of needed short and long-term assistance to the project. Table 13: Summary Table of Short-Term TA Support to the Project Type of Technical Assistance Consultant Start/End Detailed Implementation Planning Joseph de Graft-Johnson and Eric Swedberg March 2001 Production/promotion of clean Birth Kits Joseph de Graft-Johnson Oct. – Nov. 2001 Marketing of Birth Kits Suzanne Smith, SC December 2001 Lot Quality Assurance Sampling Joseph de Graft-Johnson April 2002 Midterm Evaluation Eric Swedberg, SC Ana Paula de Morais Oppenheimer August 2002 Final Evaluation Armand L. Utshudi September 2003 V. OTHER ISSUES IDENTIFIED BY THE TEAM During field visits to Nacala-a-Velha and Memba Districts, the FE team observed that SC had already established a field level Development Area Program (DAP) Office in Nacala Porto. Staffing included a chief of logistics, secretaries, receptionist, bookkeeper, agronomists, and drivers. The office space was being shared with the CS-16 program staff who depended heavily on the DAP staff for logistical and administrative support. From the discussions held with the SC and the DAP field staffs, it was learned that the DAP has been in the area of Nacala Porto for more than three years and that they are expected to remain there for an additional two years. The DAP focus has been on agriculture with special interest in improving food security in the communities. From the interview conducted with the health program staff, the FE team learned that although DAP/Food Security programs had different funding sources and overall goals, there are aspects of the two programs that are complementary and that could have been emphasized to ensure closer collaboration among the two program teams in Nampula Province. Conclusion and Recommendation: There is an opportunity to improve collaboration between the two USAID-funded programs in Nacala through joint planning and implementation of nutrition related activities. Since DAP/FS program areas overlap with CS-16 and the Bridges-to￾Health program activities, efforts should be made to jointly plan and implement growth CS-16, Mozambique, Final Evaluation Save the Children 36 monitoring and nutrition education activities that can contribute to improved child health outcomes. Valuable lessons can be learned from this experience with respect to the advantages and challenges of integrating DAP/FS and CS efforts. Lessons learned from such an integrated program approach could be shared with the broader PVO community, as well as donor agencies, in the form of case study in Nampula Province. VI. CONCLUSIONS AND RECOMMENDATIONS A. Overall Achievement of Objectives and Project Status The project made excellent progress toward the achievement of program goals and objectives during the life of the project. The FE team was able to verify progress through document review and visits to the project sites of Memba and Nacala-a-Velha. During field visits, the team also observed the impact this program has had on the knowledge of mothers and care takers in the communities relative to immunization, prenatal consultations, case management of diarrhea at home, establishment of community plans to address emergency obstetrics, and the increased number of deliveries assisted by trained health workers (TBAs and nurses). The project should be commended for achieving the above results in the face of major difficulties and constraints. These include; turnover at the DHO and SC field staff levels, the fact that the project has been implemented in the most difficult areas of the country due to poor road conditions, and a lack of sufficiently trained personnel at the district-based facilities to assist with facility and community-based child survival activities. B. Important Achievements and Constraints The following is a list of major accomplishments under the project during the three-year life of the Strength Project: · 46 CLCs have been established to participate in the planning and implementation of community-based activities; · 16 health facilities in the two districts have at least one trained health worker to assist with births and the promotion of child survival activities; · 322 CLC members have been trained to support and promote child survival program activities; · 144 TBAs have been trained and 219 have participated in refresher courses organized under the project to support and promote maternal and newborn care at the community level; · 60 Traditional Healers have been trained to support and promote appropriate child health services including referrals; · 28 APEs have been trained to assist with the provision of quality basic health services at the community level; · 19 selected communities have received bicycle ambulances and have plans to address obstetric emergencies in the communities; · 75 communities have benefited from child spacing related activities; and · 158 communities have received support from Mobile Brigade outreach activities in the districts. CS-16, Mozambique, Final Evaluation Save the Children 37 Major constraints and challenges that have been encountered and that are being addressed in collaboration with the MOH include: · Staff turnover at DHOs and SC field program levels slowed down project implementation but efforts have been made to address this; · Seasonal access which prohibits Mobile Brigade travel to and from remote areas to support outreach activities continues; · There is a continued lack of adequate number of qualified nurses to assist with program activities at the health facility and community levels; · Frequent stockouts of essential drugs continue to have a negative impact on the planning and implementation of IMCI-related activities in the districts; · There is a need to improve the logistics of vaccines and the cold chain system to support vaccination programs and Vitamin A distribution activities in the districts; and · The project is scheduled to end on September 30, 2003. However, recently trained CLC members (including TBAs, APEs, Community Leaders, and nurses) still need further follow up and support from project staff and district level health personnel so that they can serve as effective trainers and promoters of health and hygiene activities in the communities. C. Lessons Learned · Increasing the number of trained health workers at district-based health facilities to provide counseling and health education, and deliver curative and preventive health services can contribute to improved health of mothers and children in the communities. · Community traditions that are deterrents to improved child health can be changed through training and motivation of community members, and through good dialogue between trained TBAs, traditional healers, and the CLC members to support behavior change. · Bicycle ambulances are an acceptable and sustainable means of transportation for transporting people (including women) with medical emergencies to nearby facilities for appropriate medical care. · Very little can be accomplished to improve vaccine coverage of the target population in the districts without adequate planning and supervision, and an enhanced vaccine logistics and cold chain system in place. · Mobile Brigades are an appropriate approach to extend the provision of health services to areas outside the reach of health facilities. Their services must be regularly scheduled and coordinated with the CLCs, with at least 4-5 visits per year to ensure increased vaccine coverage of the target population. · Assigning one new vehicle to Memba and one vehicle to Nacala-a-Velha to support the Mobile Brigades’ community-based activities would improve transportation in the two target districts. · Knowledge about malaria case management and prevention can be increased among the target population by increasing group-counseling activities and by the involvement of theater groups and trained CLC members in IEC/BCC activities. · Case management of malaria and pneumonia at home and at health facilities can be assured by improved logistics of essential drugs (Chloroquine, Fansidar (Sulfadoxine and Pyrimethamine, and antibiotics) at district-based health facilities. · Involving other partners (UNICEF, UNFPA, the Dutch Embassy, NGOs, and the MOH, etc.) in support of IMCI program activities that complement partners’ overall goals and objectives, CS-16, Mozambique, Final Evaluation Save the Children 38 during budgetary shortfalls, to support the expanded program activities and contribute to the achievement of IMCI related targets. · Oral rehydration therapy practice combined with the promotion of breastfeeding, immunization, and nutritional supplement during diarrhea, can dramatically reduce mortality caused by diarrheal diseases. · The following three key behavior change practices could have been emphasized in the project area as part of primary prevention of diarrhea: hand washing at critical times and with appropriate technique, sanitary disposal of feces, and protection of drinking water from fecal contamination. · Creating an enabling environment through training of the community to increase awareness about the need for child spacing (including increased involvement of men in child health related activities) can facilitate behavior change. D. Recommendations for USAID (1) USAID/Mozambique should consider providing additional support to Memba and Nacala-a￾Velha Districts so that USAID’s investment in the districts can be expanded, sustained, and fully integrated with the ongoing DAP and other planned development activities in the two districts. (2) USAID/Mozambique should be prepared to supplement funds from other donors to support integrated public health approaches that include: (a) case management of malaria, diarrhea, and pneumonia; (b) disease surveillance; and (c) water and sanitation activities in remote areas of Nampula Province. E. Recommendations for Partners (Including Save the Children) (1) Nampula Province is the second largest province in Mozambique but has not received support proportional to its size or the needs of its population. For this reason, it is recommended that donors (including NGOs) and the MOH work together toward the establishment of an integrated plan for the development of Nampula that includes agriculture, health and rural development. (2) UNICEF/Mozambique should conduct the inventory of cold chain equipment in the districts of Memba and Nacala-a-Velha, and provide needed cold chain equipment to support vaccine logistics for the districts. (3) UNFPA/Mozambique should be prepared to supplement USAID/Mozambique assistance in the two districts with the equipment for maternal and newborn care activities at the district-based health facilities. These include health posts, health centers, and district hospitals that serve as primary and secondary referral facilities in the districts for obstetrics emergencies. (4) MOH (Provincial and District Directorates) should work with SC to develop a plan to improve water and sanitation activities in Nampula that can be funded by donors (USAID, UNICEF, etc.). Without additional investments in this area (water and sanitation at the village level), the recent investment and successes under the CDD component of CS-16 would not contribute to sustainable outcomes. (5) MOH (DPS/DDS) should continue to recruit and assign trained nurses to district-based health facilities to support the community-based child survival activities. This action, including the efforts that have been carried out at the community level, will contribute to the sustainability of project activities that have been carried out to date. CS-16, Mozambique, Final Evaluation Save the Children 39 (6) Save the Children/US should work with the MOH to translate the Detailed Implementation Plan (DIP) into a language or terms that can be understood and used as instrument for project implementation by the field level MOH staffs. F. Plan for Dissemination of Information by PVO/HO Save the Children/US has a number of mechanisms to absorb, apply, and share lessons learned and results from CS-16. Internally, SC has the “Program Learning Group” (PLG) with members from both the SC field and home offices who are international health practitioners and communicate regularly by e-mail and telephone. One of the objectives of the PLG is to share lessons learned and best practices in health programming and innovative ideas on addressing implementation or operations challenges, and monitoring and evaluation issues. Jeanne Koepsell and Eric Swedberg are SC/HO staff members with field level experience who will continue to share the CS-16 experience through the PLG. Other mechanisms to share lessons learned are through the presentation of the lessons learned at regional and country level workshops and conferences. The SC experience working with Activistas under CS-12 and more recently with CLC members under CS-16, is well documented and should be shared with other partners through publications that include lessons learned, and presentations at organized regional and international conferences, including the CORE monitoring and evaluation working group. CS-16, Mozambique, Final Evaluation Save the Children 40 VII. RESULTS HIGHLIGHT The CS-16 program activities have been successfully implemented by SC in Memba and Nacala￾a-Velha, in collaboration with the provincial and district level health workers. The Strength Project focused on: (a) the establishment of system strengthening through the establishment of a Program Management Team (PMT) at the district level to enhance SC-DHO partnership for program planning, monitoring, evaluation and sustainability of program activities; and (b) the establishment of Community Health Teams (CHTs) at the community level through strengthening the community component of MCH to monitor and support the activities of the community health workers. The above two aspects of program activities were reviewed during the FE and it was confirmed through interviews and document review that this novel approach that focused on capacity building of health workers at the district and community levels has been a success. This innovative approach should be shared with partners in Mozambique so that it can be replicated in other parts of Mozambique, where such an approach has not been fully tested. Reinforcing the community component of the program that focused on working with CLCs as a functional unit rather than depend on community volunteers (i.e. Activistas), has proven very successful and sustainable because leadership already exists around the community leaders. The only remaining issue is the lack of additional support provided through training, and capacity building of members so that they can work together to address the needs and priorities of the constituent communities. From the review of progress towards the achievement of objectives, the team concluded that: · Targeted training and capacity building of MCH nurses, TBAs, APEs contribute to the establishment and expansion of a strong referral system for child survival and maternal care services in Memba and Nacala-a-Velha Districts. · Training and support that has been provided to CLC team members and the theater groups in the two districts have also contributed to the increase in the women’s knowledge about participation in prenatal and postnatal care. · The project staff and partners have learned that in areas where there is low capacity and shortage of personnel, such as in Nacala-a-Velha and Memba Districts, training alone without regular monitoring and follow up supervision of trained health workers and CLC members, does not significantly contribute to the learning and delivery of quality services. · Project staff and partners have also learned that providing an enabling environment to community members through CLC members and health workers for information disseminate contributes to behavior change. This was essential for community participation in program activities and the success of CS-16. During the last three years of project implementation, the project has been able to achieve most of the program objectives with varying degree of success. Some highlights of accomplishments to date are summarized as follows: CS-16, Mozambique, Final Evaluation Save the Children 41 Table 14: Summary Table of Selected Project Indicators: Objectives and Desired Outcome/Practice Baseline 12/2000 Final KPC 08/2003 (average of both districts) 55% of children 12-23 months will be fully immunized by age 12 months. 32% 55% 50% of births attended by trained personnel. 34% 52% 50% of women with children < 24 months will know at least two modern methods for child spacing. 24% 53% 80% of mothers with children < 24 months with fever in previous 2 weeks sought care within 48 hours. 66% 80% 85% of women have at least two antenatal visits with trained health personnel during their last pregnancy. 74% 85% Attachments A. The Evaluation Team Members and Their Titles B. Final Evaluation Schedule C. List of Persons Contacted and Interviewed D. Final Evaluation Questionnaires CS-16, Mozambique, Final Evaluation Save the Children 43 Attachment A The Evaluation Team Members and Their Titles The team members for final evaluation of the Strength Project in Nampula, Mozambique included the following participants: Mr. Armand L. Utshudi, Pharm., and MPH, External Consultant and Team Leader Mr. Salazar Portugal, Health District Director, Nacala-a-Velha District Ms. Ellen Warming, Health Program Manager, SC/Maputo Ms. Greta Stina, Field Program Coordinator, SC/Nacala Porto Ms. Maria Adelia Matchine, Nurse and Assistant Field Program Coordinator Ms. Christina José Cardoso, Nurse and Project Assistant Ms. Adelina Armando Xavier, Nurse and Project Assistant Ms. Maria Manaquela, Nurse and Project Assistant During field visits to the districts, the above team members were divided in two groups. Each group was assigned to a particular district to administer a questionnaire and collect qualitative data and information that have been used to prepare the final evaluation. CS-16, Mozambique, Final Evaluation Save the Children 44 Attachment B Final Evaluation Schedule Date Morning Afternoon Monday 09/01/03 Briefing with key Save the Children Country Office technical and Administrative staff (Peter Nkhonjera, Ellen Warming, and El-Nour Omer El-Basha). Document review and more follow up meetings with the Health Program Manager (Ellen Warming). Tuesday 09/02/03 Travel to Nampula and take part in a courtesy meeting with the DPS/Chief Medical Officer (courtesy meeting). Travel by road to Nacala Porto; meet with SC Nacala Porto District staff to plan for provincial planning meeting with partners. Wednesday 09/03/03 Provincial planning/review meeting with USAID, MOH, Helen Keller, and the Advance Africa project. Continue provincial planning meeting with partners, and prepare for initial meeting with the Evaluation Team members. Thursday 09/04/03 More follow up planning/review meetings with partners in Nacala Porto and finalize plans for initial meeting with the Evaluation Team. Participate in the initial planning meeting with the evaluation team members to review team composition, scope of work, and finalize schedule for final evaluation activities. Friday 09/05/03 Meet with the District Health Officials; more follow up meeting with team members to present and discuss findings of the final KPC for Nacala-a-Velha and Memba Districts. - Review and summarize accomplishments to date based on project reports and interviews with key staff; - Develop questionnaire to collect qualitative data and information during field visits; - Discuss the logistics of field visits and divide the team in two groups for travel to the two nearby districts. Saturday 09/06/03 Field visits to Nacala-a-Velha and Memba Districts to observe project activities including drama for BCC activities. Other activities include interview with CLCs, mothers following encounter with nurses at health facilities; health workers to obtain qualitative information on CS-16 activities. Return from field visits; teams meet to debrief on findings and conclusions of the field visits. Sunday 09/07/03 Day off Day off Monday 09/08/03 - More interviews with SC staff as required - Debriefing on the results of the field visits - Qualitative data/information review and analysis. - Formulation of draft conclusions and recommendations. - More formulation of conclusions and recommendations; - Data review and analysis; - Start drafting of the report with support from assigned team members. Tuesday 09/09/03 More data analysis and writing assignments between the Team Leader and the Project Manager More analysis and writing with support from assigned team members (Ellen and Greta) Wednesday 09/10/03 First draft conclusions and recommendations are shared with team members for review, discussion, and revisions. Continue drafting of key sections of the report and prepare for debriefing with DDS, DPS, and partners. Thursday 09/11/03 Debrief with MOH and SC staff; prepare final version of debriefing materials for USAID/Mozambique (HPN Office) Travel to Nampula by road from Nacala Porto Friday 09/12/03 Debrief with DPS Travel to Maputo Saturday 09/13/03 Travel to the US start in Maputo in late Afternoon Overnight in South Africa Sunday 09/14/03 Travel to the US start in South Africa Reagan National Airport CS-16, Mozambique, Final Evaluation Save the Children 45 Attachment C List of Persons Contacted and Interviewed Dr. Alberto Vaquina, Provincial Director of Health, Nampula Province Dr. Anselmo Tomas, Chief Medical Officer, Nampula Province Mr. John Mitchell, Country Director, SC/Mozambique Mr. Peter Nkhonjera, Deputy Director, SC/Mozambique Sra. Crina Ismael, Helen Keller Representative, Maputo, Mozambique Sra. Lydia Cardoso, USAID/Mozambique, HPN Office Dr. Titus Angi, Child Survival Specialist, USAID/Mozambique HPN Office Dra. Catarina Regina, SC Project Coordinator, Bridge to Health Project Sra. Greta Stina, Field Program Coordinator Sra. Ellen Warming, Health Program Manager, SC/Mozambique Sr. Salazar Portugal, District Director of Health, Nacala-a-Velha Mr. Steve McSween, Agronomist, SC/Nacala, Mozambique Ms. Maria Adelia Matchine, Assistant Field Program Coordinator, the Strength Project Ms. Christina José Cardoso, Nurse and Project Assistant, the Strength Project Ms. Adelina Armando Xavier, Nurse and Project Assistant, the Strength Project Ms. Maria Manaquela, Nurse and Project Assistant, the Strength Project CS-16, Mozambique, Final Evaluation Save the Children 46 Attachment D QUESTIONNAIRE FOR CLC MEMBERS (QUESTIONÁRIOS PARA O CLCs) 1. As CLCs tem tido encontros regulares? 2. Quantas vezes as CLCs se reuniu durante o último ano? 3. O que trataram neste(s) encontro(s)? 4. Qual o Plano de trabalho que vocês tem? Que é que já fizeram? 5. Que problemas encontraram? 6. Como fazem o registo de informação de Saúde Comunidade? 7. Qual o propósito de fazer o registo? 8. Qual é a relação existente entre as DDS e as CLCs? 9. Contar as mulheres do CLCS (perguntar se falta algum membro). Contar todas as pessoas presentes do CLCS 10. Qual é o papel / responsabilidade de cada membro? 11. Qual e o conselho que o CLCS dá para as mulheres gravidas? 12. O que que CLCS faz (ou pode fazer) para aumentar os partos assistidos por pessoal treinado? 13. O que é que os CLCs recomendam a uma mulher grávida no sentido de preparar o seu Plano de parto? 14. Por acaso tem um sistema de ALARME e TRANSPORTE para as gravidas? Perguntar se ainda existe as Bicicletas? Como é que se comunicam entre o doente e o responsável de transporte? Como é que os CLCs fazem para sustentar o seu transporte e continuar a utilizar? 15. O que é que os CLCs fazem para que as mamas conheçam os metódos do P.F? 16. Que fazem para aumentar as crianças vacinadas? 17. Quando foi a última vez que tiveram brigada móvel - BM? 18. Muitas mulheres apresentam-se nas consultas pré – natais mas nem todas são vacinadas contra tétano. Como se pode explicar? 19. Como tem sido a participação dos membros (da CLCs)? CS-16, Mozambique, Final Evaluation Save the Children 47 QUESTIONNAIRE FOR MOTHERS (QUESTIONÁRIO PARA AS MÃES) PARTO E PRE-NATAL 1. Perguntar as mães sobre o que preparam para o seu parto? 2. Porque não chamam a PT para assistir o parto? 3. O que podem fazer para aumentar a utilização das PT? 4. Já ouviu a falar-se dum sistema de alarme e transporte? 5. Tem benefíciado do sistema de alarme e transporte da sua comunidade? 6. Como são atendidas nas consultas Pré-Natais? 7. Já ouviram falar de PF? Que metódos ouviram falar? Porque não usam? 8. Quem e que vos falou dos metódos? 9. O que é que a comunidade pensa acerca do PF? 10.Qual é o intervalo apropriado entre os nascimentos? 11.O que pode ser feito para o aumentar o uso de PF? DIARREIA: 1.1 O que você faz em casa quando a sua criança apanha diarreia? Onde apanha o SRO? 1.2 Como e que se pode prevenir a diarreia? 1.3 Quando é que leva a sua criança imediatamente para P.S. / Centro de Saúde? (Quais são os sinais graves da diarreia?) 1.4 O que não se deve dar quando a criança está com diarreia? IRA/PN 2.1. Como reconhecem os sinais de IRA? 2.2. O que se faz quando a sua criança tem doença respiratórias? 2.3. Quando é que leva imediatamente ao P.S. / Centro de Saúde? 2.4. Como trata as doenças respiratórias em casa? O que não deve dar a criança doente de IRA? 2.5. Onde apanha o tratamento? 2.6. Quais são os sinais graves da penumonia (IRA)? FEBRE: 3.1 O que faz quando a sua criança tem febre? 3.2 Quais os sinais e Sintomas da Malária na criança? 3.3 Como se transmite a Malária? 3.4 Como se pode prevenir a malaria? 3.5 O que fazer quando a criança tem febres altas? ( corpo muito quente )? CS-16, Mozambique, Final Evaluation Save the Children 48 VACINAÇÃO: 4.1 Para que serve a vacina? 4.2 Quantas vezes levou a sua criança a Vacinação? 4.3 O que acontece quando a criança não apanha vacina? MUDANÇA DE COMPORTAMENTO 5.1 Nós ouvimos que há mães que levam as suas crianças doentes com febre ao hospital mas outras não. Porque as outras mães não levam ao os filhos com febre ao hospital? 5.2 Nós ouvimos que outras mães não levam a sua criança a vacinação, Porque? 5.3 Muitas mães quando a criança de febre porque não levam logo ao hospital? Porque? 5.4 Muitas mães quando a criança tem doença respiratória porque não levam imediatamente ao Hospital? 5.5 Ouvimos dizer que quando as crianças tem diarreia, muitas mães não dão água, comida, mama etc.? Porque? 5.6 Quando foi ultima vez que mãe ouviu falar sobre Saúde? 5.7 Que assunto se falou? 5.8 Quem e que falou e que forma (método) e que usou CS-16, Mozambique, Final Evaluation Save the Children 49 QUESTIONNAIRE FOR TRADITIONAL HEALERS (QUESTIONARIO PARA CURANDEIROS) 1. Quais as doenças em mulheres e crianças que acha que tem de transferir para as unidades sanitárias? Que sinais? 2. Qual o material que utiliza para vacinar os doentes? 3. Por exemplo: uma lâmina que serve para vacinar quantas pessoas? 4. Como limpa o material de vacinar ? e que mais ? 5. Como faz quando recebe criança com diarreia? 6. E se for criança com dificuldade de respirar? CS-16, Mozambique, Final Evaluation Save the Children 50 QUESTIONAIRE FOR TBAs (QUESTIONARIO PARA PARTEIRAS TRADICIONAIS) 1. Quantos partos assistiu no mes passado? 2. Destes quantos partos foram complicados? 3. O que fez nesses casos? 4. Que dificuldados encontra no trabalho do dia a dia? 5. Qual e a sua relacao de trabalho com a Parteira Elementar? 6. Como laqueiam o cordao? 7. Onde arranjam material para o corte do cordao? 8. Quando decide levar uma mulher gravida ao hospital? 9. Quando decide levar uma mulher em parto para o hospital? 10.Quando decide levar uma mulher pos-parto para o hospital? 11.Como ajuda a mulher que temproblema de parto a chegar ate ao hospital? 12.Tem ficha para registar: nascimentos, mortes, etc… 13.O que fazem com os dados desse registo. 14.O que acontece na comunidade quando morre uma mulher de gravidez ou parto? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 51 QUESTIONNAIRE FOR ACTIVISTAS (QUESTIONARIOS PARA ACTIVISTAS) 1. Comprende o conceito de mudanca de comportamento? 2. Diga as menssagens chave da a. Malaria (fever); b. IRA c. Diarrhea 3. Que material utiliza para a educacao dos seus pacientes? 4. Recebeu treino sobre tecnicas de mobilizacao da comunidade? 5. Que maneira faz para aumentar a participacao dos participantes da palestras? 6. Qual e o seu calendario de trabalho em cada semana? 7. Em que dias faz as palestra visitas domiciliarias? 8. O que voce faz depois de receber treinamento da DDS e/ou SC? 9. Que informacao (dados) e’ que voce regista? Onde regista? O que faz com esses dados? 10.Acha que o seu trabalho e’ effectivo? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 52 QUESTIONNAIRE FOR NURSES AND MEDICAL ASSISTANTS (QUESTIONARIO PARA OS TÉCNICO DE SAÚDE: SMI/CONSULTA DE CRIANÇAS) 1. Recebeu treino sobre AIDI 2. Quando recebeu a ultima supervisao sobre AIDI? 3. O treinamento ajudou a melhorar o seu trabalho? Como? 4. No mes passado quantos contactos teve com APE, PT, CLCS? 5. Porque muitas as mulheres que vao a consulta prenatal nao recebem VAT? 6. Porque que as vacinacoes estao tao baixas? 7. Porque as mulheres nao usam o PF 8. Que maneiras pode fazer para aumentar a numero de mulheres que usam PF? 9. Tem cloroquina? Para quanto tempo? Ultima rotura? 10.Tem antibioticos? Para quanto tempo? Ultima rotura 11. Tem soros IV? Para quanto tempo? Ultima rotura? 12. Quais sao os materiais que usa para a educacao e aconselhamento de pacientes para febre, IRA, diarrea, PF,e sinais de perigo da gravidez? Pode mostrar? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 53 QUESTIONNAIRE FOR COMMUNITY HEALTH WORKERS: APE/SOCORRISTAS (QUESTIONARIOS PARA TRABALHADORES COMUNITÁRIOS DE SAÚDE: APE/SOCORRISTAS) 1. Sabe sobre o sistema de Alarme e Transporte da comunidade? 2. Como e que envia doentes do PS para o Centro ou Hospital? 3. Como trata um caso de febre? 4. Que aprendeu durante o treinamento? 5. Que conselhos e que voce da a mae da crianca doente? 6. Que e que faz em caso de crianca com respiracao rapida? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 54 Perguntas aos Régulos Discussão em grupo 1. Se você ha de fazer uma comparação de hoje e de três anos atrás sobre a saúde da comunidade. Qual e a diferencia? De que maneira e que você esta a contribuir para esta mudança? 2. Há mais ou há menos casos de diarreia na comunidade hoje do que era a situação de três anos atrás? De que maneira e que você esta a contribuir para esta mudança? 3. Qual são as actividades em relação a saúde da comunidade que foram iniciadas durante estes três últimos anos passados? O que e que funcionou bem e o que foi menos bom? 4. Quem e que seleccionou as activistas? Como? Eles estão a funcionar com satisfação? 5. Quem foi o treinador/formador destas activistas? Como? Esta a funcionar com satisfação? 6. De que maneira mostra-se o agradecimento das pessoas da comunidade através as actividades feitas pelas activistas. Que tipo de compensação ou motivação existe para as Activistas? É suficiente? Se não – o que mais pode ser feito? 7. Quais são as dificuldades que você estava a enfrentar desde no início do treinamento das activistas? Como consegue sobressaltar estas dificuldades? 8. O que e que você pensa sobre as Brigadas Moveis? Tem algumas sugestões para um melhoramento das actividades das Brigadas Moveis? 9. Se você tem uma criança gravemente doente ou uma mãe gravemente doente na sua comunidade – O que e que você costumar a fazer para levar esta pessoa doente para um posto ou centro de saúde? O sistema funciona bem? 10.Qual são os problemas principais do comportamento de saúde, que o Projecto Força esta a tentar mudar. 11.Quem são as pessoas alvo para esta mudança comportamental? 12.Quais são as actividades feitas para implementar estas mudanças comportamental? 13.Estas actividades – conseguiram mudar o comportamento das pessoas dentro da sua área? Como e que se sabe? 14.Existe outras actividades que podem ser feitas pelo o projecto Força para tornar um comportamento fraca para um comportamento satisfatória? 15.Quais são dos factores/actividades necessários para convencer as pessoas de fazer uma mudança comportamental de saúde? 16.Baseado na sua experiência – qual será o conselho que você daria a uma comunidade que queres introduzir um programa semelhante com Projecto Força? 17.Você teve algumas sugestões às Parteiras Tradicionais ou às Activistas em termos de informação que você queres eles a entregar a você? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 55 18.Que tipo de decisões e que você estava a fazer baseando-se nas informações que recebeu das PTs e das Activistas. 19.Existe um sistema de monitorização para medir a saúde dentro da sua comunidade? Se exciste, por favor explicar como. 20.Descrever as suas relações com o Posto de saúde e o Centro de saúde. O pessoal de saúde estão atente às suas problemas e necessidades? 21.Quais são os problemas que você encontrou em relação a este componente do projecto. 22.Em geral, quero perguntar a você se tem alguma coisa importante para dizer a nos? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 56 QUESTIONNAIRE FOR HEALTH WORKERS (Perguntas para entrevista com trabalhadores de Saúde) 1. Qual e o seu papel dentro do projecto Força na área de saúde na sua comunidade durante os três últimos anos? 2. Quais são os seus objectivos para a vacinação das crianças e a dar vitamina A? 3. Quantos Dias por semana você esta pronto para administrar vacinações na sua unidade de Saúde? 4. Consegui as suas metas de vacinações? Como se sabe/como monitorar isto? Porque ou porque não? 5. Criancas doentes que chegam ao seu posto de saúde, voce tambem costumar a dar vacinas a eles? Conhece algumas contra-indicações para vacinar as crianças? 6. Quais são os elementos/factores mais importantes na cadeia de frio? 7. Como e a colaboração entre as Brigadas Moveis e o pessoal na unidade sanitária? 8. Algumas vezes tem ruptura de Stok de medicamentos nas Brigadas Moveis? Durante os últimos 12 meses, aconteceu alguma vez que teve ruptura de stok? O que e que você pode fazer para evitar ruptura de Stok? 9. O que mais pode se fazer para aumentar o %/numero de crianças a receberem vacinas a tempo certo e completamente? 10.Quando estas a dar informação sobre imunização/vacinação – quais são os principais assuntos que você queres as pessoas a entender bem? 11.Qual e a percentagem das mulheres grávidas na sua comunidade, que pertencer o grupo alvo do projecto, e que devem receber serviços de saúde pré-natal ou durante o parto ou pós-natal com apoio do pessoal de saúde ou parteiras tradicionais treinadas 12.Conseguiu as metas na sua comunidade? Como e que se sabe isto? O que foi feito pelo o projecto para alcançar esta meta? 13.Existe outros actividades que o projecto podia de fazer para conseguir apoiar mais mulheres grávidas? 14.Quais são os sinais gerais de perigo durante a gravidez que todas as mulheres e a suas familiares devem conhecer? 15.Quero pedir a você mencionar alguns sinais gerais de perigo durante o parto. 16.Quais são os conselhos que você costumar a dar a uma mãe sobre a saúde da sua criança recém-nascido? 17.Se uma mãe esta a chegar na unidade sanitária com uma criança gravemente doente, ou se ela tem problemas graves durante o processo de parto – O que e que você tem que fazer para referir/mandar a criança ou a mãe para o nível mais em cima no sistema de saúde. (Posto para um centro, e um centro para um hospital) Este sistema funciona bem? CS-16, Mozambique, Final Evaluation, The Strength Project Save the Children/US 57 18.Qual são os hábitos de saúde que o projecto Força esta a tentar a mudar na sua comunidade. 19.Quais são as pessoas mais importantes para esta mudança comportamental? 20.Que actividades foram feitas para exercer estas mudanças? 21.Estas actividades – tem algum impacto no comportamento das pessoas na sua comunidade? 22.Exciste algumas outras actividades que podiam ser implementadas pelo projecto no âmbito de mudar o comportamento das pessoas? 23.Quais são os sinais gerais de perigo de diarreia que indica a urgência de mandar a criança ao hospital? 24.Da Vossa opinião – qual e a diferença da saúde da comunidade hoje, e em comparação com o tempo de quatro (4) anos atrás? 25.Baseado na sua experiência – qual será o conselho que você daria a uma outra pessoa de saúde, que esta a fazer o mesmo trabalho como você? 26.Como e que você trabalha junto com Parteiras Tradicionais, Activistas, Lideres comunitários? Em do que maneira isto e útil para você em conseguir o seu trabalho? 27.Se o projecto vai terminar, você será capaz de continuar a trabalhar junto com eles? 28.Há outros assuntos em relação ao Projecto Força que você queres discutir connosco? 29.Tem algumas perguntas?