Tia Mimba, Weka Pesa: Birth Preparedness in Rural Tanzania CARE Community Based Reproductive Health Project Final Evaluation Report Frank Kaharuza Team Leader With the assistance of the Evaluation core team members Ali O. Ali Cosmos Majige Dr. Zacharia Kiatu Hon. Mishamo James C. Setzer Jane Kaiza Juliana Bantambya Michelle Kouletio Sanifa Shoo Victoria Ndalawha Community Based Reproductive Health Project FAO-0500-A-00-6039-00 December 2001 i Table of contents Table of contents................................................................................................................................i Abbreviations and Acronyms.............................................................................................................ii Acknowledgements ..........................................................................................................................iii Executive Summary..........................................................................................................................iv Summary Results Chart ...................................................................................................................vii 1 Introduction ...................................................................................................................................1 2. Technical Approach ..................................................................................................................3 2.1 Project Overview......................................................................................................... 3 2.2 Increased utilization of supervised maternal health services......................................... 6 2.3 Strengthening Emergency Obstetric services..............................................................14 2.4 Increased voluntary utilization of effective family planning methods.........................19 2.5 Increased utilization of STI control and prevention measures .....................................23 2.6 Operational research and other analyses: ..................................................................28 3. Crosscutting Approaches ............................................................................................................. 29 4. Program Management................................................................................................................. 43 5. Conclusions and Recommendations ............................................................................................ 52 6. Lessons Learned ......................................................................................................................... 54 List of Tables Table 1. Phasing-in of different project activities and interventions…………………………………..5 Table 2. Project achievements in improving maternal and newborn care ……………………………6 Table 3. Percent distribution of community education activities by VHW in October-December 2000 in 50 communities (n=109months) .......................................................................................... 10 Table 4. Project achievements on strengthening emergency obstetric services.................................. 14 Table 5. Project achievements in increasing utilization of effective contraceptive methods. ............. 19 Table 6. Family planning counseling quality characteristics from all 66 health units in Project area.21 Table 7. Project achievements in increasing utilization of STI control and intervention measures ... 23 Table 8. Knowledge of main modes of transmission of HIV/AIDS ................................................. 24 Table 9. Respondents Knowledge of STI signs and symptoms occuring in both men and women at baseline and final evaluation.................................................................................................... 24 Table 10. Training activities, persons trained, and number trained by CBRHP and Partners............ 36 Table 11. CBRHP Sustainability goals, objectives, and activities.................................................... 37 Table 12. External Technical Assistance to CBRHP ...................................................................... 50 ii Abbreviations and Acronyms AIDS ........... Acquired Immunodeficiency Syndrome AMREF ........... African Medical Research Foundation ANC ................. Antenatal care CBRHP.............Community-based Reproductive Health Project CCHI ................Care- CDC Health Initiative CDC..................Centers for Disease Control and Prevention CO ....................Clinical Officer CPR ..................Contraceptive Prevalence Rate CSPD................Child Survival, Protection, and Development Program (Government of Tanzania) CYP ..................Couple Years Protection DHMT.............. District Health Management Team DIP.................... Detailed Implementation Plan EmOC............... Emergency Obstetric Care FEMME ........... Foundations for Enhanced Management of Maternal Emergencies FP...................... Family Planning GOT.................. Government of Tanzania HIV................... Human Immunodeficiency Virus IEC....................Information, Education, and Communication IMR ..................Infant Mortality Rate KPC.................. Knowledge, Practice and Coverage MCHA.............. Maternal and Child Health Aide MNC................. Maternal and newborn care MoH ................. Ministry of Health PAC .................. Project Advisory Committee PSI.................... Population Services International PVO.................. Private Voluntary Organizations RH ....................Reproductive Health RHMT ..............Regional Health Management Team STI.................... Sexually Transmitted Infections TBA.................. Traditional Birth Attendants TQM................. Total Quality Management TT..................... Tetanus Toxoid TTBA.............. Trained Traditional Birth Attendant USAID ............ United States Agency for International Development VHW ................ Village Health Workers WHO ............... World Health Organization WRA................. Women of Reproductive Age iii Acknowledgements We would like to gratefully acknowledge the efforts of all those persons who took part in this final evaluation activity. In particular, we want to thank CARE Tanzania and CARE Missungwi for their tremendous support in making such a participatory evaluation process possible. The evaluation is better for this approach and their efforts. Special mention also goes to CARE “family members” from sister projects: Mr. Mushin Siddequey (CARE Bangladesh), Dr. Imelda Tamwesigire (CARE Uganda), and Ms. Doras Chirwa (CARE Zambia) whose views and contributions were very valuable in the whole process. We hope they will carry their experiences back to their projects as well. The contributions of all participants in the evaluation design workshop (PAC members, RHMT, DHMT, CARE Tanzania and CARE-Missungwi senior staff, and other stakeholders and resource persons) were essential to the realization of this final evaluation. We must certainly acknowledge the active support of the Mwanza Regional Health Management Team, the Kwimba and Missungwi District Health Management Teams, who allowed their staff to fully participate in the study either as core team members, supervisors, and interviewers. We acknowledge the role and efforts of the data entry teamfor their work in providing the data in a usable form. We would like to acknowledge all those other collaborators who made major contributions through supporting analysis and compilation of data in the sections of the report. Dr. Imelda Tamwesigire Improving maternal health services Mr. Mushin Siddequey Emergency obstetric care Ms. Doras Chirwa Family planning Ms. Stacy Vogan STI and HIV/AIDS Dr. Bernard Mutayoba STI and HIV/AIDS We would like to acknowledge the support of both Sara Whitehead (CDC) and Tom Schmidt (CDC) who graciously took valuable time from their busy schedules to edit sections of the report. And lastly, a very special thanks to all the community members we spoke with for their patience, time, insights and valuable information that we have tried to faithfully portray in this evaluation report. “Nakushukuru sana” Frank M. Kaharuza 23rd October 2001 iv Executive Summary Programme and objectives: In partnership with the Ministry of Health of Tanzania, the United States Centers for Disease Control and Prevention (CDC) and Columbia University, CARE Tanzania has been implementing an innovative and large-scale reproductive health intervention called the Community Based Reproductive Health Project (CBHRP) in Kwimba and Missungwi Districts, Tanzania since October 1996. The project goal was to reduce reproductive mortality by 50% by the year 2001, specifically to reduce maternal mortality among 150,000 women of reproductive age and reduce neonatal mortality among 30,000 newborns in Kwimba and Missungwi Districts. The objectives of the project were to increase: 1) Utilization of supervised maternal health services; 2) Appropriate utilization of emergency obstetric care services; 3) Voluntary utilization of family planning services and methods; and, 4) Utilization of quality STI services and prevention methods. An estimated fifty percent of project effort/resources were focused on maternal health issues, twenty five percent on family planning, and the remaining twenty five percent on STI prevention. The project design was founded around strategic interventions aimed at simultaneously increasing community capacity to promote reproductive health while improving access to quality reproductive health services. The project focused on community mobilization for the management of health problems and building capacity of the community and other key partners involved in improving maternal and child health in the project area. Main accomplishments and measurable changes Outcomes: Beliefs and behaviors – Over the course of the project, reproductive knowledge and contraceptive behavior among women has improved. Key improvements included: ÿ Approximately 60% of mothers of children 0-23 months in the project area now know two or more antenatal, delivery, and neonatal danger signs. ÿ The proportion of mothers of children 0-23 months who did not want a child in the next two years and reported using a modern method of contraception increased from 11% at baseline to 24%. ÿ The proportion of mothers of children 0-23 months who could recognize at least two signs and symptoms of STDs increased from 11% to 60%. Service utilization – Utilization of maternal health services by women in the project area increased. Key improvements noted by the evaluation team: ÿ The proportion of mothers of children 0-23 months who attended two or more antenatal consultations during their most recent pregnancy increased from 53% to 62%. ÿ Mothers of children 0-23 months who had two or more tetanus toxoid immunizations during their most recent pregnancy also increased from 53% to 60%. ÿ Mothers of children 0-23 months who reported receiving iron and folic acid during their most recent pregnancy increased from 48 to58%. ÿ Over 90% of the mothers of children 0-23 months that used maternal health services during their most recent pregnancy used those services more than once. ÿ The proportion of mothers of children 0-23 months who received postnatal family planning counseling increased from 23% to 37%. ÿ Condom sales by community agents (VHWs and TBAs) per quarter increased from 395 condoms sold in the baseline quarter to 4149 condoms sold in the final quarter. v Delivery services utilization – The utilization of delivery services by women in the project are remaining largely unchanged. Over the course of the project data indicate that: ÿ The proportion of home deliveries remained unchanged. ÿ The proportion of deliveries supervised by trained personnel increased slightly from 52% to 57%. Management of anticipated obstetric complications – There was a large relative improvement in the utilization of services capable of managing complicated deliveries. The overall level of utilization of these services, however, remains low as seen by: ÿ Hospital deliveries for obstetric complications increased from 4% in 1997, to 14% in 2001. Outputs: Community capacity building – The project spent a major amount of its time and resources to build community capacity. Major accomplishments included: ÿ At least two VHWs per village were identified by the community and trained by the project. In all, 299 VHWs were trained to conduct birth planning counseling, postpartum family planning counseling, and STI risk score assessment and counseling. They were also equipped with basic VHW drug kits and a bicycle. ÿ Two hundred twenty three TBAs, two from each village, were trained to recognize pregnancy danger signs and to conduct clean, safe deliveries. ÿ The training of ten master trainers in community empowerment skills included district level extension workers and local health providers. They were trained in community mobilization for emergency transportation and promotion of community supervision of VHWs. This team subsequently mobilized 52 communities for reproductive health improvements, especially improving community transport system for obstetric emergencies. Of these, 11 communities have a functioning transport system in place. ÿ Thirty two communities are involved in a community based perinatal surveillance system, which helps communities to target resources to reduce perinatal mortality. Strengthening health service delivery – Project achievements in strengthening the quality and availability of key reproductive health services include: ÿ Seventy staff drawn from all health facilities received a two-week refresher course on integrated reproductive health for maternal and newborn care, STIs/HIV/AIDS and family planning. ÿ Staff from 58 health units received a course in basic family planning skills. ÿ Ten master trainers were trained in Basic Emergency Obstetric Care management. They subsequently trained 16 health center staff in emergency obstetric care management. Local protocols for management of obstetric emergencies have also been developed to fit the local context. ÿ Four Health centers equipped to offer Basic Emergency Obstetric care in terms of personnel trained, IV fluids in place, vacuum extractors, and resuscitation equipment. Conclusions of this evaluation Key conclusions of the evaluation team include: ÿ The community’s increased levels of knowledge of pregnancy, delivery, and neonatal danger signs, family planning, and STI is high and appears reasonably attributable to project activities. ÿ The process of community empowerment for health action requires significant time and resources (probably more than anticipated by project designers). The project was successful in focusing community attention on health issues and provided a mechanism for collective dialogue and problem solving. ÿ Maternal mortality reduction requires all levels of the entire service delivery and referral system to function effectively and efficiently. The project made targeted/limited vi improvements in the referral system which resulted in limited impact on mortality reduction. ÿ The community information system developed and implemented in 32 villages promotes community data collection and allows community feedback, dialogue and problem solving. This increases the individual and community understanding of priority health conditions, which in turn improves the individual’s capacity to participate in efforts aimed at improving their health. vii Summary Results Chart Objectives and Indicators1 Baseline Target Achieved % of target Outcomes 1. Increased utilization of supervised maternal health services · Proportion of all women WRA who sought at least 2 prenatal consultations 55% 75% 62%2 80% · Proportion of women WRA who had a Card who sought at least 2 prenatal consultations3 90% N/A · Proportion of all women WRA who receive 2 or more Tetanus toxoid vaccines 53% 75% 60% 80% · Proportion WRA who had a card who received 2 or more Tetanus toxoid vaccines 92% N/A · Proportion of deliveries attended by trained attendant (health workers and trained TBAs) 52% 65% 57% 87% · Proportion of women who receive Iron and folic acid (IFA) during pregnancy 48% 75% 58% 77% Outputs 1. 1.1 Increased proportion of pregnant women with a household birth plan · Proportion of women with a household birth plan* N/A 35% 48%4 >100% 1.2 Increased availability of trained VHWs and TBAs at the community level · Number if VHWs and TBAs trained per village 200 VHW, 200 TBA 299 VHW, 223 TBA >100% 1.3 Increased availability of skilled health providers in antenatal care, life saving skills/safe delivery and postpartum care · Number of health care providers skilled in antenatal care, life saving skills/safe delivery and postpartum care at each type of facility 0 1 per facility (66) >1 5 (172 trained) >100% 1.4 Increase availability of postpartum care at community level · Proportion of community-based deliveries that receive postpartum care (PPC) from VHWs/TBAs within first 24 hours* 50% 10%6 20% 1.5 Increased availability of Iron and folate supplements for pregnant women · Proportion of facilities and communities with no stock out of Iron and folate in the past 30 days N/A 50% 87.5% (recent survey) 93% (purposive sample, 2000)7 >100 Outcome 2. Increase the appropriate utilization of emergency obstetric services for maternal and perinatal complications · Met need for obstetric complication management by quarter (hospital-based data only)* 4% (1997) 25% 14% 56% Output 2 2.1 Increased number of villages with emergency transportation plan for obstetric emergencies 1 Indicators marked with and asterix(*) were not measured at the baseline because they were included in the revised DIP . 2 Over 92% of mothers had 2 or more visits by self report. Furthermore, a recent MoH policy requires mothers to leave the cards at the health facility. 3 Over 90% of mothers who had retained their cards had made two or more visits. This implies that mothers who make contact with the helath system services continue to use the service. 4 Birth planning includes: knowing two or more danger signs of pregnancy, preparation of two or more essential supplies for a clean delivery, mobilising household resources for emergencies. 5 There were 172 person trainings carried out. Persons trained selected from each facility. On average there were 2 people trained per dispensary, 4 per health centre, and 10 per hospital. 6 The question was worded “same day’ which may lead to an underestimate of the postpartum check on the same day 7 Not measured because MOH does not have a system for monitoring stock-outs. Periodic surveys from project in 1998, 1999, and 2000 have indicated good supply with less than 15% facilities without supply These 2 surveys represent point assessments of iron/folate stock availability. viii Objectives and Indicators1 Baseline Target Achieved % of target · Number of villages having and implementing transport plan for obstetric emergencies8 * 0 50 52 >100% · Number of villages with functional transport system (used in the last quarter) 0 20 11 55% 2.2 Increased access of the population to basic emergency obstetric care services (EmOC) · Number of health centers providing basic EmOC services 0 4 4 100% 2.3 Increased availability of health information on pregnancy danger signs and referrals · Proportion of mothers who can identify at least 2 Antenatal danger signs of pregnancy 40 58% >100% · Proportion of women able to identify two or more danger signs for mothers during delivery 40 54% >100% · Proportion of women able to identify two or more danger signs for newborn 109 40 56% >100% Outcome 3. Increase voluntary utilization of effective family planning methods · Proportion of women with children under 23 months who do not desire another child within the next two years who are using a modern contraceptive method 11% 20% 24% >100% · Couple years protection*10 611 1222 910 74% Output 3 3.1 Increased access to contraceptive methods available at community level · Proportion increase of condoms sold from baseline quarter*11 395 494 4149 >100% 3.2 Increased awareness of family planning among couples · Proportion of couples receiving postpartum family planning counseling 23% 50% 37% 74% 3.3 Improved basic family planning services available at health facilities by skilled providers · Number of health facilities with skilled providers for basic family planning (at least one provider had received formal training in family planning) 41% 75% all health units 100% >100% Outcome 4. Increased utilization of sexually transmitted infection control and prevention measures. · Percent increase in number of clients receiving treatment for STI*12 6444 (1996) 12888 8932 Output 4. 4.1 Increased availability of information, education, and communication regarding STDs including modes of transmission, signs and symptoms and preventive measures. · Increase percentage of mothers who can recognize at least 2 signs or symptoms of STDs 11%13 20% 60% >100% · Increase population’s knowledge of main modes of transmission of HIV/AIDS and STI 60% 75% 83% >100% 4.2 Increased health facility capacity to provide STD treatment and counseling services · Increased the proportion of health facility staff able to effectively counsel patients on STD N/A 20% 67% >100% · Number of health facilities with skilled providers for syndromic management and confidential counseling 28% 75 % health facilities 80% >100% 8 Page: 8 Initial assessments in each of the 52 communities showed no organized community transport response to assist individuals during health emergencies 9 The baseline survey still considered the “risk factor approach” while current best practices consider the danger signs more than the risk factor approach assessed whether women knew two or more danger signs, either during pregnancy or delivery. 10 The project target was to double the CYP from baseline. Project records, especially community data, has been incomplete. 11 The project target was to increase the condoms sold by 25% above the baseline. 12 The baseline used was obtatined from a baseline survey done by AMREF in the project area 13 Baseline assessed whether women could name at least 1 sign/symptom ix Objectives and Indicators1 Baseline Target Achieved % of target 4.3 Increased availability of protective measures from STDs/HIV at the community level · Proportion of communities that have a condom distribution point with no stock outs in past 30 days* 14 N/A not measured N/A · Number of condoms distributed free of charge (by quarter) 2153 4300 11563 >100% 14 Condom stocks were not measured at baseline. However, 10 towns had received free condom stocks supplied by AMREF. PSI is now established in the two distct and offer a good supply of condoms. 1 1 Introduction This report presents the findings of the final evaluation of the Community Based Reproductive Health Project (CBRHP) carried out in Kwimba and Missungwi districts in Northern Tanzania. The evaluation was conducted in June and July 2001. The evaluation was intended to be a highly participatory exercise and involved key partners and stakeholders in both its design and execution. The original CBRHP DIP and logical framework had been modified following the midterm review, and it was the modified DIP and logical framework that were used as the principal guiding documents for this evaluation. ð Evaluation objectives As proposed by the CBRHP management team and in line with the USAID guidelines for final evaluation of child survival projects, the CBRHP final project evaluation had the following general objectives: ÿ To assess whether the program met the stated goals and objectives; ÿ To determine the effectiveness of the technical approach; ÿ To describe the overarching lessons learned from the project; ÿ To describe/propose a strategy to use, or communicate, these lessons both within the organization and to the partners. ð Evaluation methodology (see Attachment B) Prior to the arrival of the evaluation team, project staff conducted a number of analytical (qualitative and quantitative) activities that were designed to provide the team with important data related to project activities and success. These activities conducted by the project included: ÿ assessment of the quality of STI services at 64 health facilities; ÿ assessment of quality of family planning counseling all 66 health units; and ÿ assessment of the Knowledge Attitude and Practices for 69 VHWs and 56 TBAs from 39 villages in the project area. Participatory planning A three-day participatory planning meeting was organized early in June to review the evaluation process. Participants considered the information needs for each of the project goals and indicators and the best approaches to data collection. The project log frame was consulted. The meeting also reviewed the baseline KPC questionnaire and suggested modification to the questionnaire. Questions were included in the KPC to capture information as required in the revised DIP and log-frame. Participants also translated the new questions from English to Swahili. Knowledge Practice and Coverage survey (KPC) The evaluation was substantially based on a KPC survey. The sampling strategy used for the quantitative survey was based on the WHO 30 cluster, 10 respondents per cluster sampling (WHO, 1988) which was used during the baseline survey. All villages selected during the 1997 baseline survey were selected for the final KPC survey. In addition, villages were selected according to timing of the community intervention. Data were collected from a random sample of 30 villages where community interventions were started early in the project, 20 villages with late implementation and 10 villages where no community project activities had been introduced. A random sample of eight extra villages where the maternal and perinatal surveillance has been established was also included. In each village, ten mothers with children less than 24 months age were randomly selected and interviewed. Two mothers from each village who reported an adverse pregnancy outcome (abortion, stillbirth, or child death) were purposively sampled and interviewed by the supervisors. The evaluation planning team specifically requested this information. This information was not collected during the 1997 survey and was not used in the comparative analysis presented in this main report. 2 The final KPC evaluation report presents findings comparing the baseline findings to the final evaluation and considers data stratified into early, late, and no community intervention. Thirty participants (20 interviewers, 5 supervisors and 5 CARE field support staff) were involved in a three-day training workshop that covered the sampling technique, interviewing techniques and pre￾testing of the questionnaire. In all, four field teams visited 68 villages over a 10-day period. Community Qualitative Study Primary data were also collected during the evaluation through a qualitative study using large group meetings, focus groups and key informant interviews. Nineteen (19) participatory group sessions were held with gender-disaggregated groups from 10 communities in the project area over five days. Participatory village mapping was used to discuss factors affecting access to reproductive health services. Disease ranking and matrix methods were also used to initiate discussion on common reproductive health conditions, factors promoting and barriers to use of reproductive health services. Discussions on project strengths, weaknesses, and recommendations, were carried out with the DHMT, Project staff, TBAs, and VHWs. In addition, all ward executive committees were asked to submit their opinions about the project and project activities in their wards. A one-day plenary session was held with them to discuss their findings. Key informant interviews were held with RHMT members, DHMT members, and key project partners namely AMREF, The Roman Catholic Archdiocese, and Bugando Medical Centre, that is the main referral hospital for Mwanza region. Data management and analysis Survey - All the KPC data were entered in the field by data entry personnel using EpiInfo 6.04b. Open-ended questions were coded prior to data entry. Data cleaning was carried out and analysis conducted using EpiInfo 6.04b. Qualitative - Every evening, community study teams reviewed the qualitative data collected, translated the notes from Swahili to English, and clustered the information into the broad four main intervention area themes. Based on the these thematic areas, core team members created four data analysis and writing teams who assembled and organized the data from all data sources. The themes focused on project achievements, reasons for achievement, constraints to achievement and lessons learned. 3 2. Technical Approach 2.1 Project Overview Project location At the inception of the project, Kwimba District was the second largest district in Mwanza Region in northwestern Tanzania. In 1997 Kwimba was divided into two smaller districts, Kwimba and Missungwi. The project area is currently estimated to have 600,000 inhabitants, in 9 divisions, 45 wards, and 188 villages. (See Attachment E) Rationale for project location At the time of project design regional planners and community members indicated that Kwimba was the most underserved district in the Mwanza region. Available reproductive health indicators for the district were poor with high maternal mortality, high infant mortality, low contraceptive use and high levels of STI infection. Most mothers delivered at home assisted by untrained birth attendants. Access to health services was deemed inadequate and health facilities in the area did not deliver quality services. Experience from the UNICEF Child Survival Protection and Development Program (CSPD) in two divisions of Kwimba had demonstrated the potential for a high level of community participation in program activities designed to improve maternal and child health. Project objectives and interventions The project goal was to reduce maternal mortality among 150,000 women of reproductive age and neonatal mortality among 30,000 newborns in Kwimba and Missungwi Districts by 50% by the year 2001. Objectives, key interventions, and level of project effort were: Objective 1. Maternal Health and Newborn Care: (50% Emphasis) ÿ Increase access and utilization of appropriate ANC services, i.e., TT2 immunization, malaria control, and micronutrient supplementation by 80% over the baseline. ÿ Increase appropriate identification and management of high-risk pregnancies by 80% over the baseline. ÿ Increase emergency obstetric services, i.e. transport and care, by 80% over the baseline. Objective 2. Family Planning: (25% Emphasis) ÿ Increase use of modern contraceptive methods among WRA and their partners by 40% over the baseline. ÿ Increase knowledge of contraceptive methods among WRA and their partners by 70% over the baseline. Objective 3. HIV/AIDS/STDs: (25% Emphasis) ÿ Increase the knowledge of the modes of HIV transmission, particularly maternal-infant transmission, and means of HIV/AIDS prevention, particularly correct condom use among WRA and their partners by 50% over the baseline. ÿ Increase the detection, referral, and treatment of STDs by health providers for WRA and their partners by 40% over the baseline (CBRHP Project proposal). Maternal and newborn care At the community level, VHWs were to track all pregnancies in their villages and account for pregnancy outcomes. VHWs were also trained to carry out antenatal birth planning counseling with the families, newborn care, and postnatal family planning counseling. Village leaders were sensitized 4 on community transport planning, maternal danger signs, birth planning, family planning and VHW supportive supervision. At the health services level, the main emphasis was on improving the quality of services provided through training of health facility staff in maternal and newborn care and equipping of the health centers so they can offer basic emergency obstetric care (see crosscutting approaches: strengthening health worker performance). Other CBRHP initiated approaches to improving maternal and newborn care included: The Maternal and Perinatal Health Care Surveillance (MPHCSS) is a community and health facility driven pregnancy tracking system aimed at identifying birth-weight specific mortality concentrations/clusters in order to target interventions and address the principal causes of elevated mortality. Pregnancies are registered by the VHWs and outcomes recorded and placed on a community bulletin board. The community health committee reviews the board regularly to review trends, identify problems, and jointly propose actions with the local health facility staff. The surveillance system is functional in 32 communities in both districts and is funded under the CARE-CDC Health Initiative. Local working group for emergency obstetric care: Due to the lack of protocols on what is supposed to happen at each level of the health care system from the community to the district and referral hospitals, a team composed of local hospital managers, referral hospital obstetricians, and district and regional health management members worked with CBRHP to develop local guidelines for the definition and case management of five major maternal and newborn complications. The guidelines have been incorporated in the subsequent development of national level protocols and training curriculum for emergency obstetric care. Working group members are currently involved in training dispensary, health center, and hospital staff in EOC, and monitoring use of protocols. Funded under CARE-CDC Health Initiative/Columbia University grants. Food security and women’s nutrition promotion: CBRHP operates in an area of chronic food insecurity. Recognizing the impact of poor food production and intrahousehold inequities in resource distribution, CBRHP partnered with the Kwimba District Council’s Agriculture and Livestock Development Department to promote improved agriculture, nutrition, and gender in two divisions in the project area (KAFEM). This component is integrated with CBRHP activities and promoted particularly during birth planning and postpartum visits. Funded by the Canadian International Development Agency (CIDA). Family planning At the community level, information on modern contraception options is integrated into birth planning and postpartum home visits. CBRHP developed a VHW manual that included all important family planning information. Furthermore, VHWs promote and sell condoms in the community. At the health facility level, strengthening dispensary level provider skills in basic family planning counseling and health education was carried out. CBRHP trained at least one health worker from every health unit in the project area. HIV/AIDS/STI At the community level, STI risk score assessment was integrated into birth planning and carried out by VHWs. In addition, VHWs have also held community dialogues on risk perception, risk mapping, and have carried out social marketing of condoms in high transmission areas. CBRHP is creating linkages with traditional healers to manage STI/HIV/AIDS. At the health facility level, all hospitals and health centers carry out antenatal syphilis screening. CBRHP and AMREF trained at least one provider from each health unit in STI case management including confidentiality, syndromic management, and partner notification. 5 Clearly, CBRHP’s overall approach in project implementation emphasized capacity building and institutional strengthening both at health facilities and in communities. Health service delivery was strengthened by equipping health centres with basic emergency obstetric care equipment and through staff training in antenatal care, lifesaving skills, family planning counselling, STI counselling and management. Communities were also mobilized to develop collective mechanisms to address identified health problems and to support VHWs and TBAs (see cross cutting approaches: community mobilization). ˛ Phasing-in approach The original project implementation plan called for the project to implement activities in the entire project area at a single point in time. This was not feasible and the project devised a phase-in approach (see Table 1 below). The project also aggressively sought for counterpart funding for some of the activities (see Project management: Planning). Project activities following the midterm evaluation strategically used the lessons learned from the communities where the interventions were begun to develop a better program in subsequent villages. To date only two divisions have not started any community interventions yet. Table 1. Phasing-in of different project activities and interventions Division Health Facility Intervention Phase I (community intervention initiated in 1997) Phase II (Community intervention after Jan 1999) MPHCSS (CCHI) Community Empowerment (CCHI) (selected villages) KAFEM (Maternal Nutrition Promotion) Kwimba District Ibindo X X X X Mwamashimba X X X Ngudu X X X Ngula X Nyamilama X X X X Missungwi District Inonelwa X Mbarika X X X X Missungwi X X X Usagara X X X 6 2.2 Increased utilization of supervised maternal health services Fifty percent of the project effort was directed toward improving maternal and newborn care in the project area. Table 2 shows the project achievements in improving maternal and newborn care. Table 2. Project achievements in improving maternal and newborn care Outcome Indicators Baseline Target Achieved · Proportion of all WRA who sought at least 2 prenatal consultations 55% 75% 62% · Proportion of WRA who had a card who sought at least 2 prenatal consultations 90% · Proportion of all WRA who receive 2 or more Tetanus toxoid vaccines 53% 75% 60% · Proportion of WRA who had a card who received 2 or more Tetanus toxoid vaccines 92% · Proportion of deliveries attended by a trained attendant (health workers and trained TBAs) 52% 65% 57% Increased utilization of supervised maternal health services · Proportion of women who receive Iron and Folic Acid (IFA) during pregnancy 48% 75% 58% Outputs Indicator Baseline Target Achieved 1.1 Increased proportion of pregnant women with a household birth plan · Proportion of households with pregnant women that have functional birth plans N/A 35% 48% 1.2 Increased availability of trained VHWs and TBAs at the community level · Number of villages with trained TBA assisting 3 or more deliveries in past 6 months · Number of villages with at least one VHW tracking at least 10 pregnant women at any given time N/A 200 VHW 200 TBA 299 VHW 223 TBA · Number of health facilities with skilled providers 1.3 Increased availability of 66 66 skilled health providers in antenatal care, life saving skills/safe delivery and postpartum care · Number of skilled providers at each type of facility 0 1 per facility (66) >1 (172 trained) 1.4 Increased availability of postpartum care at community level · Proportion of community-based deliveries receiving postpartum care (PPC) from VHWs/TBAs within first 24 hours N/A 50% 10% 1.5 Increased availability of Iron and Folate supplements for pregnant women · Proportion of facilities and communities with no stock out of Iron and folate in the past 30 days N/A 50% > 87.5% CBRHP’s strategy to increase utilization of maternal health services was carried out through three main approaches: ÿ Training and support to VHWs to do birth planning counselling; ÿ Training of TBAs - to recognize danger signs and to refer women; and ÿ Improving the quality of reproductive health services at health facilities. Achievements by indicator: ð Outcome Indicator: Proportion of all pregnant women who sought at least two prenatal consultations increase from 55 to 75%. In 2001, 62% of all mothers had attended ANC two or more times, which would make it appear that the target of 75% was not met. There are, however, several mitigating factors challenging interpretation of this result. Measurement of this indicator presupposes that mothers are able to keep their antenatal cards or notebooks and present them at the time of the survey. This assumption may lead to an underestimate of the antenatal attendance because mothers may lose their ANC card (28% of mothers interviewed had reportedly lost their cards at the time of the interview). Furthermore, a new MoH policy requires/suggests maternal health cards be kept at health facilities at delivery, which 7 means that recently delivered mothers would not have been in possession of their ANC card at the time of the household interview. Clearly the 62% rate estimated by the survey is the most conservative estimate possible and the true rate almost certainly higher than that. Of the mothers who had a maternal health card, 90% (184/204) had made two or more ANC visits. This suggests that mothers who contact the services continue to use them. Based on data from the maternal health cards, 50% of the mothers first attended ANC in the second trimester of pregnancy. The median age of gestation for first booking was 6 months suggesting that mothers in this area start attending antenatal services relatively late. Results from the qualitative data suggest that mothers attend late because of the distance to the health unit, cost incurred at the health units, and cultural beliefs. Women book at the gestation of 5 months and above because below that pregnancy is still in "liquid form" and not seen. (FGD/Women group Igongwa) Women are booked at 5-6 months though they are advised to go as early as 3-4 months; this is related to culture, taboos, and feel tired of attending ANC several times. (FGD Men Wazamiso) Review of MPHCSS data from two divisions shows that approximately 90% (825/932) of mothers attended antenatal at least once, and 67% (627/932) of the mothers had made two or more antenatal visits. ð Outcome Indicator: Proportion of pregnant women who receive 2 or more tetanus toxoid vaccinations increased from 53 to 73% Although KPC data show an improvement in TT immunization from 53% at baseline to 60%, the project target was not achieved. The measurement of this indicator also assumes that mothers without a TT card are not immunized. This may result in an underestimation of the true rate of coverage for reasons similar to those advanced for the lack of maternal health cards. Of the mothers with TT cards, (92% 179/194) received two or more doses. This suggests continued and appropriate use of health services – at least for women who are also careful about keeping their cards. Card retention was similar in the community intervention and non-intervention areas. Only 38% of mothers said TT immunization protects mother and baby from tetanus. A stratified analysis of the community survey results, based on the time of introducing the community intervention, indicated that 62% of respondents in the early intervention communities knew the correct reasons for TT immunization, compared to 31% in the non-intervention communities. This increased level of awareness strongly suggests that the community education activities of the VHWs and TBAs may have a role to play in increasing the level of knowledge among the mothers in the project area. It also suggests that this sort of knowledge transfer takes time to be effective. ð Outcome Indicator: Increase the proportion of deliveries attended by trained personnel from 52% to 65%. This project target was largely not met. The evaluation team found that the proportion of deliveries attended by trained personnel increased from 52% to 57%.15 Although the 1997 baseline did not differentiate between trained and untrained TBAs, TBA assisted deliveries increased from 3% in 1997 to 9% in 2001 with the majority of deliveries carried out by trained TBAs. Some TBA assisted deliveries took place at health units either during TBA training or under the supervision of trained health providers. The proportion of home deliveries remained largely unchanged (1997 baseline 48%, 2001 evaluation 47%). Approximately 5% of home deliveries were assisted by a trained health 15 The CBRHP definition of a trained attendant at birth includes trained TBAs. 8 provider16, while 4% of mothers delivered on the way to health facilities. Figure 1 shows the different personnel who assisted mothers at delivery. Figure 1: Who assisted most recent delivery 0 10 20 30 40 50 60 Health worker Relative Mother TTBA TBA Baseline Final Evaluation There were geographical differences in the proportion of pregnancies that went for health facility delivery. Approximately 73% of deliveries in Ngudu division occurred at health facilities while only 28% of deliveries in Mbarika Division occurred at health facilities. A similar pattern of health facility delivery in Mbarika division was observed from the MPHCSS data. Only 34% of mothers in Mbarika division were delivered at a health facility. Mbarika division is a poorly served area with most health units found along the main roads, which may be the reason for mothers opting for home delivery. Of the women who planned to deliver at a health facility, only 60% did so (the remainder delivered at home). Of those planning to deliver at home, 89% did so. Almost all the women who delivered at a health facility (95%) had planned to do so. This suggests a low number of transfers to a health facility during delivery. A limitation of the KPC survey was that mothers who delivered where they had not planned to deliver were not asked why they changed their mind. ð Outcome Indicator: Proportion of pregnant women who receive iron folate during pregnancy increase from 48 to 75% Although CBRHP did not have direct control over supply and distribution of iron and folic acid to pregnant women , the initial project plan was to monitor changes in iron and folic acid coverage among pregnant. Mothers reporting receiving iron and folic acid (IFA) increased from 48% to 61%. Of 19 mothers pregnant at the time of the survey, 59% (11) had received iron and folic acid. Despite this apparent increase in coverage rates, the project target was not met. A recent study by the CBRHP showed that a number of issues influenced the distribution of IFA to pregnant women. These included distribution of IFA at the health unit, instruction to and counselling of the mother by the health provider, and women’s attitude about taking IFA. Most health units had adequate stock of IFA. Health staff gave little time to advise and counsel ANC mothers on iron supplementation, assumed that women would not take iron folate even if given to them, and therefore often did not offer it. In order to improve IFA distribution, VHWs and TBAs were given IFA to sell or distribute in their villages. The effect of this recent project innovation was not able to be determined by the evaluation team due to its very recent initiation. 16 The baseline KPC did not give this information for comparison. However anecdotal information suggests that some health provider assisted home delivery for a high fee 9 Results related to Outputs ð Output 1.1: Increased proportion of pregnant women with a household birth plan Output indicator: Proportion of households with pregnant women that have functional birth plans increased to 35% No baseline data exist for this indicator. A birth plan was defined by the project to include planning the place of delivery and emergency transport, preparing clean delivery items, and awareness of danger signs during delivery. According to the birth planning guidelines, VHWs are required to conduct three home visits and each visit should cover a particular component of the birth plan. VHWs, community leaders, and health workers also emphasized birth planning at community health education sessions and antenatal health sessions on birth planning. Virtually all of the women interviewed in the KPC survey reported that they had a plan for where to deliver, so this element was excluded from the working definition of a birth plan. The respondents interviewed were able to mention that money (57%), gloves (47%), razor blades (44%), and mother and baby clothing (28% & 23% respectively) were important items in birth planning. Only 7% of the mothers did not mention any items for preparation. Overall, 48% of women surveyed had prepared at least two items for delivery (not including clothing), knew at least two dangers signs, and had saved some money. “Tia mimba weka pesa”(make her pregnant, prepare money.) Prepare up to 100,000/= and if not available they sell cows. Government leaders help them on security bond. (FGD/men, Mwabachuma village) Using the baseline-final comparison villages, the KPC survey data also showed that 57 mothers (19%) had been visited at least once by a VHW. Of these, 39(68%) had an adequate birth plan. Only 39% of the mothers who had not been visited by a VHW had an adequate health plan.17 Analysis of the MPHCSS data also showed that 28% of mothers had the recommended three or more birth planning visits from VHWs. This suggests that VHW visits had a positive impact on the birth planning practices of the mothers in the project area. ð Output 1.2: Increased availability of trained traditional birth attendants and Village health workers at the community level VHW training in Tanzania initially focused just on environmental sanitation and immunization. The CBRHP project, in collaboration with the MoH, revised the VHW training curriculum to include maternal and newborn care. Communities were mobilized to select two VHWs and two TBAs for each village. To date, there are 299 VHWs and 214 TBAs who have been trained. However, some of the villages are so big that two VHWs could not adequately cover them. The project staff noted that, since villages are so large (often consisting of five or more sub-villages) it was very difficult for just two VHWs to cover their assigned area. Similarly, community members observed that in some villages, even the recruited VHWs who lived in the same sub-village could not cover the whole sub￾village. In the MPHCSS surveillance villages, more VHWs were recruited in each village to help effectively monitor mothers in that area. The evaluation team also noted that approximately 61% of the trained VHWs were active (defined as regularly reporting community activities). Although CBRHP monitoring data for October to December 2000 were incomplete, Table 3 shows a large variation in reported VHW activity. Education on maternal danger signs was the most common community activity carried out. On average, VHWs tracked seven pregnant mothers per month. 17 In the Before-After comparisons, 94% of the mother from the non-project area were not visited by a trained VHW. Using all 680 questionnaires, data from intervention areas show that 28% were seen by a VHW and 65% of them had an adequate health plan. 10 Table 3. Percent distribution of community education activities by VHW in October-December 2000 in 50 communities (n=109 months) Community education activities Persons seen per month (results in %) 0-10 11-50 51+ Median (Min, Max) Maternal danger signs 29% 42% 28% 21 (0, 450) Postpartum care 35% 39% 27% 20 (20, 451) STI 42% 30% 28% 20 (0, 670) Family Planning 41% 35% 24% 15 (0, 450) Pregnancies Tracked (number) 60% 31% 9% 7 (0, 287) Trained TBA assisted deliveries increased from 2% at the baseline to 8% in the final evaluation. This suggests a small but increasing role of TBAs in delivery and a possible reduction in the number of deliveries assisted by untrained relatives. ð Output 1.3: Proportion of health facilities (58) with at least one skilled provider in antenatal care, lifesaving skills safe delivery and postpartum care The project target was achieved through the integrated reproductive health training offered to 70 health workers. The CBRHP trained at least one health worker from each of the 66 health units in the project area. The staff cadre selected for training was mainly MCH Aides who are not commonly transferred from health units and are the ones providing maternal and newborn care. A recent health facility assessment showed that all health facilities had at least one staff trained in maternal and newborn care. In addition to the basic training, sixteen health centre staff, four from each health centre, were trained in basic emergency obstetric care. ð Output 1.4: Increased the availability of postpartum care at community level to 50%. Output indicator: Proportion of community-based deliveries receive postpartum care from VHW/TBAs within the first 24 hours The final KPC survey indicates that only 10% of mothers who delivered at home were visited by a VHW within 24 hours of delivery. Therefore, this project objective was not met.18 However, the KPC survey shows that 51% of recent mothers reported receiving some type of postnatal care. Postnatal carried out by health facility staff focused on child weighing and immunization. Of the 153 mothers who reported receiving postnatal care, 25% said nothing was done to them, 24% had an abdominal exam, 20% were examined for bleeding and vaginal tears, 14% reported other examinations. The level of knowledge about postpartum and neonatal danger signs is high. More than half of mothers interviewed were able to mention two or more danger signs in the postpartum period (58% of mothers) and neonatal period (56% of mothers). The neonatal danger signs mentioned included fever (72%), poor feeding (39%), and difficulty in breathing (22.4%). Maternal postpartum danger signs included: bleeding (83%), fever (22%), and retained products of pregnancy (20%). Overall, the project set an ambitious target that could not be attained by the VHW, given the amount of work and geographical coverage. ð Output 1.5: Increased availability of Iron and folate supplements for pregnant women. Output indicator: The proportion of health facilities and communities with no stock out of Iron and folic acid in the last 30 days 18 The Final KPC wording of the question on “time of the visit” was ambigious. in that “same day ” was not adequately translated to mean 24 hours and therefore one would misclassify a mother who delivered in the evening and was seen in the morning of the next day as not been seen within 24 hours. However, only 28% of women who delivered at home had postnatal visits within a few days after delivery 11 This indicator was not measured by the final evaluation team. The MoH, which is responsible for the IFA supply and its distribution, does not track IFA distribution. The project also did not put in place a mechanism to monitor IFA supply and stock outs. However, periodic surveys done in 1998, 1999, and 2001 by the project indicate a good supply of IFA in the health units with less than 15% of the facilities without supply. Other maternal and newborn care indicators Information on clean delivery practices were not collected at the baseline but were key messages from the VHW and TBAs. The final KPC survey data show that 60% of deliveries occurred on an appropriate delivery surface (mackintosh/polythene/ mattress). Gloves were used in 60% of the deliveries, 60% of cords were cut with a new razorblade, 75% of cords were tied using appropriate cord ligatures, and 61% (67/109) of the attendants at bare-hand deliveries washed their hands with soap and water. This suggests a good community uptake of messages on clean delivery practices. Bare hand deliveries occurred in home deliveries and on the way to health units.Unfortunately three￾quarters of home deliveries (101/139) were bare hand deliveries. Issues/factors supporting the achievements Close collaboration among key actors in maternal health service delivery was a key determinant of project success at all levels: health facilities, the community and partner agencies. At the health facility level, VHWs and TBAs were allowed to conduct maternal health education sessions at health facilities. Health facility staff recognized and acknowledged the roles of VHWs and TBAs in referral of mothers to the heath units. Health facility staff also trained TBAs in safe delivery at the health facilities. Health centre staff carried out support supervision visits to TBAs resident in their catchment areas. This was done in collaboration with CBRHP field officers. At community level, VHWs in some communities enjoyed the support of local leaders and were given audience at village meetings and gave birth planning, family planning, and STI education talks. Together with partners (MoH, UNICEF), a concerted effort was made to educate the community on danger signs and prompt management of emergencies through songs (e.g., the TBA danger sign song), and posters. MoH IEC poster materials were used. The VHW manual developed by CBRHP elaborates all the key maternal health messages and is an effective tool for training VHWs to be effective health educators in the community. CBRHP also developed a maternal health card, which was regarded more as an educational tool for danger signs by the community than a pregnancy￾monitoring tool. Anecdotal information shows that this “three coloured” card was used as an educational tool. Training of health workers and equipping health units has apparently improved the quality of services offered at some health facilities. Community members noted the improvement in services rendered at some health facilities. The capacity of the four health centres to provide basic EmOC was strengthened. TQM methodologies were introduced in eight facilities and helped to identify problems and consider appropriate solutions to improve reproductive health services in the area. Constraints Prevalent beliefs and values limit the use of delivery services. Though complications of delivery were adequately identified, understanding of their causes and management of emergencies were not appropriate and this affected the community management of these complications. For example, “Luwikilo” is believed to be a result of having intercourse with another man besides one’s husband. Complications like obstructed labour, retained placenta, and eclampsia are poorly understood and related to “Luwikilo”. [FGD women-Luhala] Mothers do not announce onset of labour - they believe that if they talk, the labour pains will disappear. [FGD men- Mbarika] 12 A pregnant woman should not cross a river as she may abort – this may be a factor that affects use of services in an area with rivers during dry season. [FGD Men- Mwadubi] CBRHP developed some communication strategies to address these beliefs and, as demonstrated in some case studies, these are gradually being overcome. Regarding health seeking behaviour, the first provider of health care actually used in most areas is the traditional healer, who sometimes retain mothers and delay access to modern services. However, most TBAs trained are also traditional healers that handle these complications. CBRHP also worked carefully to discuss danger signs and their causes with prominent traditional healers to influence prompt referrals. Unfortunately, a major limitation to this approach is the highly mobile nature of traditional healers. It is a reality of the current economic situation in Tanzania that there are high informal charges at health facilities. Although maternal and child health services are supposed to be free, informal charges are common and are a deterrent to care seeking. Furthermore, the formal charges at paying hospitals were said to be out of reach for some members of the community. The quality of services was quite poor at some health facilities. Some of the health facilities were poorly equipped, lacked privacy, were severely understaffed, and staff lacked skills and/or motivation. Some of the factors were beyond the scope of the project. The project was able to train some health workers and but could not equip or improve the infrastructure of the health units. Health facilities are geographically inaccessible. The roads are poor, especially in the rainy season and villages are remote from even very basic care. Again, remedies for this problem were beyond the scope of the project. Referral to the tertiary facilities remains a challenge. VHWs refer to the nearest health facilities, but this level often finds it difficult to refer to next level, due to lack of communication and transport. Male involvement in reproductive health is low. CBHRP targeted pregnant mothers and expected husbands to attend birth-planning discussions. VHWs reported this rarely occurred. In 1999, a study carried out by CBHRP on gender issues in reproductive health came to the same conclusion (Kaiza, et al 1999). Village health workers did not sustain their activities in some of the areas. This is an ongoing and unresolved concern. The village health workers are now not working. Previously they were doing a very good job of visiting pregnant mothers, postnatal mothers and were giving advice and encouraging them to attend the clinic services. … They were able to recognize the number of pregnant mothers and those with complications in the village. (FGD, Women group Mwasonge) Lessons learned and recommendations Some cultural taboos and social norms are harmful to women’s survival. More education and work with VHWs, community leaders and other community resource persons is required to sensitize community members and to dispel harmful beliefs and attitudes. Set appropriate and feasible project targets. Two output targets set were not feasible given that the project was not directly controlling the whole process. Male involvement in maternal health is an important but neglected area. There is need for strategies that mobilize and involve men in all reproductive health activities. Men asked for more information and discussion of reproductive health issues. 13 Traditional healers are important and widely utilized community reproductive health providers: Communities should be sensitized about what conditions can be managed by traditional healers. CBHRP needs to specifically target traditional healers with information about the best practices in reproductive health. This would reduce delays in seeking health care in reproductive health emergencies. VHWs need to be able to identify appropriate referral health units. The VHWs refer to the nearest health facilities where there is no transport and this increases the delay in receiving services. VHW distribution and coverage is inadequate. Communities and projects should develop better strategies of selection, recruitment, expected coverage, and motivation of VHWs to improve contact with mothers in all the sub-villages. CARE should engage national level policy makers in dialogue around this issue based upon their experiences. 14 2.3 Strengthening Emergency Obstetric services Project Objective 2: To increase appropriate utilization of emergency obstetric services for maternal and perinatal complications Project activities aimed at improving access to quality emergency obstetric care (EmOC) services focused on working with villages to improve emergency transportation planning, upgrading health centers to provide basic Emergency Obstetric Care (EmOC) services, and community based education about danger signs during pregnancy, delivery, and in the newborn. Table 4 presents project achievements and shows that CBRHP achieved considerable success in improving the capacities of health facilities to manage EmOC and increased community awareness of maternal and neonatal complications. The table also includes project achievements for two additional outputs included after the Mid-term review. Table 4. Project achievements on strengthening emergency obstetric services Outcome Indicator Baseline Target Achieved Increase the appropriate utilization of emergency obstetric services for maternal and perinatal complications Met need for obstetric complications (number complications / number of expected complications (15% of all pregnancies) 4% 25% 14% Outputs Indicators Baseline Target Achieved 2.1 Increased number of villages with emergency transportation plan for emergency obstetrics Number of villages implementing transport plan Number of villages with functional transport system 0 0 50 20 52 11 2.2 Increased access of the population to basic emergency obstetric services Number of health centers providing basic EOC services 0 4 4 2.3 Increased availability of health information on pregnancy danger signs and referrals Proportion of adults with children under 24 months able to identify 2 or more pregnancy related danger signs: Pregnancy Delivery Newborn 10% 40% 58% 54% 56% 2.4 Increased access of the population to comprehensive emergency obstetric services* Number of hospitals providing comprehensive emergency obstetric services 0 4 4 2.5 Improved community based surveillance of maternal and perinatal outcomes Number of communities with functional MPHCSS 0 32 32 *Output 2.4 and 2.5 were additional outputs included after the midterm evaluation Achievements by indicator: ð Outcome indicator: Met need for obstetric complications Met need for obstetric complications is defined as the proportion of expected obstetric complications managed at a health facility able to offer EmOC services. It is generally estimated that 15% of the all pregnancies will be complicated and therefore require such services. Although some data are available at health centers that have now been upgraded to offer EmOC, health units still refer obstetric complications to the hospital. Only hospital data were used to estimate the met need to avoid double counting of obstetric complications that may have been recorded at the health centers and the hospitals. The baseline data used were obtained from the hospital assessment survey carried out by the project in 1999. Overall, met need for obstetric complications through the hospitals improved from 4% in 1999 to 14% 2001. Although this was half of the expected target, this suggests an improvements in diagnosis and management of emergency, which may be at least partially attributed to training 15 of health facility staff, and equipping health facility to offer EmOC. It may also represent improvements in the ability of patients to actually go to referral facilities when need be. ð Output 2.1: Functional emergency transportation systems in 50 communities Figure 1: Sumbugu village opted for a boat as an emergency transport system The project worked to mobilize communities establish an emergency transportation system in 50 villages. Because of demand from communities, 52 villages were mobilized by the end of the project. To date, 54% of these communities have up-to-date action plans and over 20% of the communities have reported using them in the in the last quarter. This is both impressive and encouraging. Communities themselves, with CBRHP support, identified the different transport options, which included tri-cycles, ox-cart, boats, and emergency medical funds. Data from the qualitative study suggest that in villages where community mobilization work has taken place, emergency transportation is considered a shared, community responsibility when it is needed. A community plan of action was available and used. This was different in non-mobilized areas where decision-making was still left to the individual family members, which often results in critical delays in seeking urgent attention. ð Output 2.2: Basic EmOC services functional at 4 health centers EmOC training of staff and upgrading of facilities brought all 4 health centers in the project area to a level capable of providing basic EmOC services (capacity to administer antibiotics for infection; administer anti-convulsions to treat pre-eclampsia; administer oxytocics; perform manual removal of retained placenta). Ten district level staff were trained as trainers who would then train others in EmOC case management. An EmOC protocol was developed to be used in training other health facility staff. These efforts have been complemented by one partner, UNICEF, who supplied delivery beds to health units in Kwimba. ð Output 2.3: Increased availability of health information on pregnancy danger signs and referrals Indicator: Proportion of adults with children under 24 months able to identify two or more pregnancy related danger signs 16 The baseline survey used the risk factor approach to obstetric management. In 1998, CBRHP reviewed this approach and is using the current approach which focuses on danger signs during pregnancy. Bleeding in pregnancy was mentioned as a risk factor in the baseline and as a danger sign in the final evaluation. The proportion if mothers who mentioned bleeding in pregnancy as risk factor rose from 19% (baseline, 1997) to 62% (Final KPC, 2001). Almost all respondents recognized that mothers with danger signs should be managed in health facilities. The final KPC showed that 59% of the mothers could spontaneously mention two or more danger signs for the mothers and the newborns. Approximately 58% of women in the 2001 KPC could identify at least 2 danger signs during pregnancy. Those most commonly identified were bleeding, fever, and swelling/edema. Furthermore, 54% of women could identify at least 2 danger signs during delivery, including bleeding, fever, retained placenta, and convulsions and 56% could identify at least two danger signs in a newborn, including fever, poor feeding, breathing problems, not crying, and convulsions. This suggests that bleeding and fever are overt complications that mothers and communities identify, and may attract prompt action from family members or attendants at delivery. ð Output 2.4: Comprehensive EmOC services functional at 4 hospitals From the outset, CBRHP assumed that the 4 hospitals in the districts had adequate skilled personnel and equipment to provide comprehensive EmOC. It became apparent that this was not the case, and that the level of need for both skills training and facility/equipment upgrading was well beyond the means and mandate of CBRHP. During the last quarter of 2000 CARE through CBRHP launched the Foundations for Enhanced Management of Maternal Emergencies (FEMME) project for the four hospitals in the project area. A detailed needs assessment has been completed, training curriculum developed, and the first training completed. In preparation now are individual hospital feedback and action planning, recruitment of staff, development of memoranda of understanding, and equipping of the hospitals. FEMME will continue to be implemented over the next 2 years. ð Output 2.5: Maternal and perinatal health care surveillance system (MPHCSS) managed by 32 communities Figure 2. Community monitoring board at Lyoma health centre 17 As one of the elements of the CARE-CDC Health Initiative partnership, a maternal and perinatal health care surveillance system (MPHCSS) was established in 32 villages to monitor both maternal and perinatal/infant outcomes. Village health workers were trained to collect data during health education visits to pregnant and postpartum women. Information on socio￾demographics, medical history, delivery characteristics, and pregnancy outcomes were collected. Maternal and fetal/infant survival or deaths were tracked on a community monitoring board, organized by birth-weight and age at death, and grouped according to underlying causes and associated potential interventions. During the first phase (January - May 2000) a review of the MPHCSS was done. The review showed that 904 pregnant women were followed through delivery, with 828 (91%) followed at least to 42 days postpartum. Implications/interpretation of the surveillance data were discussed at village meetings and with health center staff to assist in targeting interventions and monitoring changes. Initial assessment and discussions with partners show that the MPHCSS surveillance is a successful system for focusing community attention on reproductive health problems. It has generated community discussion on maternal and neonatal mortality and has helped community define strategies for improving maternal health and survival in their communities.19 (See Project management: information and logistics) Factors supporting achievement of project objectives At the community level a strong collaboration and partnership between the CBRHP and community leaders was a major factor in creating support for emergency transportation. Intensive supervision of trained VHW and TBAs by village leaders, health facility staff, and CBRHP staff helped mobilize communities to recognize obstetric emergencies. The “TBA danger signs and referral song”, the antenatal card, and IEC messages, and birth planning visits by VHW focused on the increasing awareness of maternal and neonatal care and prompt referral of obstetric complications. The strong partnership and collaboration between CBRHP, DHMT, and local consultants in training of health workers in MNC and EmOC was instrumental in increasing the detection, management, and referral of obstetric complications at the health units. A Basic EmOC manual for the region was developed that took into account local obstetric problems and effective management with available resources. In addition, the DHMTs, through support supervision of health units, identified problem areas at health unit staffing and addressed some of them through redeployment of staff. Constraints A major constraint is that dispensaries currently do not have the capacity to provide basic EmOC and lack the capacity to refer and/or assure the transfer of patients experiencing obstetric complications. The majority of health units in the project area are dispensaries. VHW and TBAs refer mothers to “the nearest health unit” where medical workers must then again refer mothers to the next level of care if they cannot manage the complication. Unfortunately, there is no system of referral or transfer or communication between levels of care, making it very difficult for peripheral facilities to refer, get advice, or transfer complicated patients. Another constraint is the delay in decision making at the community level. Focus group data showed that men are the main decision-makers who may cause delay in seeking transport given that they are either not at home or are not aware of the emergency. An earlier study carried out by CBRHP that assessed male involvement in reproductive health showed that 19 MPHCSS evaluation was carried out just after the final evaluation and should give more information on the impact of the surveillance system 18 there was limited couple communication in pregnancy and delivery. Some factors that limit male involvement identified included cultural beliefs, sayings and taboos, absence from home, feeling of shame if seen supporting his wife, and lack of money. CBRHP also attempted to improve couple communication through birth planning visits by VHWs, community leader discussions at community meetings, and even community education on market days. The market day strategy proved unsuccessful and was discontinued. New strategies must be developed therefore if this approach is to be successful in the future. Cultural beliefs, myths, and sayings limited prompt management of obstetric complications. Eclampsia is also believed to be due to multiple partners and is treated by taking sand mixed with water in a glass. The sand should be from place where a dog has delivered [Men Mwabachuma] Adequate supplies of consumables (such as IV fluids, gloves and sutures) continues to be a challenge for health facilities dependent upon MoH logistics and supply systems. The CBRHP did not have direct control over supplies of these items. The MoH supplies these consumables to hospitals and health centers and this supply is erratic and inadequate. The FEMME project will attempt to address some of these issues. Lessons learned Involving communities in collection, monitoring, and analysis of maternal and newborn data helps focus attention on maternal and neonatal health. The MPHCSS has been a useful tool in assisting community leaders, health care managers stimulate discussion on the causes, complications and management of pregnant mothers that the newborn. Men are the main decision makers and are difficult to reach with appropriate reproductive health messages. Better/new/effective strategies are required to increase male involvement in reproductive health and improve couple communication. Developing an effective referral system requires a system able to refer or communicate with the next referral level. Developing a community referral system without access to communication and higher referral levels and transportation will have a limited impact on management of obstetric emergencies and maternal mortality. Mobilization of communities for maternal health service use should go hand in hand with improving health facilities at all levels in terms of skilled providers, equipment, and supplies. A system of transportation and communication from health units to the next referral level should be developed. Involve as many community social groups as possible in developing or uptake of the new technologies. Involvement of all levels of leaders from different parties makes the establishment of the transportation system easier) Exercise caution in adopting new technology. There is a fine line between offering a community some options (e.g. for community transportation) and being seen as “selling” a specific solution. For examples, several communities latched on to the idea of tricycles even if they were not yet available or if other options may have been more suitable. Multiple local options should be explored before adopting new technology. 19 2.4 Increased voluntary utilization of effective family planning methods The CBRHP approach to increasing voluntary utilization of family planning was based upon increasing community awareness of modern contraceptive methods, increasing access to non￾prescription contraceptive methods at community level and improving the quality of family planning service delivery at health units. Table 5 presents project achievements, which show that CBRHP met most of its objectives in this technical area. Table 5. Project achievements in increasing utilization of effective contraceptive methods. Outcome Indicator Baseline Target Achievement Increased voluntary utilization of effective family planning methods Proportion of women with children under 23 months who do not desire another child within the next two years who are using a modern contraceptive method 11% 20% 24% Couple years protection 611* 1222* 910* Output Indicator Baseline Target Achievement 3.1 Increased access to contraceptive methods available at community level Number of condoms sold by VHWs Number of condoms distributed free of charge 395* 2153* 25% above baseline 4149* 11563* 3.2 Increased awareness of family planning among couples Proportion of couples receiving postpartum family planning counseling 23% 50% 37% 3.3 Improved basic family planning services available at health facilities by skilled providers Number of health facilities with skilled providers for basic family planning (at least one provider had received formal training in family planning) 41% 75% 100% * Quarter Achievements by Indicator ðOutcome indicator: Proportion of women with children under 23 months who do not desire another child within the next two years who are using a modern contraceptive method increases from 11% to 20% In both the baseline and final evaluation surveys, 153 mothers (53%) of the mothers interviewed said they did not want a child in the next two years. The proportion of those mothers using modern contraceptive methods increased from the baseline 11% to 24%. Injectables were used by 45%(17/37) of these mothers and oral contraceptive pills were used by 40%(15/37) of the mothers. Thus, CBRHP was successful in achieving its target. ðOutcome indicator: Couple years of protection (CYP) increased from 611 to 1222 CYP is usually used to monitor progress in the delivery of contraceptive services at program and project levels One of it advantages is that it is calculated from data collected routinely from all the different service delivery points in place in the project area such as health units and social marketing at community level. Health facilities and VHWs are expected to submit quarterly reports to CBRHP. Unfortunately, reporting by health facilities and communities was incomplete in most quarters limiting the reporting of this indicator. Despite this limitation, project-monitoring data shows an increasing trend in CYP by quarter. The project must however, be cited for its lack of an adequate monitoring and evaluation system capable of tracking its defined indicators. 20 Review of MTUHA data shows that the number of new family planning clients increased from 8,5551 in 1996 to more than 8712 in 2000. Results by Output ðOutput 3.1 Increased access to contraceptive methods available in the community. Increased access to contraceptive methods at the community level was measured by increase of condoms sold or freely supplied in the community. Project database showed that condom sales have been rising from 395 pieces sold in since the first quarter of 1999 to 4,149 in October-December, 2000. Free condom distribution at all MoH facilities in the project area increased five-fold in the same period from 2,153 supplied in the baseline quarter to 11,563 in the October-December, 2000. The evaluation team noted that this indicator is a vague and perhaps inappropriate indicator for access to contraceptive methods. It monitors only one contraceptive method, and measures both access to a service and behavior of community members. A more appropriate indicator for increased access would be the number of service delivery points available in the community. A VHW should then be considered a service delivery point given that VHW kits are stocked with condoms for sale to community members. The evaluation team did not assess other condom outlets, which include drugshops, ordinary shops, and some bars that have personnel trained in condom social marketing by PSI. ðOutput 3.2. Increase awareness of family planning among couples Family planning awareness was measured by spontaneous recall of family planning methods among mothers who had received postpartum family planning counseling. This is a poor indicator of awareness since it only counts those who received counseling and is a better indicator of that counseling effectiveness in transferring information. The KPC survey showed that approximately 75% of the those mothers could spontaneously mention two or more methods of family planning. The most common methods mentioned were oral contraceptive pills and injectable contraceptives (Depo-provera). CBRHP’s target was for 65% of couples to receive postnatal family planning counseling by the VHW. Only 37% of the mothers reported having received postpartum counseling in family planning. Of these, 11% of mothers said VHWs carried out the postpartum counseling sessions. Although the project did not meet the expected target, one needs to acknowledge the potential and increasing role played by VHW in postpartum family planning counseling. While the majority, 93%, of mothers identified health facilities as the source of family planning information, only 13 % of the mothers mentioned VHWs and TBAs as another source of information. Traditional healers were also mentioned as a source of information and services. Mothers consider these the traditional methods offered by healers as effective contraceptive methods. This may limit modern contraceptive uptake by couples. ðOutput 3.3 Improved basic family planning services available at health facilities by skilled providers CBRHP conducted a one-week orientation course on basic family training for 59 health workers (MCHAs and COs) from all facilities that did not have a trained family planning provider. A CBRHP assessment of all health facilities showed improvement in the quality of services offered at health facilities (Table 6). Excluding on-the-job trainees 70% (n=46) of the health units had a trained FP counsellor available. Of all observations, 38% demonstrated high counselling quality; 40.9% demonstrated moderate counselling quality, and 15% demonstrated low counselling quality. 21 Table 6. Family planning counseling quality characteristics from all 66 health units in Project area. Quality Characteristic Number n=66 % Adequate space 48 73 Privacy 38 58 Visible IEC for Family planning 41 62 Counseling 37 56 Utilization of FP guidelines 39 59 Good Communication skills 32 48 Positive non-verbal communication 31 47 Services are quite good nowadays. The majority of the community are utilizing the health facility There's no bribes/charges within the health providers There is client satisfaction, through sometimes there are no drugs Counseling and advice is given to client and patients. (FGD Men_Mbarika) Factors supporting achievement of project objectives Qualitative data and field assessments suggest that increased sources of information (e.g. VHW, TBAs, health facility staff) have led to an increased level of knowledge of family planning methods in the community. Community leaders have supported VHWs by allowing them to talk about the family planning at village meetings. Project data show that on average, 60 people a month in the reporting villages were counseled for family planning education by VHWs. At health units there was close collaboration between the health workers and VHWs in health education activities. VHWs help health facility staff educate mothers on maternal health and family planning during antenatal and child health clinics. Increased availability/access to non-prescription family planning methods through VHWs, shops, and health facilities increased significantly over the life of the project. Furthermore, PSI trained some VHWs in social marketing of condoms which resulted in high condom sales in those communities. Participants in some of the focus group discussions noted improved quality of family planning services. Pills, condoms, and injectables were freely available at the health units, and some health workers had good client provider relationships. The MoH had a strong team of Family Planning trainers and CBRHP management effectively tapped this resource. The MoH was receptive and adapted the basic family planning curriculum to a shorter orientation course in basic family planning that was developed and tested and used by CBRHP. This allowed effective use of resources to train more health workers in family planning. Constraints Strong cultural beliefs, myths, and sayings still hinder the use of family planning services in these communities. The key issues that emerge from the focus group discussions were ÿ spousal attitudes and behavior,(e.g. attitude to vasectomy, value of many children); The Ideal family size five & above. Less than four children you have not harvested properly from your wife. Those who have few children have few eggs – are unfortunate. One who gets one or two is called “ubulinaneja” – was a prostitute before marriage. (Women’s group _Luhala) ÿ fear of side effects, especially physical weakness and illness; ÿ fears about sexuality (both male and female performance), 22 ÿ prevalent misconceptions about family planning methods (e.g. bursting of condoms). ÿ Women reported that despite high level of knowledge of family planning methods, male commitment to contraceptive use is low CBRHP family planning IEC strategy was to use existing family planning IEC materials supplied by the Ministry of Health. Although VHWs were given manuals with adequate training and educational materials, IEC materials e.g. posters at health facility and community were lacking. Lessons Learned and Recommendations Close collaboration between community volunteers and health workers improves activity of the community volunteers. The collaboration between VHWs and health workers earns them community respect and this motivates them. Basic family planning course should be adapted to local conditions. A shorter training course on basic family planning and increased support supervision of trainees produced efficient trainees. Community data collection remains a challenge. A simple mechanism of data collection, analysis, interpretation and storage is required. 23 2.5 Increased utilization of STI control and prevention measures Achievements by indicator CRHP strategies to increase STI control and prevention measures were increasing community education on STI, social marketing of condoms, and training of health workers in STI diagnosis and management. Table 7 shows that the voluntary utilization of STI prevention measures and more than 75% of the health facilities had a trained provider and offered quality services. Table 7. Project achievements in increasing utilization of STI control and intervention measures Outcome Indicator Baseline Target Achieved Increased utilization of sexually transmitted infection control and prevention measures Increased number of STI clients attended at health facilities providing syndromic management 6,444 12,888 11,833+Kwi mba 2000 summary Output Indicator Baseline Target Achieved 4.1 Increased availability of information, education, and communication regarding STDs including modes of transmission, signs and symptoms and preventive measures Increase percentage of mothers who can recognize at least 2 signs or symptoms of STDs 11%20 20% 60% Increase population’s knowledge of main modes of transmission of HIV/AIDS and STI 60% 75% 83% 4.2 Increased health facility capacity to provide STD treatment and counseling services Increased the proportion of health facility staff able to effectively counsel patients on STD NA 20% 67% Number of health facilities with skilled providers for syndromic management and confidential counseling 28% (n=19) 75 % (40) health facilities 80% (53) 4.3 Increased availability of protective measures from STDs/HIV at the community level Proportion of communities that have a condom distribution point with no stock outs in past 30 days 21 0 N/A Number of condoms distributed free of charge (quarterly) 2153 4300 11563 ðOutcome Indicator: Increased utilization of sexually transmitted infection control and prevention measures A study carried out in 1996 found that 6,444 clients sought STI treatment from health units in the project area. In 2001, data collected just rpior to the evaluation team’s arrival from 63 health units showed that 2,233 clients were treated for STI in the three months before the survey. MTUHA data show that in 2000 more than 11,863 clients sought treatment for STI. Although the project target was met, the CBRHP assessment of STI services in 2001 found that 25% of clients were misdiagnosed by the health workers. 20 Baseline assessed whether women could name at least 1 sign/symptom; baseline value for the indicator used in later surveys would be less than 11% 21 defined as villages with active VHWs having kits 24 ðOutput indicator 4.1: Increase knowledge of the main modes of transmission of HIV/AIDS and STIs from 60% to 70%. Most KPC survey respondents have heard of HIV/AIDS. The proportion of mothers who reported having heard of HIV/AIDS, increased form 84% at baseline to 95% at final evaluation. An important step in increasing awareness and preventive practices is an understanding of how the virus is transmitted. Table 8 shows the level of knowledge of the main modes of transmission of HIV/AIDS. The proportion of mothers who did know any mode of transmission reduced from 33% to 10%, while those who recognize sexual transmission increased from 60% to 82%. Less than 10% of the mothers could mention the other main modes of transmission. Thus, the project partially met their objective. Table 8. Knowledge of main modes of transmission of HIV/AIDS Mode 1997 2001 Sexual contact 60.0% 82.0% Razor blades N/A 21.0% Blood transfusion 20.7% 6.7% Mother-to-Child transmission N/A 4.3% Don’t know 33.0% 10.0% ðOutput indicator 4.1: Percentage of population who can recognize at least two signs or symptoms of STIs will increase from 11% to 20%. Respondents were asked to mention all signs and symptoms of STI in both women and men. The 1997 baseline survey did not specify signs and symptoms by gender. Table 9 presents results of respondents knowledge about STI signs in both men and women. Overall, 60% of respondents knew two or more signs of STI. Table 9. Respondents Knowledge of STI signs and symptoms occurring in both men and women at baseline and final evaluation STI sign or symptom Baseline 1997 Final evaluation 2001 Women Men Lower abdominal pain 16% 59% 40% Genital sores N/A 49% 47% Discharge 13% 35% 31% Painful urination 8.7% 26% 32% Blood in urine N/A 10% 10% 2 or more signs 60% ðOutput indicator 4.2: The number of health facilities that have trained personnel to detect, counsel and refer STI cases will increase from 19 to 40. Prior to the arrival of the evaluation team, CBRHP conducted an assessment study at 63 health facilities reviewed and assessed STI counseling techniques used by health workers. Of the health workers observed, 93% demonstrated active listening skills, 90% conducted sessions in privacy, and 85% used the syndromic approach to STI management. However, only 46% used the STI risk score to screen potential clients, and only 67% demonstrated the correct diagnosis using the syndromic approach. Good quality counseling was offered by 67% of the health providers observed. Of those that gave the incorrect diagnosis 30%(6/20) were providers not trained in STI management. Furthermore, 75% (6/8) of untrained workers gave the incorrect diagnosis and 25%(14/54) of the trained providers gave the incorrect diagnosis. Thus untrained providers, and those that do not use a flow charts, were more likely to give an incorrect diagnosis. This suggests that training of health workers improved STI case 25 management. There appears to be an important, ongoing need for this type of training and supportive supervision in its application on the job. CBRHP achieved its objective of having more than 75% of health facilities with at least one person trained in STI management. The number of health facilities with at least one health worker trained in STI syndromic management increased from 19 to 53. All hospitals, health centers, and 75% of the dispensaries have key equipment for the diagnosis and treatment of STI (Speculum, sponge holding forceps, and registers). At the district level, 65% (19/29) of health facilities in Kwimba, and 27% (9/33) of those in Missungwi had 8 or more STI drugs. Although the project was not directly involved in STI drug distribution and supply, there is a large disparity in STI drug availability by district with more than two-thirds of health units in Kwimba and less than one third of health units in Missungwi well stocked with 8 or more types of STI drugs. Other indicators related to HIV/AIDS and STDs: ÷ STI prevention The level of awareness about STI prevention amongst the respondents is high and increased over the life of the project. The proportion of mothers who reported condoms as a method of prevention of HIV/STI increased from 22% to 65%. Those who reported having one sexual partner as protective behavior increased from 7% to 51%. Knowledge about the health unit as a source of condoms increased from 56% to 74%. However, VHW as source of information or prevention was mentioned by only 5%, which suggests that VHW are not yet recognized by the community as a source information on of STI/HIV prevention. ÷ STI treatment source Respondents at the baseline and final evaluation were asked to mention STI treatment sources. In the final evaluation survey, 95% of respondents mentioned health units and 24% of respondents mentioned traditional healers as sources of treatment for STIs. Traditional healers are frequently consulted in the management of STI either before going to the health facility or after (for “cleansing”). Other reasons for visiting traditional healers include privacy and confidentiality. Factors supporting achievement of project objectives At community level, VHW were trained to carry out community education on HIV/ STI prevention and are key people in community to provide STI/HIV education, symptoms and prevention and use of condoms. Community monitoring data of the three months before the survey show that VHWs on average educated 60 people per month (range 0- 670) in STI and HIV. Some of these people were educated at community meetings. CBRHP, in collaboration with PSI trained VHWs in identified high transmission areas in social marketing and promotion of condoms. Some of these VHW have been very successful in marketing condoms in these areas. CBRHP enlisted village leaders’ support for HIV/STI education and prevention activities through orientation seminars and meetings. Some village leaders addressed HIV/STI issues at community meetings and allowed VHWs to talk about HIV/AIDS/STI at community meetings At health facility level, factors that supported project achievements include: condom availability, improved quality of care for STI clients by trained health workers, and VHWs carrying out STI/HIV/AIDS education at health units. Other partners, especially the MoH and PSI have supported provision of condoms to health facilities and the communities. Assessment studies showed that condoms were available at 85% of health facilities visited. 26 Support supervision by both MoH and Project staff to health facilities and VHWs was important for maintaining the quality of services provided. Anecdotal information shows that the DMO’s made “corrective” transfers and of staff where staff performance or service delivery was considered inadequate. Constraints to program achievements Other than access to health units that can diagnose and treat STI, the main constraints to increased utilisation of the STI control and prevention measures are people’s perceptions about management of STI and community attitudes to STI infections. These include: ÿ Perceptions of care seeking behaviors: There is a sense of shame to go to health facilities to get condoms even though these are free at the health facilities. Health facility are said to lack privacy and confidentiality. ÿ Perceptions of STIs and HIV/AIDS illness: Relatives do not believe that patients have HIV/AIDS. They perceive it as being bewitched due to stealing and are buried with their property. People say that they see signs and symptoms related to bewitching not AIDS. “kudhulumu mali ya watu”. (Mens group Wazamiso) ÿ STI is stigmatized in the community. A person with AIDS is regarded as a misfortune and is isolated from the community and his/her relatives. People with STIs or a woman buying condoms are regarded as prostitutes. ÿ Prevalent myths and beliefs about STI, HIV/AIDS revolve around condom safety and reliability, the origin of STI and AIDS and complications of complications of STI and HIV/AIDs. These myths will affect condom use and health seeking behavior of community members. ÿ No partner notification: Partner notification was a major concern for both men and the women. Partner notification not done because women feared the repercussions following the disclosure. Wives will never tell their husbands if they contacted the infection from outside as they are worried of being beaten or divorced. (Womens group￾Wazamiso) Lessons Learned and Recommendations: Work with Traditional Healers: People are going to see traditional healers more just as they are going to the health clinics more. There should be an effort to work with Traditional Healers and work together with the health facilities and traditional healers to coordinate efforts in STI/HIV prevention and care. In order to begin this process we need to understand better what Traditional Healers are diagnosing, what the practices are in terms of prevention and care and how we can best work together. Dialogue and communication needs to begin to include the traditional healers in CARE efforts. The approach needs to be respectful and inclusive. Continued support and encouragement for the VHW doing STI education Community health volunteers need to be encouraged to carry out more STI/HIV/AIDS education in the communities (see other lessons learned for VHW support) Identify People Living with AIDS (PLWA): Identify people living with AIDS in order to support them and educate the community on AIDS prevention and care by attempting to decrease the stigma that is associated with AIDS. Integrate HIV/AIDS into all STI training and educational materials: The evaluation team had an impression that more emphasis was place on STI management than on HIV/AIDS. 27 HIV/AIDS counseling on prevention and care should be integrated into all training on STIs, with traditional healers, health facility personal, VHWs, etc. IEC materials (Brochure, video or materials that identify myths, testing, knowledge about HIV/STIs etc) should be produced. 28 2.6 Operational research and other analyses: The CBRHP has carried out 4 separate operational research and assessment activities. These have been extremely useful in documenting the project’s progress, tailoring project activities to identified needs and constraints and disseminating lessons learned. Publications and presentations done by CBRHP staff include: ÿ Kaiza J, Kouletio M, and Ali A. “Couple Communication for Reproductive Health.” Paper presented at the Annual Conference of the American Public Health Association, November 1999. ÿ Kouletio M, Kanenda O., and Myeya J. “Applying Principles of Community Participation to Improve Emergency Transportation: A case example from Tanzania.” Paper presented at the CORE Maternal and Newborn Working Group’s Effective Strategies to Promote Quality Maternal and Newborn Care Workshop in Nairobi, Kenya on 22-26 May 2000. ÿ Kilonzo A, Kouletio M, Whitehead S, McCarthy B. “Improving maternal and perinatal health care surveillance in two districts of rural Tanzania.” Paper approved for publication in Special Issue of American Journal of Public Health, October 2001. ÿ Schmid T, Ahluwalia I, Kanenda O, and Kouletio M. “Safer Motherhood in Tanzania: Evaluation of A Community-Based Approach.” Paper approved for publication in Special Issue of American Journal of Public Health, October 2001. 29 3. Crosscutting Approaches The CBRHP strategy employed two basic approaches across all four of the project’s technical areas: community mobilization and capacity building. It might be argued that when community mobilization and capacity building converge, that an environment is created where by empowerment can take place. a. Community Mobilization Community mobilization was central to CBRHP implementation. A discussion of CBRHP efforts can be organized around two key community mobilization themes: ÿ mobilizing communities to choose and support community health agents (VHWs/TBAs); and ÿ mobilizing communities to develop collective mechanisms to address identified health problems (the need for transportation in the case of obstetric emergency). ð Mobilizing Communities To Choose And Support Community Health Agents (VHWs/TBAs) Community health agents (VHWs/TBAs) are a recognized part of the Tanzanian health system. However, years to neglect had meant that this important resource had withered from neglect and by the time CBRHP became operational in 1996 they existed in name only in the project area. The MoH had criteria for their selection and basic job descriptions outlining their role as community health agents. The initial community mobilization efforts by CBRHP were designed to resurrect these agents and to make them an effective part of the health system. CBRHP limited the role of VHWs and TBAs to one which was directly linked to the four technical areas of the project and its objectives. CBRHP successfully mobilized 143 communities to come together and select their own VHWs/TBAs according to MoH selection criteria. A total of 299 VHWs and 223 TBAs were identified and trained. The mobilization process occurred over the life of the project (several divisions were begun in 1997 and the last initiated in 2000). This means that communities in the project area have had an active VHWs/TBAs for varying lengths of time. Given the time/contact intensive nature of community mobilization this may have implications for the sustainability of VHW/TBA activities in the relatively newer areas. Beyond the simple selection of their VHWs/TBAs agents, communities were responsible for the support of their agents. This support was to be both supervisory/consultative and financial. It was recognized from the start that the community must be involved in the review and discussion of their agent’s activities. The project made efforts to build capacity and involve community leaders in reviewing the work of their VHWs. Community reports were field and meetings held to discuss. In the case of 32 surveillance villages, a community bulletin board was established o publicly display the results of VHW activities. It was intended that these boards serve as a mobilization tool to increase involvement of the entire community in monitoring the activities of their VHWs. Anecdotal evidence suggests that this may be the case. It appears that these 32 villages are far more active in support for their VHW and other project initiated activities than their “non-surveillance” neighbors. Of course the greater attention and time paid to these villages in order to establish the surveillance system is an alternative explanation of this phenomenon. Technical assistance from the CDC is planned to further explore this effect. Communities were mobilized to support their VHWs/TBAs financially. This was done by supporting a cost-recovery approach to sustaining the contents for their drug kits. It was also done by efforts to identify a mechanism capable of providing some form of incentive to the agents in order to motivate them and diminish dropout. While no final solution to this issue has been developed, the communities have identified and are working through several possible solutions. 30 One possible solution identified by the community was to allocate local government revenues to the VHWs/TBAs on a monthly basis (approximately US$ 11.50 to be divided among all active VHWs and TBAs in a given village). While the implementation of this proposed solution has proven problematic (local governments do not generate sufficient revenues) it is important that it has been proposed by the community. Some of the local areas have in fact made partial payments to the VHWs/TBAS. However, this mechanism does not appear to be the solution to this problem. Clearly CBRHP has facilitated community mobilization and dialogue in search of mechanisms to support their health agents. A more recent community solution to this issue has been to create ward-level VHW/TBA associations. The objective of the association is to develop micro-credit and other economic activities among the group. This may provide a greater incentive to VHWs/TBAs than simple payments each month. Additionally it will provide a sustainable mechanism to continue to meet with, re-train and support participating VHWs and TBAs. In 21 out of 32(66%) wards, associations have been formed, by laws written and contributions collected towards the creation of their own lending capital. Another three wards have written by laws but not yet amassed any capital. Six wards are currently in the process of developing their by-laws. The creation of these associations as a community identified and initiated response to the problem of VHW/TBA incentives is truly remarkable. It is a product of community mobilization efforts by the project but one which the project did not initiate. This solution arose from a joint meeting with division secretaries and VHW representatives to review the problems of VHW sustainability and identify solutions. The solution was their own. It suggests that past project efforts have been successful in focusing community attention on health issues and provided a mechanism for collectively dialogue and problem solving. Donors such as USAID should carefully examine this experience and the success of the associations in keeping community health agents actively engaged in service to the community. ð Mobilizing Communities To Develop Collective Mechanisms To Address Identified Health Problems (The Need For Transportation In The Case Of Obstetric Emergency) The project mobilized communities to address the specific issue of maternal mortality. It did so by a series of deliberate steps to expose all community members to opportunities to discuss problems and collectively debate solutions. Through this process CARE has recognized greater overall participation in these discussions and especially seen an increase in the participation of women. While conventional wisdom suggests that “Sukuma women will not participate in public discussion/debate” the CBRHP has proven the contrary. And while the development of specific emergency transportation plans has been the immediate outcome, it is hoped that the process of facilitation and mobilization has provided communities with an ongoing mechanism for collective identification of health problems and the search for possible solutions. Results to date have been impressive. According to the April 2001 Community Empowerment Report: Total communities mobilized = 51 87% mobilized communities have written plans (n=45) 54% have up to date action plans (n=28) 37% actively contributing funds/materials (n=19) 19% functional transport systems used in past quarter (n=11) b. Capacity Building Approach Capacity building of communities and partners was a central element of the CBRHP strategy. CBRHP recognizes that building the capacity of communities and their partners requires more than simply training. Capacity building was seen as creating an environment by which project partners can work together to improve maternal and child health in the project districts. CBRHP was not able to provide all of the resources necessary or imaginable that might be required. The project sought to 31 create an environment in which the partners could practice sustainable behaviors and actions capable of contributing to sustained improvements in the health of communities. (i) Strengthening the PVO Organization The CBRHP has had a major effect on CARE’s ability to address maternal and newborn issues related to child survival. This is important, as maternal and newborn health and mortality have in many ways constituted the “missing/forgotten” piece of the child survival puzzle. Addressing this piece of the puzzle has become increasingly important as child mortality due to other causes has begun to fall making maternal/newborn mortality a relatively larger contributor to overall mortality. This shift in the relative importance of this piece of the puzzle has not been lost on CARE. CBRHP has played a major role in recognition of this issue and the search for community based feasible and sustainable strategies to address it. CBRHP experience and results were a key factor in CARE’s successful proposal to Columbia University for funding to improve EmOC in four countries (Tanzania included). CARE is now a collaborator with Save The Children/US to further explore strategies to improve maternal and newborn mortality through a grant from the Bill and Melinda Gates Foundation. CBRHP, through its collaboration with CDC and the CARE- CDC Health Initiative (CCHI) has made significant scientific resources available to CARE/USA and CARE/Tanzania in this area as well. The sharing of results and experiences through the CCHI mechanism has been significant. A number of papers and presentations have resulted from this collaboration on CBRHP. These papers have informed CARE as well as the broader child survival community of CBRHP strategies and their results and impact. The project has served as a field leader for CARE’s global “Maternal and Newborn Health Team” within the Health and Population Unit of CARE/USA. This team aims to spearhead best practices in maternal and newborn health in CARE’s programs around the world through, field-testing best practices and documenting lessons learned. CBRHP served as the lead for the emergency transportation component and made important contributions to the birth planning and emergency obstetric care components. As a result of this effort, three project staff from other projects (Bangladesh, Zambia, and Uganda) addressing maternal/newborn/family planning issues were invited to participate in the CBRHP final evaluation. In addition to contributing high quality observations to the evaluation team, these members will return to their projects ready to implement CBRHP lessons learned into their respective projects. CARE/Tanzania has also benefited from the CBRHP experience. Through it, CARE has developed a strong project implementation and management capacity in the health sector. This capacity was instrumental in CARE’s ability to secure resources through Columbia University, CIDA, CCHI and the LIFE initiative to continue key CBRHP related projects and activities after the termination of USAID funding to CBRHP. The project capacity that CBRHP has built has allowed it to secure more than US$ 1 million over the course of the 5-year CBRHP life of project. CARE/Tanzania has recently been awarded a US$15 million dollar bi-lateral cooperative agreement from USAID/Tanzania. This grant was awarded on a competitive basis and provides resources for CARE to make sub-grants to local NGOs/CBOs in partnership with District Councils. The Voluntary Sector Health Program will cover 30 districts in regions to improve the health status of women and children while also mitigating the impact of HIV/AIDS. The CBRHP capacity and track record in building capacity of partner institutions to effectively improve health services was, undoubtedly, a key factor in CARE’s successful proposal. At the regional level, CBRHP has made CARE a major partner to the MoH in improving health. Its strong collaborative and capacity building approach has led the MoH to view CARE as a (perhaps THE) major partner in the two project districts and the region. This is based upon CARE’s strategy of collaboration and the technical quality of CBRHP interventions. The Regional Medical Officer 32 (RMO) has recently expressed an interest in CARE’s support in order to replicate many of the CBRHP interventions throughout the region and perhaps even throughout the entire Lake Zone (comprised of Mwanza, Kagera and Mara Regions). CARE has organized a follow-up technical workshop for regional representatives with UNICEF, WHO to share its experience with the maternal and perinatal health care surveillance system later this month. In addition, the CBRHP area has been selected for a PATH/USAID funded trial on the effectiveness of disposable clean delivery kits that will be managed by the National Institute for Medical Research. Clearly, both CARE/USA and CARE/Tanzania have benefited from the CBRHP experience. They have both increased their capacity and credibility as a PVO capable of mobilizing communities to improve reproductive health. They have aggressively sought to share those experiences within the CARE community as well as the broader child survival community. They have effectively leveraged this increased capacity to obtain significant additional resources to continue to build their capacity and improve the lives of women and children in Mwanza, Tanzania and around the world. (ii) Strengthening Local Partner Organizations ÿ AMREF: CBRHP collaborated with AMREF to organize training of their staff and health facility personnel in the syndromic treatment and management of STIs. AMREF carried out the ground-breaking research that first linked STI treatment with reductions in the rates of HIV/AIDS incidence. As part of its efforts to assist health facilities to initiate strategies of syndromic treatment of STIs CBRHP built AMREF’s capacity to train health facility personnel in the implementation of the strategy. ÿ PSI: CBRHP and PSI collaborated to expand the network of community agents selling condoms, a key HIV/AIDS and family planning intervention. CBRHP added PSI marketed condoms (Salaama) to VHW drug kits. The condoms are sold at a profit retained by the VHW. The reach of PSI social marketing activities for condoms was expanded considerably in the project districts as a result. PSI conducted a training of trainers in social marketing and supported the team to train 17 village health workers residing in high transmission areas. PSI has provided continued follow-up support and assisted the staff to review issues of condom use among uncircumcised men. ÿ UMATI: CBRHP collaborated with UMATI initially in training of CBRHP staff in family planning methods and promotion strategies. UMATI supported the project with technical assistance during the review of the VHW, integrated reproductive health for health providers, and orientation to family planning training curricula. ÿ NIMR: CBRHP and NIMR are beginning collaboration for LIFE Initiative baseline survey in terms of measuring population sero-prevalence levels and sharing best practices in behaviour change. Both organizations also collaborated in the protocol for clean delivery kits in Kwimba and Missungwi Districts. ÿ UNICEF: Review of district annual plans, complementing each others efforts – training, supplies – for safe motherhood, promotion of newborn immunizations, sharing of IEC materials. ÿ Archdiocese of Mwanza: CBRHP and the Roman Catholic Archdiocese of Mwanza collaborated in the assessment of hospital obstetric services and participation in case management guidelines, total quality management trainers. (iii) Strengthening Communities Community mobilization to improve the health and well being of mothers and children was central to the CBRHP strategy. CBRHP targeted its efforts community capacity building efforts at two levels: community providers (VHWs and TBAs) and community leadership structures. 33 vVHWs/TBAs: The Tanzanian health system has long recognized the potential role of community-based agents (VHWs and TBAs) in providing key basic preventive and (limited) curative services to remote communities. CBRHP also recognized their potential role to monitor pregnant women and their newborns, promote improved care seeking behaviors for at-risk deliveries, promote utilization of services and as communicators of key health messages (maternal and newborn health, family planning, STIs). VHWs/TBAs were trained and supported extensively by the project to insure that they were able to carry out their defined duties and roles as community health agents and promoters. The VHWs and TBAs were equipped with initial drug and supply kits to reinforce their role, Mechanisms were put into place to assist the VHWs to replenish their kits. The project continued to experiment with other mechanisms to improve the ability of VHWs to maintain their kits and the availability of drugs in the community. The project provided bicycles to VHWs to allow them to cover the extensive distances implied by their catchment areas. Management of this resource proved problematic in some instances. Some communities saw the bicycles as incentives to the VHW and therefore felt no need to develop other mechanisms to compensate or “encourage” their VHW. This led to dissatisfaction on the part of some VHWs. It is obvious the CBRHP made significant efforts and impact and build the capacity of a community level resource (VHW/TBA) which at the start of the project existed in name only as part of MoH policy. These capacity building efforts were not without problems and challenges. VHW drop out has and continues to be a concern and a threat to CBRHP sustainability. This issue is not unique to CBRHP and is one that is faced by similar projects around the world. v Community Leadership Structures: In order to support and supervise the community level workers (VHWs/TBAs) and to act as a mechanism for collective action (i.e. emergency transportation plans), the capacities of community leadership structures was strengthened. This was done largely through training and intensive support from the project. All community visits by CBRHP staff were conducted in collaboration with community leaders. The project effectively used a variety of participatory techniques to build the capacity of communities. The effects may be seen in the: ÿ number of villages with active VHWs/TBAs (61%); ÿ ongoing discussions to find effective incentive mechanisms to combat VHW/TBA drop out; and ÿ the number of villages with emergency transportation plans in the case of at risk labor and delivery. (iv) Health Facilities Strengthening A major component of the CBRHP strategy has been the strengthening of all licensed health facilities (MoH and private) to provide quality reproductive health services (especially in the area of EmOC). CBRHP recognized the futility of creating demand for services through community mobilization and building the skills of health providers if those same providers do not have all of the tools necessary to meet demand through the provision of effective, high quality services. CBRHP also recognized that it was not able to meet the routine recurrent needs (in terms of drugs and other supplies) of health facilities. It was also recognized that CBRHP could not meet all of the possible equipment needs of health facilities (the DMOs in both districts have expressed a need for improved communication between health centers and hospitals as well as the need for an ambulance to facilitate transfer of patients from health centers to hospitals). CBRHP inputs were limited and targeted to meeting the basic needs of facilities to provide services at the level intended. CBRHP sustainability concerns meant that it was expected that the MoH (or private partners, principally the Archdiocese of Mwanza) would commit its own (or perhaps those of other donor partners) resources to maintain a reliable supply of these necessary drugs and supplies and obtain additional equipment. 34 An initial assessment of the capabilities of facilities was conducted. This assessment was designed to identify the needs of health facilities in terms of equipment, supplies and skills. The assessment was carried out in both MOH and private health facilities in both districts. Both hospitals and health centers were assessed. As a result of the assessments, CBRHP ensured that all health facilities had the basic equipment to provide their defined/intended level of basic EmOC, family planning, and STI services. As a result of these inputs all four health centers in the two districts were capable of and providing EmOC services including: ÿ administration of antibiotics (IM and IV) for treatment of infection; ÿ administration (IM and IV) of anticonvulsants for treatment of pre-eclampsia; ÿ administration of oxytocis (IM and IV); and ÿ performance of manual removal of retained placenta. An assessment of facilities carried out in May 2001 indicated that all 66 health facilities in the project area were capable of providing basic family planning services, and 56 had personnel and drugs capable of managing STIs. The project strengthened DHMT training and supervision capabilities. CBRHP used MOH “master trainers” in developing its training strategy. These master trainers received important training in reproductive health issues targeted by the project. They are now better trainers and ready to continue providing training in those subjects. By including MOH personnel in its supervisory activities where possible, the project has served to strengthen their abilities and skills in supervising and supporting community level activities. This is an activity which the MOH has largely neglected in the past. Their ability to continue to apply those skills to support VHWs/TBAs and community leadership structures will be an important factor in the sustainability of CBRHP activities and impact. Project data collection and information utilization activities also served to strengthen the ability of DHMTs and health facilities to “manage by numbers”. By collecting key data on key project monitoring indicators from the MOH information system, MUTHA, CBRHP helped to strengthen the MUTHA system. The project also fed back community level information from its own project monitoring system through important feedback mechanisms also provided information to DHMTs and facilities that they previously did not have. DHMTs and facilities are now able to manage better based upon greater information availability The project has also initiated TQM programs in eight health facilities. This is a program of continuous quality improvement which also seeks to involve the community/client in developing plans to improve the quality of services. This can be seen as a capacity building activity which has strengthened both health facilities and community structures. A TQM training and implementation manual has been developed and key DHMT members trained in TQM implementation. The programs in the eight facilities are active and working to identify feasible solutions to quality issues within the resource means of the facilities and the communities they serve. They now have the capacity to continue to spread the methodology to new facilities. Clearly, the DHMTs now have the capacity to continue to improve the quality of services in an increasing number of facilities. Few external resources are required. This constitutes true capacity building. (v) Strengthening Health Worker Performance Directly related to CBRHP efforts to strengthen health facilities and their ability to provide quality services, have been its efforts to strengthen the performance of the health care providers working in those facilities. CBRHP efforts to strengthen health worker performance has been targeted in two key areas: ÿ Provision of Emergency Obstetric Care (EmOC): CBRHP has developed the capacity of four health centers in the project area to provide key EmOC services which were not available before. Providers were trained in techniques and facilities were provided with necessary equipment and an 35 initial stock of required drugs. Capacity building for EmOC was driven by an initial capacity (human and other resources) carried out by the project. The tool developed, is available for similar efforts in Tanzania or elsewhere. ÿ Delivery of Basic Family Planning Services: Health care providers in 66 facilities were trained in counseling and delivery of basic family planning services. Despite some initial discussions surrounding the length and content of the one week CBRHP training curriculum (as opposed to the three week MOH standard training), regional and district MOH officials expressed appreciation for the quality and effectiveness of the training and increases in utilization of family planning services which followed it. (vi) Training Building human resources and capacity through training was central to the CBRHP capacity building approach and touched all of the technical areas of the project. Table 10 summarizes CBRHP training activities (see page 36). 36 Table 10. Training activities, persons trained, and number trained by CBRHP and Partners Persons Trained Number Trained Topics VHWs ÿ 299 ÿ Maternal and newborn care, child spacing, HIV/AIDS/STIs and hygiene and sanitation/basic public health. (Five week course plus one week practical training) TBAs ÿ 214 ÿ Risk factors, danger signs, newborn care, Village Leaders ÿ 286 ÿ Community transport planning Danger signs Birth planning Newborn Care Family planning (modern and natural) STIs/HIV/AIDS VHW supportive supervision (3 meetings x 2 days each) DHMT Members ÿ 4 ÿ 4 ÿ 10 ÿ 2 ÿ Total quality management (5 days) ÿ Information for intervention (5 days) ÿ Emergency Obstetric Care (3 days) ÿ Participatory rural appraisal (21 days) District Councils ÿ 7 ÿ 8 ÿ 20 ÿCross-Visit to Community Health Fund in Igunga ÿ Community mobilization/empowerment ÿ Quantitative and qualitative surveys Health Care Providers/Health Facilities ÿ 66 ÿ 70 ÿ 28 ÿ 59 ÿ 16 ÿ 15 ÿ 16 ÿ 4 ÿ integrated reproductive health ÿ maternal and newborn care ÿ syndromic management of STIs ÿ basic family planning skills ÿ Total Quality Management ÿ Auxillary Nurse Safe Motherhood and Immunizations Training (21 days) – joint with RHMT ÿ EmOC (10 days classroom, 4 days practical at Zonal Hospital) ÿ Participatory rural appraisal (21 days) Health Care Providers/Health Facilities (joint training with UNICEF in Kwimba) ÿ 50 ÿ safe motherhood Project Staff ÿ All ÿ 8 ÿ 4 ÿ 2 ÿ 1 ÿ All ÿ All ÿ 1 ÿ 2 ÿ 5 ÿ 8 ÿ 1 ÿ 1 ÿ 5 ÿ 2 ÿ Community mobilization/empowerment ÿ Participatory rural appraisal ÿ Family planning ÿ STI case management ÿ Breastfeeding and HIV/AIDS ÿ Societal exploration ÿ Total quality management ÿ Sustainability ÿ Project Design ÿ KPC surveys ÿ EmOC protocols ÿ REFLECT methodology ÿ Best practices in child survival ÿ Information for intervention ÿ Supportive supervision 37 d. Sustainabilty Strategy The CBRHP project strategy was conceived in order to create sustained demand and utilization for quality services capable of improving maternal and child health (specifically through interventions to improve/expand maternal and newborn care services, emergency obstetric care, child spacing and HIV/AIDS/STD). The increases in demand, utilization and quality of specified/specific services the project sought to bring about were chosen for their potential impact and ability to bring about significant reductions in maternal mortality and improved maternal and child health. This was to be achieved through community mobilization and capacity building of both communities and MOH health facilities/personnel in the project area. The sustainability of increased demand, utilization and quality of services was central to the project strategy. If those increases could be sustained, then impact could be sustained as well. The major sustainability question facing the evaluation team then is: Can project supported/induced results (increased demand, utilization and quality of key maternal, family planning and HIV/AIDS/STD) capable of producing sustained improvements in maternal mortality and health be sustained with the termination of project inputs? Previous sections of this final evaluation report have addressed the question(s) of whether the project specified outputs were in fact produced and then whether those outputs were, in fact, capable of reducing maternal mortality as specified in the CBRHP goal. The question remains as to whether the project has put into place the structures and skills necessary to sustain those reductions. The project sustainability strategy was well chosen to do so. The CBRHP sustainability strategy rests upon three major pillars: ð sustained availability of quality maternal health, family planning and HIV/AIDS/STD services at MOH facilities in the target districts; ð sustained availability of essential drug and commodities at the community level through community finance mechanisms; and ð improved availability of essential drugs and commodities through improved community based logistics and supply mechanisms. The sustainability goals, objectives and activities are reproduced from the DIP are presented in Table 11: Table 11. CBRHP Sustainability goals, objectives, and activities Sustainability Goals Objectives Activities 1) All MOH facilities will be able to conduct and support maternal health, family planning and HIV/AIDS/STD interventions by the end of the project ÿ facilities up-graded and covering both basic and comprehensive services ÿ critical drugs and commodities are available 90% of the time ÿ in-service training and supervision are functioning and monitored ÿ in-service training ÿ facility improvements ÿ drugs and family planning commodities are supplied on a routine schedule ÿ supervision, training and feedback functioning 2) All participating communities are able to provide cost recovery/replacement support for all VHWs/CBDs on a scale of 35% or higher ÿ communities select and support VHWs/CBDs ÿ cost recovery incentives are established ÿ VHW/CBD replacement program operational ÿ routine meetings with VHWs/CBDs and community leadership conducted ÿ VHW selection criteria based on capability and longevity ÿ performance-based training and re￾training conducted regularly ÿ community provides direct support for training ÿ community leaders given management training needed to support VHWs 3) Establishment of a community-based system for improving the supply and distribution of essential drugs and family planning commodities through a cost sharing mechanism in conjunction with the Regional Medical Supply Department ÿ communities establish organized groups and volunteers to develop a logistics, storage and supply system ÿ community groups establish a cost recovery mechanism ÿ train and organize community members ÿ develop guides for logistics and supply system ÿ develop cash or in-kind payments Source: Child Survival XII: CARE in Tanzania Detailed Implementation Plan: page 58 38 The wording of sustainability goal 2) is ambiguous. It is unclear as to what the “scale of 35% or higher” (35% of VHWs? 35% of costs?) actually refers to. However, numeric target aside, it is clear that the goal for some level of sustainable community financing of the services and commodities supplied by VHWs/CBDs is necessary for sustained impact. Several assumptions appear inherent in the development and presentation of the CBRHP sustainability strategy: ÿ Knowledge transfer is sustainable. This is why the knowledge transfer intermediate objectives of CBRHP are not included in the sustainability strategy. Once knowledge regarding pregnancy/delivery danger signs, STDs, and HIV/AIDS has been transferred to communities through CBRHP structures and community mobilization activities it is no longer necessary to sustain those activities. The CARE strategy recognizes the need for continued support, supervision and feedback to VHWs/TBAs and MOH health personnel in order to sustain the quality and volume of the services they provide. Supportive supervision will continue through pre-existing systems such as: District Health Management Team support/supervision of local providers and health facility in-charges support to community leadership. ÿ The sustainability of increased demand and utilization of services rests with their sustained quality and availability. This is true in many environments. The key to sustained demand and utilization rests with sustained quality and availability. The capacity building approach of the project is central to this assumption. The project also upgraded the equipment of health facilities in the health centers and hospitals and provided basic first aid and clean delivery supplies to communities. It is encouraging that these kits have been kept-up and replenished in many communities. These are common assumptions in child survival planning. They seem reasonable ones upon which to base the CBRHP sustainability strategy. They require no further inputs or activities to be fulfilled. There appear two other external assumptions upon which the ultimate success of the CBRHP sustainability strategy rests: ÿ The MOH is willing and capable of assuming responsibility for continued support, supervision and feedback to community agents and structures as well as its own personnel. This is a fundamental assumption and challenge of many child survival strategies. Lack of resources, weak systems, lack of motivation and inadequate supervision from higher levels conspire to keep MOH structures from fulfilling this key sustainability assumption. However, the availability of additional resources to local government structures under the multi-donor basket funding mechanism, UNICEF and bi-lateral support focused on meeting the needs of district health plans assures some additional resources will be available to the districts to continue the work in improving reproductive and child health. However, these are issues which are clearly beyond the scope of CBRHP. ÿ A reliable supply of essential drugs and commodities will be available to the district level. While the CBRHP strategy clearly recognizes the need for sustainable availability of drugs and commodities to communities it cannot address the availability of those drugs and commodities to those structures responsible for supporting the communities. If drugs and commodities are not available to the district, it cannot, in turn, assure their availability to communities. CBRHP has guided the two districts to establish a Community Health Fund, a social insurance scheme, designed to insure adequate drugs and supplies of health facilities. Both districts have initiated preparations and have demonstrated an active commitment to start such funds in their districts. It would appear that these assumptions (MHO willingness and sustained drug availability to the districts) pose the greatest threats to CBRHP impact sustainability. While they should not be deemed “killer assumptions” which have absolutely no chance of holding up and therefore could doom the 39 strategy to failure from the start, they certainly pose a substantial risk to long-term, sustained impact. To its credit, CBRHP has recognized these threats and has made reasonable efforts to address them through its project interventions. It should be noted that the relatively short time frame of the project (five years) and newness of some of the activities (i.e. ward level VHW associations, district level VHW re-supply mechanisms cast doubt over the sustainability of project impact. Not all divisions may have benefited from project inputs for a sufficient period of time (as a result of the CBRHP phased approach to implementation required due to resource limitations) for them to “take hold” or produce measurable impact. It is also recognized that four years is arguably insufficient to sustain impact of a large-scale project requiring significant behavior and institutional change such as CBRHP. The lack of funding to extend the project time frame is regrettable. However, CARE is actively engaged in and will continue to seek additional funds to complete the project The project has done an admirable job in identifying and leveraging other external resources equivalent to over US$1 million, more than doubling the original USAID grant resources. Many of these resources will continue to support activities related to CBRHP and its objectives for the near future, including: ÿ Columbia University Comprehensive Emergency Obstetric Care intervention which provides over $660,000 to upgrade hospital staff skills, equipment and supplies, and quality assurance processes: ÿ LIFE/GAP initiative which will provide $125,000 to establish the capacity to provide voluntary counselling and testing for HIV/AIDS in the project; and ÿ CCHI to continue the maternal/neonatal surveillance activities in the project area. ÿ The project is to be congratulated for its success, commitment and diligence in this respect. It is hoped that these continuing resources will help to offset the termination of USAID funding for CBRHP and continue to reinforce the strategy and its chances for sustainable impact. Discussion by sustainability goal: ÿ All MOH facilities will be able to conduct and support maternal health, family planning and HIV/AIDS/STD interventions by the end of the project: The project has pursued this objective aggressively. The project has undertaken to significantly improve the skills of District Health Management Team, local health care providers, as well as make supply and equipment upgrades to facilities to insure that all facilities in the project are have the requisite skills and tools to deliver quality services. All four health centers in the project area have trained personnel, equipment and supplies necessary to provide important EmOC services22. In Missungwi District a total of 35 health care providers have been trained in integrated reproductive health, 8 in EmOC skills and 15 SMATs trained in basic reproductive health. In Kwimba District, 35 health care providers have been trained in integrated reproductive health, 8 in EmOC skills, and the project collaborated with UNICEF/Kwimba District Council to train at least one health care provider from each health facility in safe motherhood practices. Seventy percent of facilities (66) now have trained family planning counselors available (with 79% demonstrating high or moderate quality skills when observed/evaluated on-the-job). And 88% of facilities now have at least one provider trained in STIs (58/66). All of these formal trainings were conducted in collaboration with DHMT personnel and collaborating agencies including AMREF, PSI, and UMATI. Supervision and on-the-job training is a key function 22 EmOC services are defined as: ability to administer antibiotics (IM and IV)for infection, administer anti-convulsants (IM and IV) to treat pre-eclampsia, administer oxytocics (IM and IV) and perform manual removal of retained placenta 40 of DHMT and it is expected that they will continue to perform this function effectively and regularly as they did during the project implementation. While limited on-the-job training was conducted (due to time constraints and concerns about its effectiveness), project staff carried out at least quarterly visits to the facilities for technical reviews of activities and data collection. The CBRHP sustainability objective which remains problematic is that of essential drug and commodity availability. CARE has worked to improve the management of drugs and logistics at MOH facilities, particularly at the hospital and health center levels. The project has established mechanisms to improve the availability of drugs through the initial provision and establishment of a re-supply mechanism for essential drugs and commodities to VHWs and TBAs. In some cases community health volunteers have resolved the re-supply issue on their own in collaboration with private pharmacies in their areas. While perhaps sustainable, these solutions may mean higher prices to the VHW and ultimately the community with obvious resulting questions regarding their availability to all members of the community. It must be recognized that the broader issue of drug availability and sustainability goes beyond the district level and ultimately lies outside of the scope of the project. In nine sites the project has successfully initiated active Total Quality Management (TQM) programs. The process has been clearly documented in a facilitator’s guide. The TQM program in those sites will provide an ongoing mechanism to insure and improve the quality of services, including training of trainers. The DHMTs in both districts are now capable of initiating TQM programs in the remaining facilities following 10 days of CBRHP sponsored training and their participation in field￾testing the TQM tools in the project area. With DHMT supervision, this process of initiating quality improvement mechanisms at MOH facilities should be sustainable. ÿ All participating communities are able to provide cost recovery/replacement support for VHWs/CBDs on a scale of 35% or higher: All VHWs/CBDs were selected by the communities themselves according to MOH criteria in order to assure their place (and hopefully support) in the community. In addition, the project has begun to establish ward-level VHW associations as an additional mechanism by which communities (in association with the VHWs themselves) can support VHWs and insure their continued presence and function. These associations were not part of the original project design. They bring together VHWs who pay dues which are matched by the District Council and ward level structures. The dues are used to provide small loans and credits to VHWs as an incentive for their service to the community. The lack of such incentives is the most often cited reason for VHW dropout and low activity levels. This approach is a direct extension of the overall CBRHP community mobilization strategy and is a terrific example of CBRHP innovative application of the strategy to addressing a key threat to program sustainability. The project has worked hard and advocated aggressively to identify and establish community based incentive mechanisms to support VHWs. Some VHWs have stated that they see the CBRHP provided bicycle as an incentive to encourage their activity. In January 1999, both Kwimba and Missuingwi approved the use of local tax revenues to provide direct financial incentives (10,000 TAS or approximately US$ 11.50 per VHW per month) to all VHWs in the district. This is an important political commitment to the function and sustainability of VHWs. The payment of the incentives to date, however, has been sporadic or even non-existent. It is unclear that the 5% margin that VHWs collect on sales from their drug kits will ever generate significant revenue and provide any real incentive to VHWs. This is a MOH policy issue beyond the scope of CBRHP as cost-recovery for drugs is, by policy, limited to 105% of actual costs. Other commodities, such as Salaama condoms, are sold at a 50% profit allowing for additional income for active VHWs. In some areas, VHWs are selling enough of these condoms, for example, to generate monthly profits representing approximately US$1 profit. Another threat to CBRHP sustainability is the question of whether communities will continue to value and support the activities of their VHWs. The ability to meet this threat rests with the success 41 of the CBRHP community mobilization strategy and ultimately the value placed upon VHW activities by the communities they serve. However many factors beyond the control of the project will influence whether this assumption will actually hold up over the long term. It maybe that certain VHW activities will be supported and therefore sustained (i.e. those necessary to insure the availability of drugs and other commodities as they are generally highly valued by communities) by communities and others deemed less valuable to the community (perhaps educational activities?) may not. In this sense then, the role of the VHW, while sustained, may evolve based upon community support/value for certain activities. In addition, VHWs are well known to the members of their respective District Councils. The councils recognize the VHWs represent a potential human resource to available to future health programs for further support and skill-building. ÿ Establishment of a community-based system for improving the supply and distribution of essential drugs and family planning commodities through a cost sharing system in conjunction with the Regional Medical Supply Department. The project has established a system of health facility based pharmacies which will operate as re-supply points for VHWs in each district. The establishment of these pharmacies is an important step towards insuring the potential sustainability of VHW drug supply to the community and strengthening linkages with the District Health Management Team to ensure drug policy compliance. The establishment of these pharmacies had been delayed due to contradictory MOH policies (health facilities were not allowed to sell drugs at the prices necessary, lack of reliable source of drugs, limited drugs available to the health facilities themselves). The new pharmacies offer promise that drug supply can be sustained and managed by committees which include representation of VHWs and TBAs. The success of these committees (or any other drug supply mechanism at the District level) will rest on their ability to procure drugs at the regional level on a reliable and sustainable basis. A cost recovery mechanism to sustain the VHW drug kits is in place. VHWs were given an initial drug kit. There has been a myriad of problems in establishing an effective mechanism for their re￾supply (in part due to MOH policy and drug availability issues). The VHWs are instructed to sell drugs at 105% of their cost. Evidence suggests that some VHWs have been charging higher prices. This is also true with regard to condoms. The established price is 50 TAS while it is known that many are selling them for 100 TAS (the same has been true for sales agents established by PSI who supply condoms in the area). Free condoms are also available at health facilities, guesthouses, and other strategic locations. The latest project assessment of the ability of VHWs to sustain their drug kits using the established cost recovery mechanisms indicates that 66% of VHWs have kits in “good/active” condition (defined as containing supplies of drugs and commodities as originally supplied after initial training). The same assessment indicated that 61% of VHWs were currently “active” (defined as performing defined activities and submitting activity reports as defined by the project information system). In general, it can be concluded that the CBRHP has met the goals and objectives set forth in the DIP’s sustainability strategy. In many cases, it has identified and implemented solutions to sustainability issues which were not contained in the DIP. This is clear evidence of CBRHP’s strong commitment to establishing sustainable systems and impact in the project area. It has given communities and health facilities the tools and skills necessary to continue to deliver quality services capable of saving the lives of women in Kwimba and Missungwi Districts. Due to the CBRHP duel strategies of capacity building and community mobilization, no additional hand-over or transfer plan is necessary. CARE staff never created or assumed a role that would ultimately be taken over by another institution. All local structures were trained and supported in the role they must play to insure sustainability of project impact. 42 The CBRHP has made a concerted effort to see that its sustainability strategy has been implemented. Results are positive and give reasonable hope for sustained impact. Of course the ultimate sustainability of reductions in maternal mortality will be the result of many factors, most of which are beyond the scope of CBRHP and its sustainability strategy. It would be of interest however, to revisit the project site three to five years in the future to assess the sustainability of both impact and project outputs. It is recommended that BUR/PVC set aside funding to allow for such an evaluation of the project’s sustainability. Such an evaluation would provide valuable insight and lessons learned into project and impact sustainability. Lesson Learned: Community mobilization and capacity building as practiced by CBRHP appears capable of establishing systems and structures which give strong indications of being sustained beyond the termination of USAID funding. Future projects must recognize the time and labour requirements imposed by such an approach when estimating geographic coverage and resources required. 43 4. Program Management The CBRHP benefited from generally solid management and support of its activities including planning and evaluation. The highly collaborative and participatory approach brought by CARE and its staff to the project was the right approach to build capacity and sustainability. Strong information systems allowed staff to manage by the number and closely monitor project activities. It appears to have contributed to the overall achievement of project objectives (especially in the case of the maternal/newborn surveillance system developed by the project). As originally conceived the CBRHP was ambitious (perhaps overly ambitious) in terms of the broad scope of activities it sought to achieve and the large, rural area it sought to serve/cover. Project staff adapted the DIP and implementation plans in response to this challenge. They were also both aggressive and highly successful in leveraging USAID funding to CBRHP to obtain additional funding and resources to meet the challenge. The project far exceeded its required 25% matching fund requirement as defined by the original grant. To date it has in fact generated more than US$ 1 million in matching funds. This record is impressive. It is a strong complement to CBRHP sustainability planning and will allow important activities to continue after the termination of USAID funding. These resources, in conjunction with the skills (strengthened by the project) of staff, partners and communities give strong indication that CBRHP impact and results can be sustained. 1. Planning From the original development of the project/grant proposal in 1995, through implementation and evaluation the CBRHP has taken a highly participatory approach. It has sought to include its partners (communities, MOH, PVO institutions) every step of the way. This has strengthened the project. At the local level, DHMTs, District Councils (including councillors), ward and village-level leaders, and VHWs were involved in project planning and implementation. They were included as full partners who, in collaboration with project staff contributed their field implementation experience and needs, policies, priorities and desires to expand upon the DIP in order to implement the project. Extensive participatory community and health facility assessments were conducted during the midterm evaluation and immediately prior to preparation of the third annual report. These activities were instrumental in guiding the development of CBRHP implementation strategies and plans. These activities were supplemented with regular reviews of project plans with regional and district MOH and administrative officials. A project advisory committee was formed which allowed the project to communicate and discuss plans with all partners (including other PVOs) to insure strong planning and maximum collaboration and communication. The project suffered to some extent, from its original DIP and logical framework which were overly ambitious. The original DIP objectives were not well structured or hierarchical. The logical framework was, in many places unclear, and its implementation plans and did not fore see the need to phase-in activities (probably division by division) but rather laid out a plan to go forward with implementation everywhere at once. This was quickly recognized as infeasible and beyond the resource and management capabilities of the project. DIP reviewers recognized this but requested changes found in the 1st Annual Report were not accepted by USAID. Following the mid-term evaluation, the DIP and logical framework were focused and modified with USAID approval. This was a realistic change which is an example of critical and reflective management. In addition, the DIP did not reflect good/thorough understanding of ongoing government reforms, particularly those in the health sector. This led to conflict in some cases (development of cost￾recovery and drug availability plans) between strategies and activities contained in the DIP and the 44 prevailing policy environment. This could have been corrected by including national level MOH personnel in development of the DIP. In addition to these conflicts, the DIP seriously under estimated the resource needs of such an ambitious project. The DIP contained a number of activities for which resources were not contained in the project budget. These gaps might have been identified at that time had the budget been broken down by activity. A single project vehicle, for example, was anticipated for a project designed to cover over 180 villages over a wide, rural area where access is often difficult. For example, the final budget contained inadequate funds for development of a maternal referral system and training of health workers, VHWs and TBAs according to national MOH guidelines for content and duration. This led to conflict over the implementation of training plans. Funds were not included to involve national MOH training experts in developing and carrying out training activities. These problems were partially addressed by project efforts to: ÿ aggressively identify and obtain external resources to supplement USAID funds; ÿ negotiate with regional and district partners to modify training curricula to reduce in￾class time and sharpen/focus training to meet the specific needs of the project; ÿ leverage implementation efforts in collaboration with partner efforts and activities; and, ÿ mobilize community resources to organize local transport systems to fill gaps in referral system adopting strategies which required minimal additional resources. 2. Staff Training Staff training included family planning basics by UMATI, STD management by AMREF, social marketing training by PSI, group formation training by Community Development Department, societal exploration by RTA, and in-service training in community empowerment and reproductive health epidemiology. Other opportunities included participation in regional and international workshops on breastfeeding, safe motherhood, participation in Maternal and Newborn Health Team and regional meetings. Three staff made cross-visits to CARE Bangladesh’s Safe Mother project. CARE USA conducted annual workshops on supportive supervision, community empowerment, sustainability, and best practices in child health in which two project staff attended. CBRHP staff training and development lessons include: ÿ Training large groups together is difficult because of time demanded and staff absence from the field. A more efficient model of training 2-3 staff as trainers who share with the rest of staff as trainers and through in-field demonstration should be employed. ÿ Staff training and development is an on-going process. Project management must seek to identify and provide opportunities for staff to share experiences and skills together. The project team successfully developed and conducted quarterly three-day retreats to facilitate such a process. 3. Supervision of Program Staff VHWs and TBAs were co-supervised by community leaders and local health providers as well as project staff. This is carried out through monthly report sharing and scheduled review with leaders and regularly scheduled visits to assigned health facilities. At this point it must be recognized that supervision and support of community leadership structures is still is weak due to continued lack of ownership of TBAs/VHWs by the communities and tensions/ambiguity over payments and other incentive mechanisms for VHWs. Currently, supervision and support to health care providers by the DHMTs appears adequate for their technical support. Project field officers supervised community structures (at least) every six weeks and health facilities every quarter. Ultimately, the project has decided that this was insufficient at both levels. It must be noted that the frequency of supervision by project staff was, in large part, dictated by available staff resources and the large number of villages to supervise and the time consuming nature of true 45 supportive supervision and community mobilization that was their job. A more optimal ratio of villages per field officer (probably no more than 10) and health facilities per field officer (no more than 3 or 4) would have produced better supervision and support to those structures. Senior project staff supervised field officers at least once per quarter in the field. In addition, they did monthly debriefings and quarterly analysis of their activities. Towards the end of the project, the organizational chart was restructured so that one supervisor was in each district and able to conduct more joint field activities in collaboration with the field officers allowing more coaching and feedback. Health facilities will continue to be supervised by the DHMTs according to their routine schedules. They have capacity (financial, technical, and logistical) to maintain the system. VHW/TBA supervision is weaker but at the level of health provider technical support it can be maintained. The problem of VHW motivation to continue to work remains a major concern for the sustainability of their activities. In areas where UNICEF and other sponsored programs continue to operate, these same VHWs will be incorporated in their programs. VHWs are also now beginning to form their own ward-level associations to strengthen their identity and motivate each other through creation of revolving funds for provision of small loans to active VHW members, sharing of experiences and working together to change health behaviors. These associations are supervised and assisted by key community resources persons including divisional secretaries, ward executive officers, head school teachers and local health providers. They are supported by the community development department at the district level. This is an exciting and encouraging community identified initiative towards solving the problem of VHW motivation and sustainability. It deserves close monitoring and documentation. There is strong evidence that the CBRHP approach to supervision and monitoring has had an effect on systems which will remain in place. Examples include: ÿ health centers continue to supervise dispensaries in their areas; ÿ TQM program functioning effectively in eight facilities; ÿ some local health providers supervising VHWs and TBAs; ÿ some VHWs/TBAs coming to health facilities for feedback and support; ÿ some health providers participating in village PHC meetings; ÿ DHMTs reviewing availability of essential antenatal and delivery supplies in facilities; and ÿ DHMTs more able to coach staff on client’s rights for free services. 4 Human Resources and Staff Management The project has benefited from CARE/Tanzania’s well established personnel management procedures and system. These procedures include; annual appraisals, written and jointly reviewed employment contracts; continued staff development plans, promotion opportunities and the like. These systems will remain in place as activities continue with diverse funding sources. CBRHP has successfully created a staff/office environment that is conducive to team work under sometimes extremely difficult conditions (such as busy implementation schedules, no communications, lack of electricity). Project team spirit is good. Field staff appear to be working together effectively, sharing constraints and problem-solving together. There is a strong “can do” spirit in the project. Efforts to involve the entire staff in project planning and monitoring have obviously paid benefits. Project staff are clearly enthusiastic and proud of the project and its achievements. The project has suffered some from turnover of senior members: the Assistant Project Manager and Project Manager each had to be replaced over the life of the project. The Office Manager changed three times, mostly due to termination at end of annual contract. There was little turnover of field staff with only one actually leaving the project. Several staff were promoted and others were added as 46 coverage expanded. The impact of these changes was a loss of project memory in senior management. However, this was minimized by maintaining close communications between management and field staff and with former senior employees who have continued a relationship with the project (e.g. former APM now serving as Regional Nursing Officer). At the time of the final evaluation, it appears that all project staff have work opportunities within the project which will continue and in other health projects for CARE Tanzania. In addition, opportunities from outside the organizations are posted and encouraged. Staff have highly desired skills and experience. 5. Financial Management Despite some initial problems in identifying required external matching funds and the actual needs of the project to implement its ambitious work plan, the project has been highly successful in this respect and has integrated these resources into a single financial framework. CARE has demonstrated strong budgeting and expenditure monitoring capacity through the development of annual budgets, quarterly reviews, and adjustments. Evidence of this capacity may be seen in the fact that the project recognized a significant financial shortage based upon original budgeting following the mid-term evaluation. This problem was met with and aggressive and successful efforts to meet the financial gap. The financial sustainability of activities will be met through: ÿ assumption of continued supervision to health facilities by the DHMTs which is already in place; ÿ funding support available for hospital upgrading and further work in STD/HIV/AIDS through CARE and other organizations which is already in place; and ÿ central government basket funding, UNICEF support and other external support to districts available to meet priority district needs which includes reproductive health. Project staff attended financial sustainability training by CARE USA. They have worked diligently and aggressively to assure the financial sustainability of project activities needed to sustain impact. Despite the termination of USAID, funding important activities to support communities to improve reproductive and child health will continue in the project area. 6. Logistics The project faced a number of logistical constraints/issues. Among those identified were: ÿ High vehicle operation costs and frequent breakdowns often limited ability of travel to field sites; ÿ Delays in procurement (largely due to lengthy tax exemption processes) affected availability of office supplies and spare parts; and ÿ Difficulties in communication between the project offices and CARE regional, country, and headquarters offices. Telephone lines to the project area were unreliable and did not support email or other electronic data transmission: It should be noted that CARE information services staff provided excellent support in procuring and maintaining computer and related equipment. Regular trips to the field office, trouble-shooting support, and assistance in short-term lending of printers and computers were provided as needed. Project productivity and management was enhanced as a result of this strong support. Clearly, efforts to sustain project related activities in the project area will continue to face the constraints identified above. 7. Information Management The CBRHP developed and implemented an effective system to collect, tabulate and utilize data to track project implementation and to engage its principal partners (communities and MOH) in monitoring project implementation and results. It provided information to guide supervision of 47 community structures as well as progress towards project objectives. While the project computerized the data collected in order to facilitate compilation, tabulation and reporting it was also effective “on paper” at the VHW and community level in stimulating discussions, identification or problems and improvements and guiding supervision. The system linked the community to the health facility by combing data collected from VHWs with the routine MOH data collection and reporting system (MTHUA). The system is represented by the schematic diagram below. CBRHP Monitoring Framework: The project made effective and extensive use of the system to monitor progress. It is a challenge to develop an effective monitoring system that is not so heavy and cumbersome that it risks collapsing under its own weight and is therefore seen as burdensome and is ultimately not sustained. This does not appear to be the case with the CBRHP system. The system includes the key elements found in an effective information and monitoring system: VHWs -maternal registry -schedule -activities notebook -family planning register -sales receipts Health Facility RH Report Copy to catchment facility and collected by FO Feedback loop and data quality assurance Field Officer’s Quarterly Report Village Health Committees Community Report Hospitals Health Dispensaries Centers TBAs -delivery register -TBA kit distribution/ notebook -Data extracted from monthly MTUHA reports Feedback loop and data quality assurance Quarterly District Reports by Field Officer Leader Qtrly implementation review w/ stakeholders (DHMT, HF reps, community reps, and all staff Project Implementation Report Written and Disseminated -copies to: National: RCHU and USAID Regional: RMO, UNICEF, AMREF, UMATI District: DC, DED, Divisional Sec, local partners Community Pregnancy Outcome Board 48 ÿ Tools which guide and facilitate patient/client interaction: The project developed forms and registers for use by the VHWs and TBAs to monitor and track their interactions with pregnant women (mother/newborn card) and communities (TBA activities notebook, VHW activities notebook and registers). The maternal cards/forms, registers and reports are clear, easy to follow and well laid out (the maternal card is the result of several drafts and modifications). The cards and registers incorporate pictographs to facilitate comprehension by users (VHWs, TBAs, mothers, family members). ÿ Tools which facilitate the collection of project activity data: The monthly reports allow for the easy compilation of data by village TBAs and VHWs for reporting purposes. Forms were developed to allow for quick collection of project related data from facility based MTHUA forms. ÿ Tools which facilitate supervision: The monthly community reports filed by the TBAs and VHWs formed the basis for their supervision by both project field officers as well as the communities they serve. Collection of MTHUA data at health facilities provided opportunities for project staff to supervise the collection of data (thereby improving its quality and ultimately its utility) and service delivery methods (thereby improving the quality of services at the facilities). This has resulted in better record keeping at health facilities. ÿ Mechanisms to promote feedback, discussion and utilization of information: As seen in the schematic representation of the system, its design calls for extensive feedback at all levels. This is consistent with the CBRHP community mobilization strategy. It is clearly a system which is well designed and promotes the integration of data and its collection as an activity which is supportive and in fact part of the project implementation rather than a separate activity to appease project management. The system operates in all 143 project villages. Of course it is based on those women who are in contact with the VHWs and TBAs and is therefore not population based. Confusion over the question regarding whether the KPC survey respondent had a project “mother’s card” limited the survey’s ability to estimate the system’s coverage.23 A more extensive perinatal health care surveillance system was also put into place in 32 of the project villages. This system is based upon the same concept of registration and tracking pregnant women through delivery and up to one year postpartum. It allows for the tracking of individuals and their outcomes (both mother and child) with birth weight. This differs from the basic CBRHP system which is designed to track project progress at the village level. Anecdotal evidence suggests that in the surveillance villages coverage is much higher. A special study will be conducted in order to estimate the levels of coverage by this system. The surveillance system also incorporates the concept of a “community board” which is used for the public display of information regarding birth outcomes in that village as an added mechanism for assessment, analysis and action/decision making. The Regional Medical Officer (RMO) in Mwanza has expressed to CARE his interest/desire to extend the surveillance system throughout the entire Region and perhaps to other Regions within the Lake Zone. Evidence also suggests that project progress and birth outcomes may be better in the surveillance villages as well. This suggests that the more extensive follow-up, feedback and communication with villages required by that system actually complements and contributes to CBRHP community mobilization activities. This is worthy of further study and could in fact lead to a modification of the CBRHP community mobilization model in order to improve its coverage and effectiveness. 23 The KPC survey asked whether respondents were in possession of a “CARE card”. It was not until mid-way through the survey that it was understood that in the village the card was in fact known as the “three color card” as it is printed using separate colors (3!) for separate sections. 49 The CBRHP system is relatively simple and the project has built capacity so that may continue to function beyond the termination of project funding. The key to the system’s future functioning will be in the value that partners ultimately place on the information that it produces. Resources are in place to permit continued support to the 32 village surveillance system (supply of registers, local health provider supervision, community boards in place, community structures mobilized and individuals skills enhanced). The DHMTs in the project routinely incorporate system data into their planning and reporting activities. With continued reproductive health activities taking place in both districts, there is a high degree of probability that the system will continue to function and inform planning and service delivery strategies. While the routine system has furnished information for monitoring project progress it has been important to supplement its information with that generated by smaller, targeted studies to define/refine project strategies and approaches. Key, illustrative examples include: ÿ Assessment of health facilities’ capacity to deliver quality reproductive health services: This study allowed the project to define both equipment and drug needs as well as its training strategy and program for facility personnel. ÿ Assessment of couples’ communication regarding reproductive health: This informed the development of strategies to involve men and their role in important reproductive health decisions. ÿ Assessment of barriers to iron folate distribution to pregnant women. Using a TIPS (Trial for Improved Practices) methodology, this study identified constraints to implementation of the current policy to provide iron folate tablets to pregnant women. Results of the study informed CBRHP strategy to improve this important intervention. ÿ Qualitative study to develop a strategy to promote voluntary HIV/AIDS testing and counseling (VCT). The MOH has contributed 2,000,000 TAS for this on-going study. CBRHP has submitted proposals for funding the development and implementation of a VCT program. CBRHP has done an excellent job of information feedback to its partners (communities and MOH). Feedback is regular and is used to stimulate discussion about the project and its results. It appears that the community chalkboards in the 32 surveillance villages have been effective in creating real community concern about birth outcomes and active participation in those communities in developing mechanisms for their improvement. The CBRHP project, thanks in large part due to the quality of its data, has assumed a key role in the development of CARE maternal health programs worldwide. Project personnel from CARE maternal health projects in Zambia and Bangladesh have participated in this evaluation in order to facilitate the transfer of the CBRHP lessons learned and their integration into other reproductive health projects. Data from the project were a critical part of CARE’s planning and submission to Columbia University for funding of projects to improve EmOC. Clearly, through its ability to document project progress and success CBRHP has been able to assume a leadership role within CARE’s strategic vision for improving reproductive health worldwide. 8. Technical and Administrative Support The project was supported by external technical resources from a variety of sources. Table 12 summarises the technical assistance to the project 50 Table 12. External Technical Assistance to CBRHP External Technical Assistance Source Brief Description on Utility Emergency Obstetric Care/Life Saving Skills CDC, American College of Nurse Midwives, Bugando Medical Center Training of trainers, sharing materials, linkage with WHO, development of case management guidelines Maternal and perinatal surveillance CDC/Emory/WHO Collaborative Center System establishment, capacity building in epidemiology, data interpretation and decision-making, total quality management Community involvement/empowerment CDC, Judi Aubel Training of trainers, development of community mobilization models for transport and supervision, process review, monitoring and evaluation indicators and tools Malaria assessment Malarialogist from Kenya Assessment of malaria situation, recommendations on preventive practices and prophylaxis In addition to this technical assistance the project could have benefited from further assistance in the following areas: ÿ Policy support for malaria in pregnancy ÿ Assessment of VHW motivation strategy ÿ Advocacy for access to maternal health services: policy issues on charging, availability of services It should be noted that PVO headquarters/USAID could have better planned and responded to CBRHP technical assistance TA needs through more consistent visits to the field and provision of stronger national advocacy/policy role. Staff change over at PVO headquarters meant that several different people were assigned to support the project resulting in loss of continuity and familiarity with the project. Through its ability to obtain additional external funding the project was highly self reliant and imposed few technical assistance needs to headquarters. Headquarters staff visited the project yearly (1-2 week duration) to provide oversight and support. More intensive support was provided at key project junctures such as baseline and final assessments, DIP development, mid-term and final evaluation, extension proposal planning, etc. 9. Management Lessons Learned CBRHP has learned a number of lessons that may be seen as relevant to the management of other child survival and reproductive health projects. They include: ÿ Careful document of project processes is important for consistent progress. The project lost important institutional memory due to changes in senior project management. Much information about how VHWs were selected, agreements between communities and CARE, and linkage with partners was lost as it had not been thoroughly documented. ÿ Detailed budgeting by activity facilitates realistic budgeting and better financial management. Initial project financial shortfalls were largely due to underestimation of actual costs of trainings, supervision, and vehicle operations. Close monitoring and target setting can facilitate cost-cutting efforts and better planning to allow adequate financial resources for the life of the project. ÿ Additional external resources can be found by leveraging project resources through partners and other donor sources. Projects can operate at a larger scale if they are willing to find leveraging points. This includes building partnerships, expanding on existing interventions, and advocacy. ÿ Establish monitoring and evaluation system early in collaboration with those who will collect the data. Ensure simple mechanism for data input, analysis and storage from the very beginning and build upon this system as information needs expand. ÿ Schemes to motivate volunteers based solely on financial incentive/remuneration are difficult to establish and sustain. They often lead to disillusionment and dropout if they 51 are not fully realized. It appears better to use other motivation approaches such as group building, revolving funds, training, coaching, and cross-visits to stimulate and maintain activity levels of volunteers. ÿ Build on existing structures. Districts and other permanent institutions are better able to assume responsibility for programs that fit well into their operating frameworks and plans. Projects should incorporate existing staff and services to enhance sustainability in the Tanzania setting. ÿ Ensure that the re-supply system for drugs and supplies is adequately developed in consideration of MOH policies and procedures. The original plan to re-supply the community first aid kits largely failed because it was not well-informed by MoH policies regarding cost-sharing for pregnant women and children under-five. Such a system should be reviewed and modified as needed regularly with all stakeholders. ÿ Strategies involving community mobilization and strong collection, management and utilization of information can be mutually supportive in their ability to improve the health of participating communities. 52 5. Conclusions and Recommendations Main accomplishments CBRHP significantly increased communities’ knowledge of maternal and newborn care, STI and HIV/AIDS. This translated into an increased utilization of most maternal health services, family planning services and treatment of STI infections. The reasons why delivery service utilization has remained unchanged may be due to factors that were beyond project influence which include geographical and financial access to the health facilities. The CBRHP strategy of capacity building for all the health units was successful in increasing the number of health units with health workers trained in offering quality reproductive health services. The project specifically designed training modules that addressed local situations and targeted frontline health workers found active in maternal and newborn care. Equitable distribution of trained health workers amongst health facility was achieved by selecting untrained health workers from all health facilities. Achievements Community mobilisation. The community was effectively mobilised to develop collective mechanisms to address identified health problems. CBRHP main strategy of mobilizing communities was training of an existing cadre of Village Health Workers and community leaders. The project was successful in focusing community attention in regarding health issues and provided a mechanism for collective dialogue and problem solving. Examples of these include the community transportation and support strategies for support of VHWs and TBAs. Effective partnerships. CBRHP Project management extensively collaborated with all partners and stakeholders to develop an all-inclusive environment that used the available resources to meet project objectives. The collaborative and participatory approach of the CBRHP was the right approach to build capacity of the community leaders, partners and stakeholders, Effective information system. The information management system developed promotes integration of data and its collection at all levels calls for extensive feedback at all levels. The feedback allows community dialogue and decision making on factors that affect maternal and child health in their communities. The participatory monitoring systems (the community board), bridges community and health facility data for public health information/data for decision making. Financial support. The project has been successful in attracting other funding sources for the various project activities. Development of local tools and protocols. The project adapted tools and protocols for collecting data on maternal health and on management of obstetric emergencies. The project also adapted and developed realistic training manuals for VHWs training in MNC, STI, FP, and HIV/AIDS. Constraints Large geographical coverage. The initial design of the project covered a very large area which stretched resources available to them in terms of supervision and training. Financial constraints. Informal costs at health units may have limited the use of health units by poor community members. At project level, there were limited resources in terms of high operational costs Lack of ownership of TBAs/VHWs by some communities. VHWs and TTBAs were considered by the community members as paid by the training organization and incentives were not given to 53 these volunteers. Thus, community incentives suggested by the community leaders were not given to VHWs and this lead to lower morale among the VHWs and TTBAs. Key recommendations to other projects ÷ Future reproductive health projects should consider a phased approach and use lessons learned at each phase. ÷ Postpartum care is a difficult component to implement. Projects attempting to reach home deliveries with postpartum visits in order to reduce maternal death should reconsider strategy to focus on institutional deliveries given the CBRHP struggle to reach mothers during early postpartum period. ÷ Development of local transportation systems requires acceptable and affordable technology. Research and development should be done long before community mobilization and planning takes place. Communities do not have all the answers especially to this tough question, how to get there quickly. ÷ Through total quality management training and facilitation, root causes of health facility problems affecting service utilization can be successfully addressed. ÷ MPHCSS, is powerful tool to increase local community dialogue and priority setting for maternal and newborn health. ÷Household birth planning has serious limitations in terms of reaching family decision-makers. Alternative channels for reaching men and community elders should be identified. Women can be reached easier at the antenatal clinic than in their homes – in an area with nearly universal antenatal coverage, home-visiting is not the most cost-effective approach. 54 6. Lessons Learned ÷ Carefully define the work we expect a community volunteer can do. Some of the factors to consider include geographical coverage, time associated with the job, incentives requires to motivate the volunteer and community’s capacity to motivate and sustain the volunteer. ÷ Developing only parts of maternal health care system gives limited impact in improving maternal care. An effective maternal referral system requires developing the community referral transport system, the communication between the primary health units, secondary and tertiary health units. Improvements required to raise facility-assisted deliveries were outside the scope and resources of the project. Major barriers to quality of services were shortage of adequately trained staff at health facilities (very few Nurse-Midwives, Clinical Officers, and Assistant Medical Officers), poor infrastructure, and under-table charges for delivery services. These are based on national policies, long-term health sector funding, low salaries, and inadequate supervision. ÷ Community involvement in data collection and analysis improves community dialogue and decision making for addressing maternal and child health issues. The MPHCSS can be a powerful tool for focusing community priorities on maternal and child health. It can engage all members of the community in reviewing pregnancy outcomes and visualizing the extent of maternal and infant mortality. It facilitates discussion about the mortality burden, major underlying causes and potential solutions. There is emerging evidence that area with MPHCSS may have higher levels of VHW satisfaction, greater levels of community participation (more participation by women) and more progress on developing community transport plans. ÷ Support developing local innovative and appropriate mechanisms of supporting community volunteers. Despite heavy advocacy and sensitization at the District Council, ward and community levels for VHW incentives only 11% of the VHWs were paid anything by their village government. Alternative approaches that proved successful in motivating VHWs included: supportive supervision from CARE, MOH staff and Community leaders, frequent in-service visits by CARE and Master trainers, and community appreciation resulting from successful counseling and referral of mothers for emergencies. A promising innovation is the establishment of VHW ward associations. These associations will provide income generation and professional support opportunities as well as serving as a nexus for interactions between VHW s and other groups such as the MOH, local government and NGOs. These groups can be charged with general community health issues such as community transport and reporting of surveillance efforts such as birth monitoring. ÷ Community mobilization requires time and ongoing technical support. The initial strategy to mobilize community leaders who would then mobilize the community did not work in many cases. Additional support from CARE and the Master Trainers is needed during the community wide meeting and subsequent follow-up. Rather than relaying on community officials the members from the community should form a transportation committee. A committee for transportation that included members selected from the community in addition to community leaders would reduce concerns that community leaders may misappropriate the funds. ÷ Local expertise is available as a sustainable resource. Training individuals from a variety of sectors including the ministry of health, local government, community development 55 officers as "Master Trainers" provides a local and sustainable resource for future technical assistance and community mobilization. ÷ Communities and local leaders value tangible benefits. Willingness to participate is highly dependent on payments, direct contribution of supplies and goods, building and renovation of structures. People don’t appreciate health education alone