Medical Care Development International 1742 R Street NW, Washington, DC 20009 * USA Telephone: (202) 462-1920; Fax: (202) 265-4078 Internet Electronic Mail: mcdi@mcd.org URL; http://www.mcd.org MID-TERM EVALUATION REPORT CENTRAL POTOSI CHILD SURVIVAL PROJECT COTAGAITA AND PUNA HEALTH DISTRICTS (CS-XV NEW GRANT RFA 938-99-A-0500-15) April 16, 2002 MCDI USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report TABLE OF CONTENTS II. THE MID-TERM EVALUATION REPORT........................................................................1 A. SUMMARY ........................................................................................................................1 B. ASSESSMENT OF THE PROGRESS MADE TOWARD ACHIEVEMENT OF PROGRAM OBJECTIVES........................................................................................................4 1. TECHNICAL APPROACH..........................................................................................4 a. BRIEF OVERVIEW ...................................................................................................4 b. INTERVENTIONS .....................................................................................................8 1. Immunizations................................................................................................................8 2. Diarrhea .......................................................................................................................13 3. Pneumonia ...................................................................................................................17 4. Maternal and Newborn Care........................................................................................21 5. IMCI.............................................................................................................................26 2. CROSS-CUTTING APPROACHES ..........................................................................31 a. Community Mobilization...........................................................................................31 b. Communication for Behavior Change........................................................................35 c. Capacity Building Approach......................................................................................41 (i) Strengthening the PVO Organization........................................................................41 (ii) Strengthening Local Partner Organizations.............................................................42 (iii) Health Facilities Strengthening ..............................................................................49 (iv) Health Worker Performance ...................................................................................52 (v) Training...................................................................................................................53 d. Sustainability Strategy...............................................................................................54 C. PROGRAM MANAGEMENT..........................................................................................58 1. PLANNING .................................................................................................................58 2. STAFF TRAINING.....................................................................................................59 3. SUPERVISION OF PROGRAM STAFF...................................................................60 4. HUMAN RESOURCES AND STAFF MANAGEMENT..........................................63 5. FINANCIAL MANAGEMENT..................................................................................64 6. LOGISTICS.................................................................................................................65 7. INFORMATION MANAGEMENT...........................................................................67 8. TECHNICAL AND ADMINISTRATIVE SUPPORT...............................................74 D. OTHER ISSUES ..............................................................................................................76 E. CONCLUSIONS AND RECOMMENDATIONS.............................................................76 F. RESULTS HIGHLIGHT.....................................................................................................87 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report III. ACTION PLAN .................................................................................................................88 IV. ATTACHMENTS.............................................................................................................100 ATTACHMENT A: BASELINE INFORMATION FROM THE DIP......................................... ATTACHMENT B: TEAM MEMBERS AND THEIR TITLES.................................................. ATTACHMENT C: ASSESSMENT METHODOLOGY ............................................................. ATTACHMENT D: LIST OF PERSONS CONTACTED............................................................ ATTACHMENT E: TRAINING MATRIX .................................................................................. ATTACHMENT F: COST STUDY PROPOSAL........................................................................ ATTACHMENT G: HMIS.......................................................................................................... ATTACHMENT H: DISKETTE WITH ELECTRONIC COPY OF THE REPORT.................. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report ABBREVIATIONS AND ACRONYMS BCC/IEC Behavior Change Communication/Information, Education, Communication BHR/PVC Bureau of Humanitarian Response, Office of Private and Voluntary Cooperation CAI Committee for Analysis of Information. Now, “TAI”, for “Information Analysis Workshop” CHP Community Health Promoters CHV Community Health Volunteer CHW Community Health Worker CIELO Componente de Intervencion Estrategica Local (Local Strategic Intervention Component) COMUSA Municipal Health Committee CPCSP Central Potosí Child Survival Project CSST Child Survival Support Team CSTS Child Survival Technical Support DIDESCO Community Health Department(from PROSIN) DIP Detailed Implementation Plan DPT Diphtheria, Pertussis, Tetanus Vaccine EOC Emergency Obstetric Care EPI Expanded Program for Immunization GIS Geographic Information System GPS Global Positioning System HH/C-IMCI Household/Community IMCI HMIS or HIS Health Management Information System, Health Information System HP Health Post IMCI Integrated Management of Childhood Illness ISA Institutional Strengths Assessment JICA The Japanese Government International Aid Agency KPC Knowledge, Practices and Coverage (baseline sample survey) LQAS Lot Quality Assurance Sampling (sampling method for monitoring) MCD Medical Care Development MCDI Medical Care Development International MMR Measles, Mumps, Rubella Vaccine MNH Maternal and Neonatal Health MOH Ministry of Health MPH Masters in Public Health NHI National Health Insurance Program (SBS) NID National Immunization Day PCM Pneumonia Case Management PSF Programa Salud Familiar, MOH primary health care project PVO Private Voluntary Organization SEDES Secretaría Departamental de Salud, the health department RN Registered Nurse (university level) RPS Responsible Popular de Salud, Health Promoter USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report SCM Standard Case Management SBS Seguro Basico de Salud SNIS National Health Information System TAI "information analysis workshop" (formerly “CAI”) TBA Traditional Birth Attendant TOR Terms of Reference TT Tetanus Toxoid UNAP The MOH “Unit for Care to People” (the unit responsible for health services at the national level) USG United States Government USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 1 II. THE MID-TERM EVALUATION REPORT A. SUMMARY The goal of MCDI’s Central Potosí Child Survival Project is to reduce child mortality and morbidity, and to improve the health of women of reproductive age in the project area through the interventions in immunizations, diarrheal disease control, pneumonia case management, and maternal and newborn care. MCDI’s overall approach to achieving this goal in this poor overwhelmingly rural area is two-fold: changing key behaviors at the household and community level while improving the quality of health services at the facility level. Our principal partners are the Ministry of Health at the department and district levels as well as the municipalities of Cotagaita and Vitichi. The project fully supports the IMCI strategy of the MOH and is initiating HH/C-IMCI activities. The following activities are being implemented to accomplish the above-mentioned goal: · training all health workers in IMCI and maternal/neonatal health and supervising their performance. · training all health workers, women’s groups and community volunteers in HH/C-IMCI that incorporates community-based maternal/neonatal health content. · implementing a BCC/IEC plan to transmit messages, primarily through interpersonal communication by community volunteers, training women’s and community groups, counseling by health workers during consults, and limited mass communication through health fairs. · strengthening management and administration of the partners through training and supervision, most specifically in collection and use of data for decision-making, management of essential medications and supplies, management of the National Health Insurance program, and analysis of costs for primary health care. Among the main accomplishments to date is the fact that the IMCI training center in Tupiza is functioning, and nearly all health workers in Cotagaita district, representing about half the project zone, have been trained in IMCI and maternal/neonatal health. IMCI protocols are being applied in almost all health facilities. MCDI has recruited and trained a competent and cohesive technical and administrative field team. This team has successfully carried out all the main baseline quantitative and qualitative assessments necessary to refine the project’s design and strategy and has forged strong and productive relationships with both the MOH and Municipal authorities as well as with the communities in which it is working. In addition, in the 31 communities where MCDI is working, the team has formed or strengthened women’s groups, and initial training in HH/C-IMCI is well underway. The project is involved in the PROCOSI/CORE Group HH/C-IMCI coordination initiative and intends to ensure that all new materials will be consistent with the standards of this HH/C-IMCI working group. The team has developed detailed plans, supporting materials and systems for training and supervision within the HH/C-IMCI framework. The project is using the standard supervision protocol developed by the MOH for IMCI (while developing a more integrated protocol to include MNC). There is evidence that immunization coverage is above that established by the baseline KPC and there are signs of increased utilization of health services for the treatment of diarrhea and USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 2 pneumonia and a rise in the percentage of deliveries performed by qualified personnel. Women’s groups in training are responding positively to HH/C-IMCI messages. The project was plagued by serious problems during its first two years that resulted in delayed implementation. The initial partnership with Esperanza/Bolivia was dissolved near the end of the project’s first year, necessitating a redefinition of the primary partner and the recruitment and training of a new field team. The new partner, the Departmental and District MOH, suffered a very high rate of turnover of key staff that severely constrained efforts at coordination and cooperation. The project was again obliged to redefine its partnership, focusing on Municipal governments. Active community health workers proposed by Esperanza were not available in numbers anticipated, and initial attempts to identify and train new ones were unsuccessful. The project adapted its community-level strategy in favor of organizing and training women’s groups community by community, increasing the burden of time on field staff. Moreover, some of the organizations and institutions cited in the DIP as key sources of cooperation and technical assistance were ultimately unavailable to the project, obliging MCDI to assume responsibility for additional tasks. Civil unrest and road blockages coupled with the fact that the project now has only one vehicle of its own available to the expanded five-person team have impeded transportation. (The project manager has been using his own vehicle to support project activities.) Although the team has advanced significantly, activities have so far been implemented in only part of the project zone (i.e., Cotagaita health district), as per the DIP. Two years into the project the team is directly active in 14 of 38 health facilities in the district and 31 of an estimated 140 prioritized communities. There is an urgent need to scale up the scope of implementation if coverage is to be achieved. In spite of the problems in defining the partnerships, there have been positive developments in capacity-building. There is evidence that IMCI protocols are being applied in Cotagaita district facilities, the quality of data in the HIS has improved, and management of essential medications, supplies, and equipment is better. There has been successful implementation of the new MOH HIS (SNIS II) and the expanded EPI program. Because the project is working mostly to strengthen existing MOH and Municipal systems and policies rather than putting new ones in place, sustainability is maximized. The Tupiza IMCI training center is fully operational and can easily continue to train health workers to replace those that leave. Nurse auxiliaries are being trained as facilitators who in turn train community health volunteers, also allowing for replacement. By focusing on strengthening management of the National Health Insurance system, Municipalities will have greater resources to support quality health care and community development in the future. The most important recommendations are: · Scale up the scope of project implementation in Cotagaita as soon as possible. To do this, the team must complete the development of its curriculum and materials for facilitator training in CC/H-IMCI and maternal/neonatal health and complete the training of all USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 3 auxiliary nurses as facilitators of HH/C-IMCI and maternal/newborn care. Following this, the team should delegate training and supervision of community health volunteers to nurse auxiliaries. The CS team can then shift from direct community training to supervision of training. · Complete clinical IMCI training for the remaining eight health workers in Cotagaita district immediately. (Training of these eight health workers is well underway) Begin joint systematic supervision of IMCI, the NHI program and supplies and medications in all health facilities in Cotagaita district. Integrated supervision instruments will have to be finalized first (currently, the IMCI supervision instrument is being used). · Complete training in management of obstetrical emergencies and post-partum care for all health personnel in Cotagaita district, and begin supervision of implementation. The MOH (CIELO) and the MNH Project has agreed to provide training to the CS project. · Finalize the BCC/IEC plan and materials (based on existing IEC materials developed by Nur/PROSIN, CARE and, MotherCare Project and the MNH Project), using external assistance if necessary (a consultant has already been identified), and implement the plan. Develop and implement a simple monitoring system for knowledge and behavior change. LQAS would be an appropriate sampling/analysis method, though external TA will be required. · Support the training and implementation of SEDES’ community-based surveillance system (already scheduled for 2002) · The strategy for the implementation of activities in Puna will need to be reviewed to determine to what extent project resources can support expansion of the project prior to the approval of an extension application in 2003. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 4 B. ASSESSMENT OF THE PROGRESS MADE TOWARD ACHIEVEMENT OF PROGRAM OBJECTIVES 1. TECHNICAL APPROACH a. BRIEF OVERVIEW PVO/Country: Medical Care Development International Bolivia Cooperative Agreement Number: FAO-A-00-99-00023-00 Project duration: 4 years Program start date: October 1,1999 Program end date: September 30, 2003 The detailed implementation plan for MCDI’s Child Survival project lists the following overall goal and objectives for each of its four interventions: Project Goal: To reduce child mortality and morbidity, and to improve the health of women of reproductive age in the project area through interventions in immunizations, diarrheal disease control, pneumonia case management, and maternal and newborn care. Immunization · 50% of children aged 12-23 months fully immunized by card · 60% of mothers have received 2 or more doses of tetanus toxoid vaccine before the birth of their youngest child · 50% of 6-11 month old children have received one dose of Vitamin A supplementation · 50% of 1-5 year olds have received 2 doses of Vitamin A per year (MOH guidelines and based on child cards) Diarrheal disease · 70% children 0-23 months with diarrhea in the past 2 weeks received more breast milk. · 70% of children 0-23 months with diarrhea in the past two weeks received more or the same amount of food. · 70% of children 0-23 months with diarrhea in the past two weeks received more than the usual amount of fluids. · 80%of mothers of children 0-23 months can identify at least three danger signs of dehydration. · 80% of children 0-23 months with diarrhea during the past two weeks received ORT (to include ORS and appropriate home-based rehydration solutions). · 90% of children under 6 months are exclusively breastfed. Pneumonia case management · 70% of mothers of children with pneumonia seek help from appropriate providers. · 70% of mothers and caretakers able to recognize two danger signs of pneumonia as a reason to seek care at health facility. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 5 · 100% of CHPs trained to identify pneumonia danger signs and promote fast referral to health facilities. · 70% of mothers of children with signs of pneumonia seek care upon recognition of symptoms · 100% of mothers or caregivers seeking care for their children in a health facility for pneumonia received appropriate treatment as per PCM protocols [objective added after the DIP] Maternal and newborn care · 50% of pregnant women have the recommended four prenatal visits. · 50% of pregnant women take the prescribed course of Iron/Folic Acid · 50% of pregnant women deliver with trained family birth attendants at home · 90% of new mothers begin breastfeeding within eight hours of delivery · 70% of women receive Vitamin A within eight weeks of giving birth · 80% of women can identify at least three warning signs of emergencies for each stage (pregnancy, labor, birth and postpartum) · 80% of women can identify the danger signs in newborns during the first 24 hours and during the first 7 days. · 80% of new mothers have attended counseling sessions on birth spacing (LAM and modern family planning methods) · 60% of newborns are immunized with BCG within the first month, by card The project area includes two health districts, Cotagaita and Puna, in the southern Bolivian department of Potosí. The target population is largely Quechua-speaking and 98% rural, and, according to the DIP, distributed as follows: Population Totals by District Health District Department Municipality Total Pop % Urban % Rural < 5 years WRA Cotagaita Nor Chichas Cotagaita 27,484 6.2 93.4 4.469 6.620 Vitichi 15,483 0 100 2.462 3.716 Puna` Linares Puna 48,021 0 100 7.494 11.530 Caiza D 11,121 0 100 1.679 2.669 Totals 102,109 16.104 24.535 The area is vast, mountainous and sparsely populated. The Cotagaita district alone has 222 communities. The exact number of villages in the Puna district was not available for the evaluation team, but is thought to be similar. The team has defined 140 villages in Cotagaita district as high priority, as they are the most accessible, well organized, and have relatively large populations. The DIP states that work in Puna district will be phased in during the second half of the project. The network of health facilities is distributed as follows: USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 6 District Municipality Hospital Medical center Health post Total Cotagaita Cotagaita 1 0 25 26 Vitichi 1 6 2 9 Puna Puna 1 4 18 23 Caiza “D” 1 1 8 10 Total 4 11 53 68 The overall approach of the project is relatively straightforward. The project aims to change key behaviors of caretakers at the household and community levels while improving the quality of health services at the health facility level. Messages regarding desired behaviors reach caretakers through several avenues. The team is training mothers in the communities to serve as volunteers who will transmit messages to their peers during mothers’ club meetings and home visits. In addition, health facility staff are being trained to transmit key messages during consults and home visits. As the literacy rate is low and mass media such as radio reach only a small proportion of the population, written material and broadcast media are not felt to be very effective. In addition to one-on-one communication, the team is organizing health fairs in communities to help transmit messages. MCDI is using flip charts with drawings and laminated cards with drawings depicting key messages as visual aids. The materials have been developed from several sources, the most important being the USAID￾funded PROSIN project, which received technical assistance from BASICS, and is being tested by Núr University in Cochabamba. In addition, the project has supplemented these materials with other materials developed by the team, especially for the maternal/ newborn care module. Strengthening the quality of key services at health facilities is occurring at various levels and through a series of different activities, including: · support for training and supervision of all health personnel in clinical IMCI, community IMCI, and community organization. MCDI also plans to support training the same personnel in improved emergency obstetrical care once a partner has been identified (the MNH Project and the MOH (CIELO) have already been identified and will provide training support) · support for systematic supervision of health facility personnel, especially auxiliary nurses. This includes on-the-job refresher training in information management, management of essential medications and rotating drug funds, IMCI, community organization and training and supervision of community health volunteers · Nur University has participated in specific training in management as needed, according to requests and assessments carried out by the team · a study of the costs of primary care (carried out by MCDI), which is expected to lead to revisions in reimbursement rates by the national health insurance plan · logistical support to the health system, especially in transportation and occasionally manpower (especially for immunization campaigns). · participation in TAIs (periodic meetings for data analysis) to aid the MOH's health system in improving its use of data for decision-making · purchase of some key supplies, such as child health cards, and equipment (for deliveries) USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 7 The institutional strengthening component of the project has run into unexpected difficulties. The original proposal identified Esperanza/Bolivia, a local Bolivian NGO associated with the US PVO, Esperanza, as the primary partner. Esperanza/Bolivia participated as a direct implementing agency under the guidance of MCDI through the phases of the initial baseline studies, elaboration of agreements, and elaboration of the original DIP. In mid-2000, by mutual agreement, the partnership between Esperanza/Bolivia and MCDI was dissolved, and the DIP was significantly revised (the reason for this being that SEDES expressed concern about capacity of Esperanza to provide technical assistance to the MOH). MCDI identified the project’s alternative partner as SEDES, the Departmental level MOH, (together with the District and Municipal levels of the MOH as well), and identified Núr University as a second partner. Since then, experience has led the team to shift its focus away from SEDES and the Districts somewhat, toward emphasizing the municipalities as the primary partner—both at the municipal level of the MOH and the municipal governments. Nur University continues to be a partner. The first of the changes, when MCDI and Esperanza/Bolivia parted company, obliged MCDI to form a project team of its own. Subsequent staff turnover has meant that the current technical and administrative teams were constituted only within the past several months. With the exception of the Project Manager, no one currently working for the MCDI field office was involved in gathering the baseline data or in elaborating the DIP. Similar problems were encountered with MOH partners, where very high turnover has meant that with few exceptions, almost all the staff now at the Departmental and District levels is new. Municipal government and national level MOH personnel have been somewhat more stable. This rapid turnover has led to delays in implementing many of the planned activities outlined in the DIP’s work plan, especially those relating to capacity-building of the SEDES and the Districts. In addition, many of those trained using project resources have left the project zone. Examples of the degree of delay in implementation include the following: Training of health facility staff in maternal and newborn care was scheduled for completion in September 2001, but is only partially complete (only nurse auxiliaries in 38 facilities have received any maternal/neonatal training, doctors and licenciadas were not trained because they have been rotated). BCC/IEC activities were supposed to be in full operation by the mid-term evaluation, but only a few messages and materials have recently been finalized. Community health volunteers in Cotagaita district were to have all completed their training by the end of 2001, but this activity is only about 25% (31 out of 140 CHVs) complete in the district. Capacity-building activities at all three central levels have been delayed even further, especially in areas related to management, supervision, and use of information. Fortunately, capacity-building at the health post levels is proceeding almost as planned, especially with regard to the technical areas. Another reason for delays is that some key operational partners are delayed or unable to collaborate at all, for example, the expected collaboration with the “Maternal and Neonatal Health Project” for training on MNC. This project will unfortunately be unable to implement activities in the MCDI project zone as planned, however the MNH is scheduled to start TOT training in April of this year. Another example is the expected use of community-IMCI training materials being tested by Núr University for the PROSIN project. The materials are still being developed and are not yet ready for release. Another issue is that the demand for training by Save USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 8 the Children in Community Health Information System (SECI) was reportedly so high that they were unable to offer training slots for MCDI until 2002 (now that the MOH has formulated its community surveillance system the project will forgo the STC instrument). These combined with other difficulties with partners, have placed an unexpected burden on MCDI to design materials and implement activities as originally anticipated in the DIP. One final important reason for delays across the project is civil unrest. Bolivia appears to be passing through a difficult time, and the frequent strikes and road blockages have led to postponements of project activities. Unfortunately, it appears that this problem will likely worsen in the run-up to elections in 2002. In spite of the difficulties and delays, the mid-term evaluation provided an excellent opportunity to reassess and redirect the project, and it appears that the project will be able to achieve its objectives. b. INTERVENTIONS 1. Immunizations The immunization intervention was not originally a part of the project interventions and was added after the baseline evaluation found coverage to be abysmally low. The inclusion of immunization was also strongly supported by the SEDES, whose own HIS showed low coverage in the project area. The addition of immunization, however, obligated MCDI to significantly scale down its planned nutrition intervention in order stay within the project budget. Vitamin A supplementation for children and breastfeeding women is included as part of the immunization intervention, and breastfeeding promotion is part of the diarrhea intervention. ACTIVITIES AND PROGRESS The principal activities planned in the DIP and progress is summarized in the following table: Planned activities Progress Comments Review the protocol and methods for identifying children and women needing immunization (e.g., TT) The protocols have been reviewed. The team systematically reviews copies of child health cards kept in health posts during supervision visits Twice-yearly door-to-door national immunization days identify unregistered women and children Team members accompany nurse auxiliaries during home visits IMCI supervision has begun and includes supervision of immunization The evaluation team feels that evidence indicates that nearly 100% of all women and children are registered USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 9 Planned activities Progress Comments Improve the cold chain through improved management and maintenance, and “supporting SEDES’ supply and replacement of King Sealer thermos Formal evaluation of EPI and cold chain was completed by MCDI for Cotagaita District 9/01. Report in preparation—has not yet been presented. MCDI helps with transportation of gas bottles and vaccines from central level to health posts. Also, it is aiding with improved management of cold chain records through supervisory visits. Track and follow pregnant women (first child) through prenatal visits cards Pregnant women are identified and are followed by health post auxiliaries using the prenatal care card. There is yet no systematic way to identify pregnant women who do not come to the attention of the nurse auxiliary until house-to￾house home visits during NIDs are made. It is hoped that community health volunteers and the implementation of the community-based surveillance system will improve this activity next year. Purchase cards to supplement MOH supply as needed Not yet done Although the evaluation team is concerned about sustainability, the team feels that supplementing cards would be beneficial to create a “culture of cards” Train 48 health facility workers in EPI MOH completed the training for all personnel in EPI II in 2000 All but 8 MOH personnel in Cotagaita completed clinical IMCI training, which also includes EPI. (These 8 are currently being trained.) The MOH provides regular in-service training as well, but the team feels that there are still weaknesses at the nurse auxiliary level. A refresher course would be beneficial. Establish a referral system between health promoters and health facilities to encourage immunization Not yet done. Community health volunteers are still being selected and trained. Implementation of community-based surveillance in 2002 should help with referral Develop a BCC/IEC campaign for immunization Messages and materials for BCC/IEC still require refinement. Using PROSIN community IMCI materials. See BCC/IEC section below. There is only one flip-chart page on immunization. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 10 Planned activities Progress Comments Provide logistical support and manpower during NIDs MCDI provides transportation and manpower during campaigns (Cotagaita district only). Covers the most difficult areas of the district. MCDI has not been invited by the MOH to participate in planning of NIDs. There are no evaluation meetings after NIDs. Train health facility workers in vitamin A administration according to protocols All MOH personnel in Cotagaita district trained in IMCI except 8 (since the MTE, these 8 are in the process of being trained.) SEDES provided training in nutrition for all MOH personnel done 31 August to 2 September 2001, which included vitamin A administration protocols. All auxiliaries receive supervisory visits by MCDI which include supervision of vitamin A administration Note that the community IMCI materials and the course developed by PROSIN do not include vitamin A at all. The team feels that they will need to adapt existing materials (from MotherCare, PROSIN and CARE International) to include vitamin A. (Currently, the SBS Vitamin A clinical protocols are used in the health centers) The child health card does have a place to register vitamin A administration. Coordinate with SEDES for regular vitamin A supplements to facilities Review of medication system completed, report in preparation. There have reportedly been no serious shortages of vitamin A in the zone since the project began. EFFECTIVENESS Due to limited data available, it is difficult to clearly establish progress toward improving immunization coverage. The limited data from service statistics gathered through MCDI’s HIS and the SNIS reveals that the number of doses of vaccine administered in Cotagaita district remained approximately the same since the project’s inception: USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 11 Vaccine doses administered in children 0- 2 years old Year 2000 (12 months) January to November 2001 (11 months) DPT 3 510 458 Polio 3 506 458 DPT 1 503 452 Measles 506 457 Vitamin A 6-11 months 154 296 Vitamin A 1-2 years 1st dose 454 346 Vitamin A 1-2 years 2nd dose 405 401 *Source: 14 of 38 health posts in Cotagaita district currently receiving regular supervision by MCDI staff. Data collected by MCDI staff. Number of children 0-1 years in this area is estimated at 512. The SNIS gives the following results: Indicator (Cotagaita District) Year 1999 (=estimated coverage %) Jan-Nov 2001 (11 months)* Polio 1 < 1 year (doses) 1148 (87.3%) 923 Polio 3 < 1 year (doses) 1324 (98.8%) 1022 DPT 1 < 1 year (doses) 1148 (87.3%) 931 DPT 3 < 1 year (doses) 1324 (98.8%) 1038 Measles 12-23 months (doses) 1852 (143%) 1116 BCG < 1 year (doses) 1029 (77%) 943 TT 2nd dose N/A 1395 SOURCE: SNIS Cotagaita district *The reason only eleven months of data were available for the evaluation stemmed from problems accessing SNIS data. Further discussion of this and other related issues appears in the section below on Capacity Building of Local Partners. These data would indicate that immunization coverage is probably much higher than estimated by the KPC, and that it was already high in 2000. Also, the dropout rate for DPT is low. The number of second-doses of tetanus toxoid, now administered only to pregnant women, is adequate—actually higher than first doses of polio. MCDI’s HIS does not have comparable data for 1999, but coverage is not thought to be very different from these two years. The MCDI team (many of whom worked for the MOH prior to working for MCDI), the District and Municipal MOH staff, all nurse auxiliaries interviewed, and all mothers interviewed during the evaluation agree that immunization coverage is high. Nurse auxiliaries claim to have identified all the children in their respective areas; copies of immunization records are kept at health posts and they ensure that they are immunized. The only exceptions are children who move into the area between NIDs and do not come to the attention of the workers, and possibly, the few children who live outside the catchment area of any health post. A quick review of child health cards at health posts visited during the evaluation confirmed that all children with cards USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 12 are up to date. Other supporting evidence is that there have been no reported outbreaks of measles or pertussis since the beginning of the project, and no cases of polio. The coverage estimate obtained through the SNIS does not reach the expected level, however, it is higher than that indicated by the KPC. The apparent discrepancy between the CS baseline survey results, official coverage numbers, and the actual situation can be explained by several factors: The baseline survey included documented card coverage only. Only 42% of children 12-23 months of age reportedly had cards. Curiously, 45% reported having lost their card, far higher than what most other projects report. In addition, the team and health workers reported that, even when children have cards, health workers often do not record information on the mother’s copy, preferring to prioritize the copy kept in the health post. Auxiliaries expressed that this was due to time constraints. Finally, when cards are in short supply, health workers report that they give priority to the copy kept in the health post. Overestimation of the population size leads to underestimates of official coverage rates as calculated using the SNIS. There are three reasons for this overestimate: the most recent census occurred 10 years ago and at that time, many urban residents were reportedly encouraged to return to their villages to be counted. Moreover, there are financial incentives for local governments to overestimate their population, as budgetary resource allocations for local governments are based on population size. Finally, there has been much migration from the project zone to more prosperous urban areas in Bolivia and to Argentina. This was directly confirmed during visits to communities during the evaluation, where health-post maps of most communities designated anywhere from 50% to 80% of homes as “abandoned”, and health workers consistently reported that their “official” population was between 20% and 50% greater than the true population. Out-migration is especially common among younger adults, who are those most likely to have small children. All of these factors combine to under estimate true immunization coverage as calculated. A new census was performed in 2001 and once the results are revealed, more accurate estimates of denominators will be reported. The evaluation team concludes that immunization and vitamin A coverage are probably at acceptable levels in most of the project zone (and may have been higher than the coverage level generated by the KPC). However, this will need to be confirmed (see recommendations below). The cold chain is far from ideal, as many health posts lack working refrigerators and must pick up vaccines using coolers (MCDI is exploring the possibility of repairing these) This precludes daily immunization and many of these posts immunize either every two weeks or once a month. (One auxiliary nurse reported immunizing only every two months.) Nevertheless, all the nurse auxiliaries interviewed denied that this created problems for coverage, as mothers of children needing immunization were notified of immunization days in advance or received a home visit to complete required vaccines. All nurse auxiliaries reported that the logistical assistance provided by MCDI in transporting vaccines, coolers, and gas was important to maintaining coverage. There is reportedly little or no resistance to immunization by families. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 13 There have reportedly been no important vaccine or vitamin A stock-outs since the project began. However, stocks of cards for children and mothers are unreliable. Because activities in BCC/IEC have recently been implemented and some BCC/IEC materials are still being finalized it is not expected that there would be changes in knowledge yet. Recommendations: Two important recommendations resulted from this evaluation: The team should make every effort to verify true immunization coverage as soon as possible. This could be done rapidly by performing home visits during regular supervision visits, taking the health post’s copy of the cards to compare with the copy in the home. It should soon become apparent whether discrepancies in record keeping between the two copies are common or not. MCDI should go ahead with its plan to supplement maternal and child cards to guarantee their regular supply. As long as supplies are irregular, it is impossible to obtain good record keeping, and it will be impossible to create a “culture” of valuing the card among both health workers and caretakers. The cards empower caretakers and communities. The issue of reduced sustainability is less important than the benefits gained by improved monitoring, and helping families and health workers understand the cards’ value. MCDI can work to normalization MOH supplies in the meantime. Recommendation: The MCDI team should not forget to develop at least one message (and supporting visual aids) each regarding vitamin A and tetanus toxoid immunization to supplement the materials lacking in the existing community IMCI materials. In addition, a simple system for monitoring the information on copies of cards kept at health posts would be advantageous. The information could be collected from a random sample of cards during regular supervisory visits. CHANGES IN THE APPROACH SINCE THE DIP There have been no significant changes to the approach outlined in the DIP, except that the MOH has changed its tetanus toxoid immunization policy, targeting only pregnant women instead of all WRA. The DIP mentions a strategy to reach high-risk populations of isolated families through CHPs. As the project has not yet begun in the most isolated areas and the training of community health volunteers is still in progress, the effectiveness of this activity could not be assessed. (The strategy for using CHPs to provide immunizations is not embraced by the project). SPECIAL OUTCOMES, SUCCESSES, CONSTRAINTS The principal successes and constraints were described under the section above on evidence of impact. One constraint to all aspects of coordination has already been mentioned: the high turnover of MOH staff. With regard to immunization, the principal effect was to delay the drug management study. 2. Diarrhea The objectives for the diarrhea intervention are listed above. ACTIVITIES AND PROGRESS USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 14 Planned activities Progress Comments Train 48 health personnel in IMCI protocols for diarrheal disease* A total of 49 people have been trained, though of these, only 40 remain in the zone*. Eight are currently being trained. Train CHVs in community surveillance of diarrheal disease outbreak and referral of severe cases (HH/C￾IMCI). Training in community￾based surveillance has not begun. CHVs are only now beginning selection and technical training. SEDES has planned and budgeted for implementation of community-based epidemiologic surveillance for 2002. Details were not available at the time of the evaluation, but as it is MOH policy, it will be used instead of the system originally planned out by Núr and/or Save the Children. Ensure supply of ORS sachets to all health facilities and designated CHVs and Community-based organizations in coordination with SEDES. The study of drug management is complete, report in progress. There have been no significant shortages of ORS in Cotagaita district. MCDI providing logistical support as needed. When supplies are short, consider prioritizing health facilities equipped for plan “B” rather than distribution of packets to mothers Adapt and disseminate culturally appropriate BCC/IEC materials including non-literate graphic materials regarding danger signs of dehydration, and home case management (PROSIN/BASICS) Flip-chart materials from PROSIN project in hand. Baseline focus groups completed. Team has decided to supplement existing materials with laminated cards with more specific/culturally appropriate messages. See BCC/IEC section for further detail. Each auxiliary nurse will have a set of flipcharts and laminated sheets to use in training mothers. Nurse auxiliaries still need to be trained in community￾IMCI. Complete set of materials not yet available from Núr. Train health workers, CHVs, including TBAs in promotion of exclusive breastfeeding. 37 auxiliaries trained by CARE, 37 by MCDI. In addition, exclusive breastfeeding is included in clinical IMCI training (8 health workers are currently receiving training). HH/C￾IMCI includes material, TBAs are few and not very active. Focus has shifted away from them toward CHVs. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 15 Planned activities Progress Comments training begun for CHVs in 31 communities. Support and supervision to application of IMCI protocols for treatment of diarrhea in health facilities. All but one nurse auxiliary trained in IMCI*. 14 of 38 health posts receiving regular follow-up and supervision in IMCI by MCDI team. The team is currently using the IMCI supervision model developed by the MOH. The team is in the process of developing an integrated supervision system, to include IMCI/SBS protocols. Joint supervisions will be carried out with MOH personnel. *This activity repeats in each intervention. It should be noted that the numerical objective refers only to Cotagaita district. Objectives for the Puna district still must be set. EFFECTIVENESS The strongest evidence for effectiveness and progress toward achievement of goals is in the supervision of the 14 health posts in the application of IMCI protocols. In all of ten formal supervisions of nurse auxiliaries in the application of IMCI protocols, the nurse auxiliary applied the appropriate questions and protocols about diarrhea during the consult; in 70% of the supervisions, the nurse sought danger signs. All nurse auxiliaries interviewed expressed satisfaction with the IMCI strategy. They all stated that it is useful for improving the quality of the diagnoses, especially for children who are dehydrated and/or present danger signs related to illness. In 14 of 38 health posts Cotagaita district Year 2000 (12 months) Year 2001 (12 months) Cases of diarrhea in children < 2 year 1214 1334 Cases of diarrhea with dehydration in children < 2 years 69 94 Cases of diarrhea under 2 years treated with ORT 1157 1313 Packets of ORS distributed 2369 2567 Total days without ORS 46 77 *Source: MCDI HIS for 14/38 health posts in Cotagaita district. The SNIS counted 2870 cases of diarrhea in Cotagaita district in children under five during the year 1999, and 3788 cases during the first 11 months of 2001. The increase in cases was a result of an increased in the number of caretakers seeking care from health facilities because of coverage by the national health insurance system. There have been no outbreaks of cholera in the project zone during the life of the project. It is interesting to note that while the number of children under two who reported having diarrhea by MCDI’s HIS increased only 10% from one year to the next, the number identified as being USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 16 dehydrated rose 36%. This would seem to confirm that health workers are now identifying more cases of children with dehydration from among those with diarrhea. This is even more impressive considering that nurse auxiliaries completed their training in groups between April and November 2001, so a full year of impact could not yet be expected. District health officials indicated that rehydrating children using “plan B” in health posts is hindered by lack of equipment, including chairs, tables, cups, and pitchers. However, when nurse auxiliaries were questioned, they indicated no significant difficulties with this activity. A rapid assessment during supervisory visits would confirm this for all health posts. The cost of cups, spoons, and pitchers should not be prohibitive for the health system if these items need to be purchased. (MCDI will make them available as an in kind contribution, if required.) CHANGES IN APPROACH SINCE THE DIP The principal change from that outlined in the DIP is the decreased emphasis on training TBAs and the subtle change in definition from community health promoters to “madres vigilantes”, or CHVs. These changes are reflected in all the interventions. The former decision reflects the fact that it has become apparent that TBAs do not play a very significant role in the region. Deliveries are almost all carried out either by health workers or family members. According to reports, in the rare event that a TBA is called, they most commonly intervene briefly to solve an immediate problem and then leave. The increased coverage of the national health insurance program and better training and equipment of health facilities is rapidly increasing the percentage of deliveries being performed by trained health workers, either in the home or in facilities. This approach seems reasonable and appropriate. Identification and training of TBAs would likely absorb significant time and resources, which could be better utilized on other activities and strategies. MCDI’s shift in the definition of the community health worker from the “health promoter” to the “CHV” is more subtle, and is discussed in greater detail in the section on Community Mobilization below. Suffice it say that it represents more a refinement and individualization of selection criteria and the selection process for the community worker than a complete shift of focus or methodology. A likely change that has not yet occurred is the implementation of a community-based surveillance system. The DIP anticipated the implementation of a system as designed by Save the Children and/or Núr University. MCDI has not yet begun to implement either of these. During the mid-term evaluation MCDI became aware that SEDES is planning to implement a community-based surveillance system of its own, and has asked for community volunteers to be trained. It is known that the Cotagaita district plans to train 138 community members plus 44 health workers from the District during 2002. Details beyond this are unknown, but it would be logical for MCDI to support the implementation of this initiative. SPECIAL OUTCOMES, SUCCESSES, CONSTRAINTS As previously mentioned, the materials expected from PROSIN/Núr for training health workers in community-based IMCI are not complete. This has obliged the MCDI team to assume the adaptation of some materials based on existing IEC materials from MotherCare and MNH Project and from PROSIN which has led to a delay in training auxiliary nurses in HH/C-IMCI. The project anticipates working with the CORE Group and PROCOSI HH/C-IMCI initiative, USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 17 which will contribute to standardization of training materials. (MCDI is contracting a consultant to assist in finalizing these HH/C-IMCI training materials.) See the section on BCC/IEC below section for further details. (A collaboration with the MNH Project and PLAN is also anticipated.) The team also expressed the constraint of long distances between some communities and health facilities, which sometimes is five hours or more on foot. Recommendation: The team should keep in mind that, in the event of intermittent short supply of ORS, priority should go to health facilities equipped to rehydrate dehydrated children (plan B) rather than distribution to mothers to use to prevent dehydration (plan A). As the first step to prevent complications and to overcome ORS shortages the team should continue educating CHVs in the use of home-based rehydration liquids. The team also expressed a second constraint to successful implementation, which has been the difficulty with the monitoring system. They feel this system requires an inordinate amount of time for data collection. (See the section on Information Management at the end of the evaluation for further details and suggestions.) Next steps include: · Complete CHV training, finalize materials and train nurse auxiliaries in HH/C-IMCI so they can continue training and begin supervising the community-based workers · intensify supervision of clinical IMCI at all health facilities · finalize IEC materials and disseminate the messages 3. Pneumonia Many of the points made for the diarrhea intervention above also apply to the pneumonia intervention. The objectives for this intervention appear at the beginning of this report. ACTIVITIES AND PROGRESS Planned activities Progress Comments Adapt, finalize and disseminate BCC/IEC messages on pneumonia and treatment seeking for caretakers and the community Baseline qualitative studies have been completed. Flip chart pages based on PROSIN HH/C-IMCI course are already in use for training. Supplementary laminated drawings are being developed and are still being tested and reproduced. Comprehensive BCC/IEC activities have not yet begun as community workers are still in training Educate and sensitize TBAs, traditional healers, and community groups on recognition of danger signs and timely referrals related to pneumonia case This activity is being implemented in women’s groups only and is combined with selection and training of CHVs. See section on diarrheal disease for comments on TBAs. The project has not yet worked to identify traditional healers. This issue is still under USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 18 Planned activities Progress Comments management at community level discussion. Train CHVs in HH-C/IMCI concepts for pneumonia and acute respiratory infections. Training has begun for CHVs in 31 communities. None have reached the module on pneumonia. Support training for 100% of health personnel in IMCI protocols for pneumonia case management and communications skills. A total of 49 people have been trained, though of these, only 40 remain in the zone*. Eight are currently being trained. As part of the IMCI clinical training, all health workers receive training in interpersonal communication and counseling. Support the District MOH in organizing appropriate logistical mechanisms for drugs and medical supplies. Study on drug management completed. Report in preparation. As-needed logistical support underway. Support for supervision of management of supplies at the health post level occurs during supervisory visits. There have been no significant or prolonged shortages of pneumonia medications at health facilities since the project began. Train CHVs in community surveillance of pneumonia cases (HH-C/IMCI). Training in community￾based surveillance has not yet begun. See sections on diarrhea and on community mobilization for further comments. *Numerical objective set only for Cotagaita district. This will have to be repeated with all health professionals in Puna district in the second half of the project. EFFECTIVENESS Similar to the diarrhea intervention, the greatest evidence of impact to date is at the health facility level. Nurse auxiliaries are routinely asking about respiratory symptoms, consistently measuring temperature and measuring respiratory rates using respiratory timers, applying age￾appropriate cut-offs, correctly classifying disease severity, and applying appropriate treatment and/or referral. This was directly observed during the evaluation, and is evident on supervision forms. Nurse auxiliaries themselves specifically cited their improved ability to diagnose pneumonia as important in improving the quality of care being provided. They feel that more cases of pneumonia are being correctly diagnosed and treated than before. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 19 In 14 of 38 health posts Cotagaita district Year 2000 (12 months) Year 2001 (12 months) Cases of ARI treated in children < 2 years 1828 2109 Number of cases of ARI diagnosed as moderate or severe in children < 2 years 161 214 Number of cases under 2 months 3 0 Total days without appropriate medication 75 30 *Source: MCDI HIS for 14/38 health posts in Cotagaita district. The SNIS reported a total of 219 cases of pneumonia in children under five years of age treated in 1999 in Cotagaita district, and 988 cases from January to November 2001. As with the diarrhea intervention, the data would indicate that cases of pneumonia are being detected more frequently than before. In the 14 health posts, the total number of respiratory infection cases rose 15% from 2000 to 2001, but those recognized as pneumonia rose 33%. From 1999 to 2001, the number of reported cases of pneumonia in all of Cotagaita in children under five rose an astounding 350%. As with the previous intervention, this increase is even more impressive considering that nurse auxiliaries completed their training in groups between April and November 2001, so a full year of impact could not yet be expected. It is too early to expect improvements in community and caretaker knowledge compared with the baseline. CHANGES IN APPROACH SINCE THE DIP As with the diarrhea intervention, TBAs have been de-emphasized. Also, the separate objectives of training community groups and training community health volunteers have been combined into a single activity. The MCDI team is working training women’s groups in 31 communities from which the CHVs are expected to emerge. The other significant change regards training in community-based surveillance, as mentioned in the diarrhea section. This is further described below in the section on Community Mobilization. SPECIAL OUTCOMES, SUCCESSES, CONSTRAINTS The same constraints that apply to the other interventions apply here as well. The health district restricted access to information for a brief time (see section on capacity-building of local partners), and access to transportation for supervision and training is inadequate. Seven of 38 health post radios do not function for emergency referral, which is particularly important for the pneumonia and maternal care interventions. MCDI is exploring the possibility of repairing these radios. There was no specific provision in the CS project budget or the project work plan to remedy this situation, but it is hoped that better management of the national insurance fund will bring in resources for infrastructure (again, see Capacity-Building section below). USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 20 One successful aspect of the project has produced a constraint as well. As auxiliary nurses spend more time performing home visits, training and supervising community health volunteers and community groups, and going to training courses, they are spending less time in their health posts. Because of this, caretakers run the risk of not being able to receive treatment once they arrive at the facility. Recommendation: MCDI should work with the auxiliary nurses to develop these schedules for two reasons: 1) to maximize the effectiveness of their outreach activities and 2) to minimize the negative impact of their absence from the health posts. Another constraint felt by the team is that the SNIS does not include indicators that are precise enough to adequately monitor progress toward project objectives. MCDI has designed and implemented a parallel set of indicators that are being collected that are specific to the project. This system draws on information from the SNIS and complements this data with information salient to the under 2 population. There is an ongoing effort to further refine this system. A further constraint relates to limitations in the national health insurance program. Although the program covers the cost of treatment of the child as well as transportation to a referral facility if necessary, many barriers remain for successful implementation. The insurance will not cover the cost of the return trip home, nor is there any provision for lodging or food for a parent or caretaker. In addition, the staff expressed that there is reluctance of mothers to leave their husbands and other children at home to travel to a referral facility. Finally, reports indicate that patients referred to Tupiza for hospital care are sometimes turned away or treated rudely, as staff are reportedly sometimes unwilling to incur the cost of treatment for those from outside their immediate catchment area, in spite of the MOH official referral policy. (It is interesting that the same has not been reported on referral to Potosí.) All three of these problems have reportedly led some mothers to refuse referral when it is indicated, preferring instead to return home in spite of the risk to the child. This same constraint occurs for all interventions, but affects the pneumonia and maternal care interventions most acutely. The first two problems can be resolved by increasing community awareness and seeking community-based solutions, for example, encouraging the establishment of a community fund for emergency situations. MCDI could help ameliorate the last problem by documenting cases and discussing solutions with SEDES in Potosí. In addition, the plan to upgrade the Cotagaita hospital to a level two facility, complete with pediatrics, obstetrics and gynecology, surgery and anesthesia should also greatly diminish the need for referral to Tupiza. Important next steps: · improve and implement joint supervision with districts and municipalities · finish training in IMCI for the remaining eight health workers (these 8 are currently being trained) · complete IEC plan and adapt materials and implement plan as soon as possible · complete training plan and materials for HH/C-IMCI for health workers · support the training and implementation in community-based surveillance, especially with regard to selection of candidates for training USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 21 · implement the plan to improve medication management based on the results of the drug supply study 4. Maternal and Newborn Care The maternal and newborn care intervention is complex and will require significant coordination with the MOH personnel at all levels if it is to be completely successful. The objectives are listed above. To carry out this component of the project, MCDI is working in the context of the MOH's National Maternal and Neonatal Health Plan which encompasses the MNH strategies to: 1) strengthen community networks, 2) strengthen service networks and 3) build institutional capacity. Furthermore, MCDI has been maintaining regular contacts with the PROSIN community health division (DIDESCO) in Potosi. This division has the responsibility of coordinating the implementation of IMCI and maternal and newborn care activities in the districts, which include support to health facility rehabilitation and provision of equipment. MCDI has recently pursued contacts with CIELO (local strategic intervention component), the MOH's project, which focuses on reducing maternal and neonatal mortality in 17 health districts in Bolivia. A key partner in this activity, CARE, has provided training in humanized delivery and family planning for all staff in Cotagaita district (34 nurse auxiliaries) as well as the MCDI team in a two-day workshop held in March 2001 in Cotagaita. Because the CARE project has ended, MCDI is now providing follow-up supervision and support in applying the protocols in health posts. The planned relationship with the MNH project described in the DIP has been delayed because the MNH project was restructured and does not include either Cotagaita or Puna districts as their priority intervention areas. Nevertheless, MCDI is in contact with the MNH project and will utilize the experiences, methodologies and materials developed by MNH and MotherCare projects. MCDI will also keep abreast of the MNH support given to the MOH for improving the maternal mortality surveillance system, which has been implemented in Bolivia through the SNIS and the civil registry. MCDI will also keep abreast of PROSIN and CIELO's technical protocols. MCDI and the College of Nurse Midwives submitted a proposal to USAID Bolivia to secure funds to provide this training (however, funding was not approved). (Recently, MNH and the MOH have agreed to provide technical assistance and materials to the project by supporting TOT training in Tupiza and the project areas.) ACTIVITIES AND PROGRESS Planned activities Progress Comments Provide refresher training for 100% of health personnel on management of maternal and newborn health, including prenatal care, delivery and postpartum, high-risk pregnancy and emergency obstetric care. All Cotagaita staff and MCDI staff trained by CARE in family planning and “humanized delivery” (48 people trained). Routine monitoring of prenatal visit registration in 38 project health posts. Regular supervision of humanized delivery by MCDI staff in place in 13 The CARE course did not include refresher training in obstetrical emergencies or danger signs. It appears that the MNH project and the MOH will now be able to provide support to the TOT training. The MCDI team will have to collaborate with SEDES and USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 22 Planned activities Progress Comments health posts. Refresher training in technical aspects of delivery, EOC, prenatal care and neonatal care still need to be done. the districts to design and implement this refresher training. Adapt, finalize and disseminate BCC/IEC materials for MNC. Baseline qualitative information collected and key messages have been elaborated. The team is in the final stages of adaptingand finalizinglaminated pages for use in communities by auxiliaries. UNICEF's municipality/ family program has some tested materials relevant to MNC. The team is adopting existing materials from MNH and MotherCare Projects. See BCC/IEC section for more information. Train CHVs and TBAs in message dissemination and behavior change activities regarding early breastfeeding and the use of family planning including LAM. This information is to be included in the training being carried out to community women’s groups. The materials on maternal and newborn care are still being developed and tested. 31 women’s groups (potential CHVs) have begun training. None have reached these modules yet. Work with TBAs has been de-emphasized. Train CHVs and TBAs in message dissemination and behavior change activities for MNC. See above Work with TBAs has been de-emphasized. Train 30 TBAs on safe delivery, recognition of danger signs, identification of intrapartum and postpartum emergencies, essential EOC, IPCC and prompt referrals Work with TBAs has been de-emphasized in favor of strengthening communities and nurse auxiliaries. Work with TBAs has been de-emphasized. This objective will be eliminated. Introduce or adapt the notion of "birth preparedness" among TBAs, family birth attendants and CHVs (Madre Vigilante etc.). As above for MNH. Birth￾preparedness will be included in training of women’s groups and CHVs. The materials are still being developed. The DIP mentions application of the methods developed by Save the Children’s WARMI project. Given staff limitations and the size of the project zone, the full application of this USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 23 Planned activities Progress Comments methodology is probably not feasible. Train and equip health posts at primary care level for management of obstetrical emergencies. Baseline HFA complete and inventory of necessary equipment done. MCDI plans to work with municipalities to prioritize equipment purchases with funds from national insurance for upgrading facilities. It was anticipated that this training would occur through the MNH project. Restructuring of the MNH project has created delays in the process, but MCDI has maintained contacts with MNH. The MNH and MOH will be able to provide assistance on TOT training to the CS project. The trained nurse auxiliaries in 14 of the 38 health posts in Cotagaita district are receiving regular support and supervision in prenatal care, orientation about care to be taken during the pregnancy, post-partum visits, and administration of vitamin A to women who have given birth. Also, supervision focuses on utilizing maternal charts to follow-up on pregnant women and organizing clinical histories of pregnancies. UNICEF/Ministry of Human Development’s “Municipality/Family” program may be particularly valuable in the community training and BCC/IEC components of this intervention, because it offers validated community-based material in Quechua on maternal/neonatal health. The maternal/neonatal health component of the program includes messages on risks during pregnancy, delivery, post-delivery, breastfeeding, and prenatal care. The program includes a range of audio-visual support material, including videos and audiotapes. Although the Municipality/Family program currently being implemented by the District is unlikely to have a sustained effect, a more considered application with nurse auxiliaries, CHVs, mothers and communities may be more effective. One advantage of the program is that it leaves written material with each family. Due to the large number of complex messages and danger signs in this intervention, such memory aids are likely to be important if families and communities are to remember all the information. Recommendation: MCDI should carefully review and evaluate the feasibility of adapting the Municipality/Family educational material for training CHVs as well as for BCC/IEC activities. EFFECTIVENESS In 14 of 38 health posts Year 2000 Year 2001 Cotagaita district Number As % of total deliveries reported Number As % of total deliveries reported USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 24 In 14 of 38 health posts Year 2000 Year 2001 Cotagaita district Number As % of total deliveries reported Number As % of total deliveries reported New prenatal care consults before the 5th month 310 287 New prenatal care consults after the 5th month 248 218 Number of pregnant women with 4 prenatal care visits 214 307 Number receiving ferrous sulfate 426 474 Total deliveries reported 362 453 Institutional deliveries 153 42% 210 46% Home deliveries by health facility workers 116 32% 133 29% Delivery by TBA 51 14% 77 17% Delivery by untrained person 42 12% 33 7% New post-partum consults 262 72% 304 67% Source: MCDI’s HIS in 14 of 38 Cotagaita district health posts based on SNIS data Some interesting trends can be extrapolated from the existing SNIS data. Out of the total number of women who reported initiating prenatal care, 48% of deliveries were performed by MOH health providers (either in the home or at a health facility) in 2000 and 68% were performed by MOH health workers in 2001, notably, after they received training in humanized delivery. According to the baseline KPC, 80% of deliveries were taking place at home and 65% of these deliveries were attended by an untrained person (husbands, relatives and TBAs). The data above shows that there have been some improvements in deliveries by a trained person at home or at an institution. During meetings held during the evaluation, mothers repeatedly expressed their increased satisfaction with delivery in the health post, where they felt it was cleaner and safer than home delivery. They also revealed that after the humanized delivery training, mothers could deliver as they wished (squatting) rather than lying down on a gynecological table. According to the SNIS, the following totals were observed for Cotagaita district: Indicator 1999 (12 months) Jan-Nov 2001 (11 months) New prenatal consults prior to 5th month 529 521 New prenatal consults after the 5th month 961* 430 Pregnant women with 4 or more prenatal care consults 465 485 Deliveries in facilities 287 217 Home deliveries by health facility workers 215 239 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 25 Deliveries by TBAs or family members 106 131 New post-partum consults 640 324 *This number does not follow the pattern of all the other data indicating that the number of new consults before and after the fifth month are similar, and that MCDI’s immunization data for the 14/38 health posts are approximately 40-50% of that for the entire district. The same analysis carried out on the data above show that compared with the number of women initiating prenatal care, in 1999 34% had deliveries by health personnel either at home or a facility, and in 2001, 48% did so. During the evaluation, mothers expressed their increasing satisfaction with institutional deliveries and a trend away from using TBAs or even family members. Nurse auxiliaries, even those who are male, seem to be well accepted and respected in their communities. It may be surprising that prenatal care coverage is as high as it is in the zone, given the logistical obstacles and distances involved. Examination of records in individual health posts and interviews with health workers and mothers confirmed the pattern seen in the data: 70% or more of pregnant women seek care at least once during pregnancy, in contrast to the baseline KPC which revealed that only 17% of women had a prenatal visit based on their maternal card. This is important, as the first visit accrues the greatest benefit, with a rapidly falling rate of return on subsequent visits. At the time of the baseline KPC, the number of pregnant women seeking prenatal visits was low. Thus, this indicates a high degree of acceptance of health services being provided by nurse auxiliaries and other health providers, as well as increased knowledge of mothers regarding the importance of seeking prenatal care. This is a positive achievement and the project can apply this approach to other interventions as well. This serves as part of the basis for the recommendation that the nurse auxiliaries should play a role in transmission of key messages as part of the BCC/IEC activities. CHANGES IN APPROACH SINCE THE DIP The shift away from TBAs seems warranted based on the observation that they are not very active in the area. At least trained personnel are now performing half of the deliveries, and there appears to be a trend toward even greater access. MCDI correctly also cites international literature on the relative ineffectiveness of TBA training in decreasing maternal neonatal and mortality. Recommendation: De-emphasizing TBAs seems a reasonable decision Nevertheless, where TBAs are active and accepted by mothers, MCDI could explore the possibility of training them in HH/C-IMCI including the MNC component. According to field staff, JHPIEGO’s MNH project is being implemented in Uyuni, in the western Potosí altiplano, far from Cotagaita District and will not be implemented in Potosí Department as originally anticipated. However, as previously mentioned, MNH and the MOH have recently agreed to provide technical assistance for TOT training on emergency obstetrical care. This will decrease the burden of the MCDI team to develop materials and provide refresher training for MOH staff and health post staff in technical areas such as the management of obstetrical and neonatal emergencies, a subject that was not covered in the course offered by CARE. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 26 Recommendation: MCDI should move quickly to coordinate with MNH and the MOH to arrange plans and schedules for technical training of a TOT team in obstetrical and neonatal emergencies. The Municipality of Vitichi included funds in its 2002 budget to support reproductive health training for health workers with PROSIN. MCDI should urgently investigate this possibility as well. SPECIAL OUTCOMES, SUCCESSES, CONSTRAINTS The two constraints relating to problems with referral discussed in the pneumonia section above also apply to the maternal/neonatal care intervention. Briefly, seven of 38 health post radios do not function well, making it impossible to summon the ambulance for emergencies. In addition, mothers are often reluctant to seek referrals due to family problems, costs not covered by the national health insurance scheme (return transportation and costs for those who accompany her), and reports of occasional mistreatment of patients referred to Tupiza. On a positive note, the MOH plans to upgrade the Cotagaita hospital to a secondary level facility, capable of performing surgery. This will greatly reduce the need to refer mothers to Tupiza and Potosí for care. Secondly, the rapid spread of electricity and telephones in the project zone will also be beneficial. Even many very isolated communities have a single working solar telephone line, which greatly facilitates referral. Nurse auxiliaries in health posts confirm that they are aware of each pregnant woman in their respective catchment areas (as previously noted, there is no systematic way of identifying pregnant women who do not come to the nurse auxiliaries until home visits are made.) MCDI expects that the implementation of the community-based surveillance system will improve this activity by next year. This should allow for very closely targeted birth-preparation and follow￾up. Also, it reinforces the shift in project emphasis away from TBAs and perhaps, even, from family members, toward a policy that encourages institutional delivery whenever possible (or, at least, home delivery by a health worker). Recommendation: In all but the most inaccessible areas, MCDI should target messages and training toward encouragement of deliveries by MOH health personnel, at a facility if possible, or in the home. MCDI may need to tailor its community MNC training and BCC/IEC to the situation/problem of each individual community, while encouraging institutional deliveries in the majority of communities who have reasonable access to facilities. In communities where access is limited, community awareness and birth preparedness should be stressed. Next steps include: · completing training of health facility personnel in obstetrical and neonatal emergencies · finalize materials and training of CHVs in MNC messages and delegate this task to nurse auxiliaries · finalize the development of the instrument for systematic supervision of health facility personnel in MNC and work toward integrated supervision with MOH supervisors wherever possible 5. IMCI USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 27 IMCI is more a strategy than an intervention, and most points regarding IMCI have either been discussed in the individual interventions or are covered below, either in the capacity-building or BCC/IEC sections. Nevertheless, a few comments regarding IMCI as it is being implemented in this project would be appropriate at this time The project is using the tested institutional/clinical IMCI in its efforts to strengthen the quality of services, as well as a version of the newer and a more experimental community-IMCI as part of its behavior change strategy. As most IMCI curricula do not address maternal/newborn care issues in depth, MNC is being added through materials and methods drawn from alternate sources. ACTIVITIES AND PROGRESS Clinical IMCI Clinical IMCI was begun early in the project, as the training center in Potosí was already operational when the project began. The center in Tupiza lacked trainers, however. In October 2000, MCDI supported the six-day training of eight people as IMCI facilitators, including four from the Tupiza district, three from Cotagaita district and one from MCDI. CARE supported two others from Villazón district. This allowed the Tupiza IMCI training center to become fully operational, which in turn facilitated training of personnel from Cotagaita. A total of 48 health workers have since been trained in the six-day course, either in Tupiza or Potosí, including all but eight current Cotagaita district health professionals (who are currently being trained). All but one nurse auxiliary has been trained, and almost all Area physicians have been trained. Those who remain were new hires in the last quarter. Material in use was that developed by the bilateral USAID-funded PROSIN project, which drew upon technical assistance from BASICS. Although the recommendation in the DIP review was to teach the full eleven-day course to all health professionals, SEDES insisted on the abbreviated course. A second course in monitoring and supervision of clinical IMCI was offered to facilitators from the Tupiza center, four Cotagaita district staff and two from MCDI in March 2001. Systematic supervision of the implementation of clinical IMCI has not yet begun in earnest even though health workers finished their training in April, May, July, and November 2001. (The reason for the limited MOH supervision is cited elsewhere). The MCDI team has received support from the Cotagaita district to provide follow-up visits to nurse auxiliaries who received IMCI training, and the team has tested the simple IMCI supervision instrument that was presented at the supervision seminar. The instrument has only been used ten times thus far, however, the team is comfortable with its use, and tabulation of the results was done quickly for the mid-term evaluation. Informal supervision of IMCI has begun in 14 of the district’s 38 health posts during monthly supervision visits. District and Municipal MOH personnel have not been involved in this supervision, and no supervision has been done for physicians and university level nurses. It is not entirely clear why formal supervision has been delayed, but it may be partly due to the very high turnover of District and Municipal staff as well as the tremendous effort MCDI has put into community organization and training activities in recent months. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 28 One potential problem noted during the recent supervision visits is that it is unclear how consistently IMCI is used during consults. There were a total of 67 IMCI forms filled out in the health facilities that were supervised, NHI forms reported a total of 341 consults by children in those health posts for the same period. It is possible that IMCI is not being applied consistently. This can be verified once systematic supervision is initiated. Recommendation: In order to not jeopardize the valuable investment in IMCI training to date, MCDI should begin applying the systematic IMCI supervision tool to all trained personnel in all facilities. The team should also strive to engage MOH personnel at the Muncipal and/or District levels whenever possible to participate in the supervision. The recent suggestion by MCDI to encourage the municipalities to support a full-time municipal nurse supervisor with their own resources seems like a reasonable approach. Recommendation: MCDI should encourage the final training of the remaining eight health workers in Cotagaita district in clinical IMCI as soon as possible. (These 8 health workers are in the process of being trained.) Home Health/Community IMCI Community IMCI is still a work-in-progress. In Bolivia, materials developed by PROSIN are being tested by Núr University in a pilot project near Cochabamba. MCDI supported training in HH/C-IMCI for four members of its own staff, one MOH staff member, and one IMCI facilitator from Tupiza. They traveled to Cochabamba for a six-day training course in the pilot project’s methodology. Although Núr was able to carry out the training, and the materials for community use are available, Núr is not yet ready to release materials and manuals for replication of the training course for facilitators. Núr has informed MCDI that these materials should be available in early 2002. Materials for community use are based primarily on a 29-page flip-chart which includes the following topics: · how to do a home visit · danger signs for imminent child death and how to act accordingly; · ear care · home hygiene · keeping animals outside the home · management and prevention of pests · cough (serious and not serious) · diarrhea (serious and not serious), danger signs, parasites, and prevention (food, excreta, garbage) · fever · malaria (not being implemented in the project) · vaccines (by age, type, when to refer) · love and attention · feeding (by age groups) USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 29 The earliest community training began in mid-2001 with MCDI staff providing one-hour sessions on a monthly basis to women’s groups in 31 key villages. A total of about 560 women are currently participating. All communities have completed the modules on introduction to IMCI and immunization. Almost all have completed the modules on prevention, love and attention, and breastfeeding; about half of the groups have completed the module on complementary feeding. Two groups have completed the module on hygiene. Training for the women’s groups is participatory and dynamic, but there is yet no formal method in place for evaluating an increase in knowledge or the quality of the sessions. MCDI has developed an instrument to measure the quality of group education sessions and is about to begin field-testing. The instrument uses direct observation of training sessions to assess teaching methodology, completeness, and accuracy. The team expects to be able to use it to evaluate each other’s performance as well as the performance of auxiliary nurses once community-level training has been delegated to them. The HH/C-IMCI materials have been developed and field-tested with real community group, and MCDI has developed a draft training plan for HH/C-IMCI for use at the community level. They have developed another written curriculum for a six-day course to train health workers (especially nurse auxiliaries) as facilitators. These training plans have remained restricted to topics contained in Núr’s HH/C-IMCI course only, and have not been expanded to include subjects relating to the maternal and newborn care intervention. The evaluator considered that the Nur materials were too broad in their focus and diluted the key messages intended to answer to most important IEC needs of the project area. Hopefully, with assistance from the external consultant, utilization of the BEHAVE framework, and results from the gap analysis, MCDI can focus and prioritize the key IEC messages relevant to the project needs, including MNC. Recommendation: As the project does not have the luxury of waiting any longer for Núr University to finalize and release the materials, MCDI should work with PROCOSI/CORE HH/C-IMCI initiative to complete the development of the HH/C-IMCI course for facilitators and health workers and carry it out as soon as possible. These steps should be taken so that the project can move forward and be scaled up to achieve coverage. (MCDI has already begun to address this limitation by contracting a consultant who will assist the project team in finalizing this curriculum.) Recommendation: It would seem reasonable to integrate maternal/newborn topics into the course for community IMCI (both the courses for heath workers and community members) rather than trying to organize a separate training course. Recommendation: Both the team and mothers agree that the training of mothers’ groups and CHVs will be more effective if some sort of material can be left behind in the community. The project is expecting CHVs and caretakers to memorize a long list of danger signs, yet no memory aids have yet been developed for use by CHVs and/or mothers. One good start would be to look carefully at the Municipality/Family materials, which are already present in communities. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 30 EFFECTIVENESS The effectiveness of clinical IMCI will be seen in improved quality of services as measured by coverage for the various interventions. As mentioned for the pneumonia and diarrhea interventions above, nurse auxiliaries and Municipal and District personnel feel that children are being diagnosed and treated more accurately since IMCI was introduced, and there are early indications from the HIS that this is true. The effectiveness of community IMCI will be measured through behavior changes as measured in the final KPC survey and the BCC/IEC monitoring system. SPECIAL OUTCOMES, SUCCESSES, CONSTRAINTS In attempt to prioritize key BCC activities, a gap analysis, which focused on the 16 Key Family Practices, was carried out based on the focus group discussion results. A first set of prioritized messages was prepared. Although some projects have found clinical IMCI to be too time consuming to implement, this does not seem to be the case for this project. This is an issue most frequently heard from health workers in urban programs where health facilities are overburdened. In rural programs, such as MCDI’s project, health workers often see no more than ten patients each day and have time for the more lengthy process of a full IMCI consult. Another constraint is the cost of reproducing IMCI forms. The Cotagaita district alleges lack of funds for their reproduction, and MCDI will now be obligated to bear this cost. MCDI plans to negotiate with the Municipalities to assume this cost using national health insurance reimbursements. Another small problem is that there was no formal systematic baseline assessment of health worker performance in the immunizations, pneumonia and diarrhea interventions. It is therefore more difficult to attribute skills to the training they received. Qualitative information indicates that their performance did indeed improve. Recommendation: A brief baseline assessment of the degree of application of appropriate IMCI and MNC protocols by health workers in health facilities before training health workers in the expanded zone would provide better documentation of the project’s direct impact on the quality of services. (This should be done in collaboration with the MOH). The team expressed some reservations about adopting the methodology of the HH/C-IMCI component exactly as Núr University is testing it in Cochabamba. That project has a far larger staff than the MCDI team (eight full-time trainers) working in only 44 communities. The communities are easily accessible by public transportation from Cochabamba, mostly on paved roads, and it is easy for team members to reach all communities and return to Cochabamba in a morning or afternoon. Finally, that project is implementing only HH/C-IMCI. There is no effort to train MOH staff in its implementation or supervision, nor to improve quality of health services or strengthen partners. MCDI staff that attended the course in Cochabamba said that the Núr staff expressed grave reservations about the small MCDI team’s ability to scale the HH/C-IMCI activities up to cover 400 widely scattered rural villages. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 31 Recommendation: MCDI needs to finalize the HH/C-IMCI materials initially developed by Nur/PROSIN and should be done through technical assistance because the project cannot wait for Nur to finalize this. The evaluation team was unanimous in their feeling that it will be possible for the Núr/PROSIN HH/C-IMCI methodology to be scaled up and made more cost￾effective by adapting the methodology to the reality and constraints of the region and the MCDI/CS project. This will involve immediately training nurse auxiliaries as HH/C-IMCI facilitators and delegating training responsibilities to them, while simultaneously developing and implementing an intensive system of field supervision, which will allow the team to support this change. In this way, CHVs and community groups in many communities can be trained simultaneously with minimal degradation in quality. 2. CROSS-CUTTING APPROACHES a. Community Mobilization This section discusses some issues regarding community organization and mobilization, many of which overlap with topics discussed partially in other sections. The following activities best fall into the category of “community mobilization” as separate from HH/C-IMCI, training, or BCC/IEC. Community organization and formation of health committees and women’s groups MCDI evaluated the 222 communities in Cotagaita district using systematic criteria, and chose 140 communities for priority intervention based on their size, accessibility and existing level of organization to the extent to which outreach activities were being carried out by auxiliary nurses. Using this strategy the team should be able to provide coverage of project activities to well over 90% of the population while minimizing stress on limited project resources. Community-based project activities will be phased-in, in groups of 24 communities, with the current operational plan to completing the selection and training of CHVs in the 140 priority communities in Cotagaita district before the end of the project. The team has also selected 31 (out of 140) of the highest priority communities, where direct intervention has begun. Selection was based on risk (low-coverage) as determined by the SNIS during TAIs. MCDI team members held collective meetings with community leaders and the communities to provide information on project goals, objectives and the work plan to encourage their active participation in supporting health-related and project-related activities. The response to these activities to date has reportedly been very positive. Communities in the project zone are generally well organized and participatory by nature, though there is wide variation in population size and degree of participation. In the 31 initial communities, many were already able to constitute health committees, which meet regularly to discuss health-related problems and solutions. In addition, it has been possible to form or strengthen existing women’s groups in all 31 villages, and in all cases, the groups continue to meet in the absence of MCDI team members. Most are involved in income-generation activities, most commonly weaving, sewing and embroidery of traditional clothing. Meetings with women’s groups during the evaluation showed them to be participatory and independent. They expressed gratitude for MCDI’s assistance in organizing their activities, although the team only provides training in HH/C-IMCI once each month. Groups exist at USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 32 different levels of organization; some seeking to attain formal legal status as organizations, others looking to build their own headquarters, others looking for markets for their products. It is clear that the level of participation is high. In the majority of cases, the women’s group counts on the participation of all of the women in a given community, and therefore offers an excellent opportunity for community participation as well as BCC/IEC activities. MCDI is the first organization to have worked at the community level in most of these villages, and therefore the negative effects of previous programs focused on “give-aways”, so commonly encountered in many zones, are largely absent. The change in focus from training CHPs toward working with women’s groups and training CHVs has important implications for the project, however. Whereas the former were expected to be rapidly trained in groups, the new methodology requires a much greater effort from the MCDI team than originally planned. This is part of the reason for expanding the technical team from the original two members to five. Even with this expansion, the MCDI team is currently working directly in communities representing only about 20% of Cotagaita district and about eight percent of the entire CS project zone. In order to scale the project up, MCDI must delegate of activities to auxiliary nurses, with MCDI staff, and Municipal and/or District staff shifting to a supervisory role. Recommendation: The MCDI team should develop some simple criteria for measuring the level of community organization that they wish to achieve and include these goals and methodologies in the training course of HH/C-IMCI for nurse auxiliaries. Minimum criteria may include the formation of an active women’s group and health committee in each community. In addition, as it will be increasingly difficult for the team to informally monitor the success of community organization and mobilization activities as the project scales up to reach more villages, development of simple objective criteria for measuring the level of community participation and organization would be useful. In the future these could be mapped into the planned GIS if MCDI so desired. In this way, team and Municipal/District can target their supervision to those communities requiring the most attention. Recommendation: As the establishment of health committees appears to be a strategy that the team is interested in pursuing, MCDI should also work with those communities with the most active and organized committees (such as Tocla). MCDI can draw on their experience and jointly develop a description of specific responsibilities and activities for health committees so these can be included in training for auxiliary nurses. Decisions should be made about the committee’s composition, whether health committees will file reports, indicators on those reports, and to whom they will file them. This must be done as soon as possible so auxiliary nurses can be trained and community activities can be delegated to them. Selection, training and supervision of CHV’s The original strategy for obtaining a group of active community health workers involved identifying existing CHPs, the selection of new CHPs, and training 140 of them in community mobilization, HH/C-IMCI, BCC/IEC, and community-based epidemiologic surveillance. The USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 33 CHPs would, in turn, educate communities and caretakers through home visits and training during community group meetings, among other activities. After the project began, MCDI has become increasingly aware of the long and less-than￾successful history in Bolivia with volunteer community health workers in many different guises. Many CHPs were male, many were young, many were chosen for political reasons rather than being natural leaders, and their effectiveness was limited. This observation is echoed by comments from nurse auxiliaries and women’s groups themselves. MCDI was disappointed to find that there were far fewer active and trained CHPs than anticipated. In addition, two early attempts to select and train Health Promoters (or “RPS”) were evaluated by the MCDI team as unsuccessful; candidates were often selected for political reasons (in hopes of later getting a job) and in many cases were not dedicated or very interested. Instead of opting for the more traditional selection process for CHWs, MCDI decided to test a methodology of working with community groups (mostly women’s groups). All members of the group would receive basic training and mothers who emerged as natural leaders with a vocation for service would be selected for more in-depth training and supervision as “madres vigilantes” (“vigilant mothers”), called “CHVs” in this report. This new strategy appears to be showing positive results. Women’s groups are active, and leaders are emerging. While the MCDI team has developed reasonable selection criteria (married with children, over 25 years old, natural leader, willingness to volunteer) the team is still discussing the exact responsibilities and time commitment of these CHVs; if they will be required to file written reports and to whom they will be sent (MCDI, nurse auxiliary). The three￾page reporting form developed by PROSIN/Núr may be used, although with some modifications (see section on “Information Management” at the end of this report for more details on this form). If reporting will be required, the problems of data collection, monitoring, and the cost of reproducing the forms still need to be discussed. The team is still also discussing how to separate training and organizational activities for the larger women’s group from those specific activities for the "madres vigilantes." The new strategy, while apparently successful, places a greater burden on the limited MCDI staff, who must spend much more time in individual communities in the training and selection process for “madres vigilantes” rather than centralizing training of pre-selected CHPs. Even the larger, five-person technical Child Survival team is not large enough to train all the CHVs in over 400 communities before the end of the project. Training must be delegated to nurse auxiliaries if MCDI is to scale up to all of Cotagaita and eventually to Puna as well. The desired number of CHVs is still under discussion. If at least one CHV is trained in each community, each will be responsible for about 35-40 families on average. This number is a somewhat higher than average for such projects, especially in sparsely populated regions. It may be advisable to train at least two to three mothers in each community to reduce the workload, provide mutual support and provide redundancy if one does not continue to function well. This would also permit CHVs to be responsible for a more reasonable 15-20 families each. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 34 CHVs Recommendation: In collaboration with nurse auxiliaries and municipalities, the role of CHVs needs to be clearly defined to include their reporting requirements, supervisory mechanisms and relationship with nurse auxiliaries. The recommendation to increasingly delegate community-based activities to auxiliary nurses is supported by the fact that they are spending increasing amounts of time outside their health facilities in communities, were observed to be well-accepted and command great respect in their communities. One unexpected observation was the appearance of an item in the 2002 budget for Vitichi to support the training of 50 “health promoters”. Recommendation: MCDI should make every effort to coordinate this municipal effort with the CS project. Community-based epidemiologic surveillance The delay in implementing the community-based surveillance model developed by Save the Children (SECI) has been discussed above under the section of HH/C-IMCI. During the evaluation, it became known that the District is planning to implement a community-based epidemiologic surveillance system of its own in 2002. This is a very positive development, as a system developed by the MOH, will likely be much more sustainable than one implemented by MCDI using an external model. Recommendation: The MCDI team should investigate the community-based epidemiologic surveillance system to be implemented by SEDES this year and support this effort. Selection of community members as candidates for training has already begun. As much as possible, MCDI should immediately begin a dialogue with the district to coordinate the selection process for trainees with their own efforts to select and train CHVs. Mobilization for special health activities, such as NIDs and health fairs The team has noted that community participation in NIDs has been limited to announcement of the events and specific logistical support, especially for transportation. The team expects to encourage more active participation in the future, including planning and evaluation, as CHVs and community health committees become more active and functional. This is a reasonable approach. A positive example of successful cooperation for community mobilization occurs in Vitichi municipality, where there is excellent cooperation between many community health committees and nurse auxiliaries during vaccination days. The committees help with vaccine transport and provide food and refreshments to vaccinators and auxiliary nurses. One barrier to greater participation in NIDs is the top-down nature of the planning and execution of the NIDs. This may be ameliorated as the role of the Municipalities is expected to grow and that of the Districts lessen over time. A second barrier to greater participation is the existence of USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 35 some poorly organized and/or divided communities. It is hoped that the presence of organized women’s groups and health committees will help overcome this problem. Some communities have demonstrated their ability to mobilize during the seven health fairs organized during the life of the project. See the next section on BCC/IEC activities for further discussion of this activity. b. Communication for Behavior Change This activity is one of the most important to the project if sustainable reductions in maternal and infant/child mortality are to be achieved. The goal of this activity as stated in the DIP is “to achieve measurable and sustainable behavior change for maternal and child health interventions among communities and families in the project area”. The overall approach, as described in the DIP, is to include messages reinforcing all key behaviors. There will be a special focus on messages in two areas identified during baseline assessments as specifically deficient: danger signs relating to maternal and child illness and promotion of the sixteen key family practices outlined under IMCI. According to the DIP, materials are to be drawn from a variety of sources, including Núr’s HH/C-IMCI materials and materials from the MNH project. The project also anticipated taking advantage of materials and experience from Esperanza/Bolivia on a contractual basis. The DIP outlines priority targets as mothers of children under five, family birth attendants, community groups (especially “women’s groups and rural cooperatives”), and traditional healers. The DIP describes activities, which such as group meetings, interpersonal communication (including home visits by auxiliary nurses and CHVs), counseling during consultations, and cassette debates to promote behavior change. Visual aids are to be used as adjuncts during these activities. Radio as a medium is mentioned as a possibility to be further investigated, though funds were not allocated for development of materials or their transmission. As previously discussed, BCC/IEC activities are not yet fully underway. The team carried out formal baseline quantitative and qualitative assessments and has prioritized messages. At present, transmission of messages through some media is occurring. Counseling during consults is verified through supervision visits. Messages As previously mentioned, a gap analysis, which focused on the 16 Key Family Practices, was carried out based on the focus group discussion results. A first set of prioritized messages was prepared. The project targets messages that include IMCI and HH/C-IMCI (based on the 16 Key Family Practices under IMCI) plus messages on maternal health, in accordance to what was outlined in the DIP. They appear to be complete and appropriate with one exception: there is little mention of vitamin A in any of the existing materials. The team should take care to guarantee that some messages are included. More than one message package may have to be prepared, according to the characteristics of the community. For communities with access to health services, messages encouraging utilization of USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 36 health services may be most effective, whereas in isolated communities where access is limited, greater emphasis on danger signs and emergency planning may be more applicable. Targets The team has so-far targeted only caretakers (mothers), women’s groups, and the community as a whole (during health fairs). During the evaluation, discussion with the team revealed some uncertainty about the stated plan to target rural cooperatives and traditional healers. The team is uncertain how they should be reached and how effective this would be. They felt that effort that would be needed to identify and train traditional healers may not be worth the limited benefit. They were also confident that women are the primary decision-makers and caretakers, and that specifically targeting men would not yield great benefit. As the recommendation for delivery planning focuses on encouraging institutional delivery wherever possible, targeting men with messages regarding birth planning should be prioritized to those more isolated communities. Recommendation: The team should continue to give priority to targeting messages to health workers, CHVs, caretakers, and communities as a whole. Other targets (traditional healers, rural cooperatives, TBAs) may receive attention according to individual need and opportunity. For example, TBAs who are operational in the project area may be provided with MNC training based on HH/C-IMCI strategy. Media The variety of media for transmission of information is somewhat limited by low literacy rates, many people do not speak Spanish, and most communities are isolated with only limited access to radio. A local FM radio station exists in Cotagaita, but only reaches the edges of the town. No other local radio stations broadcast in the zone and rural families with radios listen to stations from far outside the project zone. These factors limit the media choices for transmitting messages. Written materials are of little use and the impact of mass media would be limited. Also, the sparse population density adds to the difficulty, as it is more difficult to transmit messages to large groups. The team is developing visual aids for communication from flip charts developed and tested in Quechua-speaking areas by Núr/PROSIN. These flip charts have 29 pages and depict key HH/C￾IMCI messages. In addition, the team is in the final stages of adapting and testing a set of 29 supplemental laminated cards with drawings depicting messages that do not appear in the flip￾charts, such as maternal and neonatal care. Some of these also depict positive and negative examples of IMCI messages that the team has discovered are very effective in stimulating discussion. The IEC plan includes the reproduction of a set of cards for each auxiliary nurse in the project zone to be used during group education sessions and home visits. The HH/C-IMCI flip charts are already in use by MCDI and accompanying auxiliary nurses during training sessions with women’s groups. The media chosen to transmit messages includes the following: Interpersonal communication USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 37 This is the most important medium in the project, and will occur as CHVs (“madres vigilantes”), auxiliary nurses and other health professionals perform home visits and counsel mothers. It can be very effective, as the messages can be tailored to the individual needs of mothers and families. Most nurse auxiliaries currently visit each home with a young child or pregnant mother every one to two months, so the opportunity for message transmission is expected to be high. Many issues remain to be decided, however, regarding the CHVs, including the desired number of CHVs for a given population, expected workload of a CHV, expected frequency of home visits to caretakers by CHVs, and reporting (if there will be any at all). In addition, the project has not budgeted for reproduction of flipcharts and laminated cards for CHVs to use during home visits. Only nurse auxiliaries will be equipped according to the current plan. The effectiveness of interpersonal communication would be enhanced if each community had a set of materials for use by the CHVs. This would boost the effectiveness of each visit, help keep messages more accurate, help ensure that all messages are covered, and increase the prestige and credibility of the “madres vigilantes”. Another concern relates to the large number of messages. For example, according to project objectives, eighty percent of caretakers are expected to be able to memorize and and list over 15 danger signs (three each for diarrhea, pneumonia, prenatal, delivery, post-partum, neonatal). Altogether there are over 30 key messages that CHVs must transmit to caretakers. Without some sort of memory aid, it is doubtful that all these will be communicated accurately, let alone absorbed. Recommendation: MCDI should consider reproducing laminated cards and flipcharts for each community in order to enhance the effectiveness of the CHVs’ interpersonal communication activities. In addition, it would be advisable for MCDI to begin developing a supervision instrument for interpersonal communication that can be applied by MCDI staff and nurse auxiliaries. Use of this instrument could be included in the HH/C-IMCI course for nurse auxiliaries so that they can begin to practice its use with CHVs. MCDI needs to work with the HH/C-IMCI working group to acquire technical skills in order to prioritize key messages in collaboration with the nurse auxiliaries and the MOH. Finally, coverage of message transmission through this medium should be monitored in some way. If the project aims to transmit over 30 messages, one or two messages are transmitted during a single visit, and each house receives a visit only every two months, it will take at least two years for a caretaker to hear each message a single time. It will also be important to ensure mechanisms to reach marginalized families, such as designating homes to the responsibility of CHVs and requiring a minimum visit frequency. Counseling during consults This is another important medium for transmission, as nearly all caretakers have some contact with a health worker eventually, either at facilities or during home visits. The quality of counseling by nurse auxiliaries is being monitored during supervisory visits by MCDI. The importance of this supervision is highlighted by the fact that this was the weakest item found during IMCI supervisions, with only 20% of auxiliary nurses informing the caretaker of signs that would indicate that the child should be brought back to the health facility immediately. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 38 MCDI should intensify this supervision and pay special attention to the quality of post-consult counseling. Group sessions, including training This medium of transmission is also underway in the 31 communities where MCDI is working with women’s groups. It is encouraging that in most communities almost all mothers participate in the groups, as this greatly increases the opportunity to provide high coverage for message transmission. These sessions are expected to be very influential in positively changing knowledge and behavior, as the “teacher” has high credibility and the “students” are attending specifically to learn. This is quite different from a medical consult, where a mother may be so worried about her child that she cannot listen attentively to instruction, or a home visit, where the caretaker may be busy and distracted. At the present time about 560 members of women’s groups are participating in HH/C-IMCI training. The MCDI team is in the final stages of testing a supervision instrument for group sessions, which evaluates the degree of participation, content, methodology, and perceived degree of learning. The team plans to use this instrument with each other, with nurse auxiliaries, and with CHVs during group sessions. It is hoped that nurse auxiliaries will be trained in its use. Recommendation: For the purpose of quality assurance, the team should be encouraged to implement the group education supervision instrument as quickly and widely as possible. The current four-page form is somewhat lengthy and may eventually require some simplification. The possibility of using job aids for this activity should also be explored. Health fairs This medium of communication was not mentioned in the DIP, but may prove quite effective, especially in a region nearly devoid of other mass media. Since the project began, seven fairs have been organized: Cotagaita 5/00; 8/00; 5/01; 11/01 Vitichi 5/01; 11/01 Calcha 7/01 Health fairs have been organized about every six months in Cotagaita. They are most commonly organized with the leadership of nurse interns working in the Municipal Hospitals or other groups, including MCDI. Fairs typically last one entire day, usually on a market day when many community members are in town. Many entities have participated, including CARE, the MOH, other NGOs, community groups and schools. Many set up stands, which demonstrate and educate community members about some aspect of health. Other activities include games, contests, and theater. Hundreds of people typically attend the fairs. The effectiveness and coverage of the health fairs is not being formally evaluated, though a simple pre/post interview of a small sample of participants would provide some feedback to improve the fairs as well as provide some measure of their impact. Other similar public activities should be encouraged as well, especially during meetings, special events and festivals. Other media USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 39 The DIP mentions radio as a possible medium in the zone, though it cautions that coverage is likely to be low. Indeed, coverage for radio appears to be low, and families with access most commonly listen to stations coming from outside the project zone. Project resources would probably best be spent on other media; if appropriate, ready-made materials can be found and broadcast locally (in Cotagaita) at very low (or no) cost. Radio may be of some limited benefit in reaching a small part of the population. MNH and PROSIN The project will draw on the experience from the work of MNH in the municipality of El Alto. The MOH (CIELO), supported by the MNH Project, has designed an integrated IEC strategy, which allows the municipality to conduct IEC campaigns that convey appropriate messages on maternal and newborn health. Furthermore, the CS project is also actively pursuing BCC/IEC materials produced by PROSIN. Municipality/Family program As previously discussed under the HH/C-IMCI section above, UNICEF and Ministry of Human Development have launched a program called “Municipality/Family” throughout Bolivia. The program is in its third (of seven) year and works to educate mothers and families about key behaviors and messages, including the dissemination of information about the National Health Insurance Program. In Cotagaita district, the program is being implemented by the district social worker who visits communities and presents materials developed by the program. The project aims to reach 7600 families, about half the population of the District, half of these having already been reached. During a typical session of four to eight hours, the coordinator presents a series of videotapes (or audio tapes if necessary) in Quechua that illustrate key messages. All participating mothers are given a packet of 14 cards illustrating key messages as well. Unfortunately, these cards are written in Spanish and rely heavily on written material, a disadvantage in this low￾literacy Quechua-speaking environment. Messages are very similar to those in the HH/CC-IMCI materials. They include: the right to participation, the rights of the child, the rights of women, the right to primary education, the right to read (including adult literacy), the NHI program, pneumonia, diarrhea, breastfeeding, complementary feeding, risk factors and danger signs during pregnancy, family planning methods, hygiene, and basic child development. No material is presented on delivery, post￾partum risks, newborn risks, or vitamin A, and there is very little on immunization. Unfortunately, due to a lack of resources, manpower and transportation, the program is being presented to caretakers as a single marathon session rather than as a series of shorter sessions with discussion. The perception of the team and community members alike is that little is retained from the sessions as there is simply too much information being presented too quickly. The MCDI team has occasionally presented the materials themselves as part of their training of women’s groups. This program, however presents an excellent opportunity to diversify the BCC/IEC component of the project. The Municipality/Family materials, especially the video and audiotapes, are already developed, tested, and are of good quality. The program also provides each participating household with written materials. The messages coincide almost exactly with those of the CS USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 40 project. As already mentioned above, the MCDI team should carefully examine the materials and take advantage of them to maximize the BCC/IEC component of the project. Monitoring As BCC/IEC activities are only just beginning, progress toward improving knowledge and practices is not yet being monitored. It is important that the team adapts and implements a simple way to monitor BCC/IEC activities as soon as possible. A very simple interview form for mothers across the project area applied regularly and systematically, would suffice. Ideally, nurse auxiliaries could be trained to ask the questions, thus improving sustainability. If so, the system must be very simple and quick to apply. LQAS was mentioned in the DIP as a possible method for providing a systematic sampling and interpretation framework. This would be appropriate, but the team would require outside technical assistance in order to start using this sampling method. It should be pointed out that LQAS is a statistical sampling tool that can be applied to a wide range of situations. It is important that the team move ahead with some monitoring tool in the meantime. The sampling and interpretation method for this tool could later be adapted to using LQAS if so desired. See the section on “Information Management” for further discussion. Recommendation: The team should not wait, but should move ahead to develop and implement a streamlined tool to monitor caretakers’ knowledge and practices. A very simple tool and a very small but consistent and regular sample should provide useful results. For example, a nurse auxiliary could apply a form with eight or ten key questions adapted from the KPC to one randomly selected mother at each health post once each week. If tabulated and monitored, this would provide very useful information. Adaptation to LQAS is possible later, if technical assistance can be arranged. Constraints One important constraint to full implementation of the BCC/IEC component is the team’s relative lack of experience with BCC/IEC activities. During the second quarter of 2000, three MCDI team members and two District workers attended a two-day workshop in Tarija on performing focus groups for qualitative assessment administered by Esperanza/Bolivia. Then, during the second quarter of 2001, four MCDI team members along with 21 health workers from the District attended a two-day workshop given by CARE on designing and validating IEC materials. In addition, the two-team members who are attending Núr University’s MPH course in Cochabamba have recently finished theoretical training in BCC/IEC. Nevertheless, the team is composed of young health professionals who have little training or direct experience in designing and executing a systematic BCC/IEC plan. The project has a long list of messages (at least 15 messages), many of which are danger signs that caretakers are expected to memorize. During one of the evaluation meetings, the CS team was asked to list the danger signs. The team of five health professionals working together had a difficult time doing so. It appears that there may be too many messages disseminated to caregivers at one given time. After the gap analyses key messages were prioritized however there still appears to be an excessive amount of messages to be memorized by caregivers. This illustration is not meant to criticize the team’s knowledge or abilities, but rather to highlight the USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 41 need to focus carefully on exactly which messages are to be transmitted, how they will be disseminated, and to which population. MCDI has already taken steps to improve its BCC strategy and plan of activities by using the BEHAVE framework. This methodology, developed by the CHANGE Project, has been tested in Bolivia and further adapted for use in Central America. Recommendation: After the gap analyses, key messages were prioritized however there still appears to be an excessive amount of messages to be memorized by caregivers. Therefore, key messages should be reassessed and reprioritized. The evaluation team unanimously felt that outside technical assistance would be beneficial to assist the team in refining a detailed written IEC plan. This would probably best take the form of a workshop focusing on the project’s specific plan rather than IEC in general, and could draw upon in-country resources discussed during the evaluation. (MCDI is in the process of addressing these issues by contracting a consultant who will support the team in improving the BCC strategy and assist in prioritizing key messages using the BEHAVE model as well as resources from MNH, MotherCare and PROSIN.) c. Capacity Building Approach (i) Strengthening the PVO Organization This project is fully consistent with and contributes directly towards the BHR/PVC Strategic Objective of increased capability of PVC’s PVO partners to achieve sustainable service delivery, as elaborated in the BHR/PVC Strategic Plan. Moreover, the project has supported the institutional strengthening objectives referenced in the DIP to include supporting the technical strengthening of the CSST, enhancing MCDI’s participation in CORE Group activities and in successfully networking with collaborating agencies implementing child survival related programs, e.g., the CHANGE Project, the NGO Networks, PEI Initiative, etc. A truly significant facet of institutional strengthening related to MCDI’s implementation of the CPCSP is that it has afforded MCDI an opportunity to work in Latin America, a geographic region where our previous exposure was limited. Home office participation in the implementation contributes directly to strengthening the capability of MCDI to more effectively implement child survival projects in a manner consistent with BHR/PVC Strategic Objectives. The technical capacity of the home office has been strengthened directly through the participation of CSST members in the myriad CS training activities organized by the BHR/PVC, BASICS, CORE Group and CSTS. Clearly, technical expertise related to the implementation of the IMCI strategy at both the facility and household levels has been reinforced, as a result of MCDI’s participation in the IMCI Working Group and the HH/C-IMCI Workshop in Baltimore, January 2001. Technical collaborations with organizations such as the American College of Nurse Midwives, and The Safe Motherhood Program and The Maternal and Newborn Health Project have contributed positively to MCDI’s efforts to improve its design and implementation of maternal and neonatal health interventions. And though there is no specific HIV/AIDS component in the Bolivia project, MCDI has acquired significant knowledge and expertise apropos of the design and implementation of STI/HIV/AIDS prevention and care programs, as a result of participation in the BHR/PVC program. It is worth noting that the dissemination of technical knowledge and lessons learned occurs throughout the USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 42 entire organization via the mechanism of the home office Technical Advisory Group which is comprised of staff members from both the international division as well as all domestic divisions. In an effort to improve its overall capacity and organizational performance, MCD headquarters in Augusta, Maine conducted a series of strategic planning workshops in 1998, in which the International Division participated. Using a synthesis of organizational assessment methods (e.g., Strategic Planning Workbook for Non-Profit Organizations, Miller Foundation; Partnering to Build and Measure Organizational Capacity, CRWR Committee; Management and Organizational Sustainability Tool, (MOST)), both the domestic and the international divisions identified strengths and weaknesses and assessed their impact on the organization’s ability to carry out its mission statement. As a result, The Board of Directors mandated the creation of an inter-divisional Technical Advisory Group in 2000 that provides technical support to the international division. During the first quarter of 2002, MDCI will embark on a capacity assessment using the Institutional Strengths Assessment (ISA) tool developed by CSTS. To carry out this assessment MCDI is collaborating with CSTS to utilize the ISA methodology and survey instrument. The assessment will include the participation of the field team in Bolivia; the results of this survey will be used to formulate an organizational capacity development plan to address the weakness identified at the home office. Subsequently, it is MCDI’s intent to adapt the ISA to assess the capacity of the field office and its partners. If the ISA instrument is not deemed appropriate for field use, we may use the MOST instrument or the DOSA as proposed in the DIP. (At the time of DIP preparation the ISA tool was not yet available. MCDI believes it to be a more appropriate instrument for organizations implementing child survival projects than previously available instruments.) (MCDI undertook an ISA in March 2002) The capacity of the field staff have been addressed above and consists mainly of training activities that enhance their capacity of developing technical interventions and improves the quality of project implementation. (ii) Strengthening Local Partner Organizations As mentioned in the first section of this report, mid-way through the year 2000, the agreement between the original local partner, Esperanza/Bolivia, and MCDI was dissolved. The designated partnership was changed to include both SEDES/District MOH and Núr University as partners to be strengthened. The DIP states, “the capacity-building indicators for MCDI and its partners including SEDES, will be finalized later this year after baseline Organizational Capacity Assessment is carried out by the organizations”. The tentative capacity-building objectives as outlined in the DIP are the following: SEDES/District MOH USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 43 At the mid-term of the proposed project, the District/Municipal MOH will have developed and implemented an IMCI program of services; and will have further developed and implemented a plan to expand and improve EPI and maternal and newborn care services. Progress: IMCI training is almost complete in Cotagaita district and is being implemented. Humanized delivery and EPI training is completed. There is still a need for refresher training in obstetrical emergencies, and essential equipment has not yet been purchased. At the end of the project, the MOH will have the ability to determine accurate costs of maternal￾neonatal care according to new national standards consistent with WHO's Mother-Baby Package. Current reimbursement ratios are lower than actual treatment costs, placing the operational viability of project facilities at risk. Progress: The initial workshop cost study was held to train staff, test the materials and adjust the software. Data-collection is in progress. Núr University, graduate department To expose Núr University staff to methods of Monitoring and Evaluation, GIS and cost analysis and cost recovery systems Progress: Staff from the graduate studies department at Núr participated in the cost-study workshop. (GIS training has not yet been introduced, however the project has mapped health facilities in the Cotagaita District.) Equip Núr University with personal computers and provide technical assistance on distance learning programs for maternal and child health. Progress: This objective is probably not needed and should be eliminated. See below for discussion. MCDI carried out a limited assessment of capacity of SEDES/Cotagaita district that included a health facility assessment, site visits and interviews. They identified weaknesses in various areas, including planning, systematic supervision, missed opportunities for immunization and an inefficient immunization program, among others. Although no specific objectives were defined for measuring progress toward alleviating these problems, the DIP proposed the creation of ad hoc committees under the auspices of the project’s Coordinating Committee to coordinate project resources to carry out training workshops for key SEDES and District staff in topics such as planning, monitoring, evaluation, and administrative skills including financial management and budgeting. In addition, the DIP mentions other activities, including joint supervision, participation in MOH data-analysis workshops to improve data use for decision-making, and assistance to MOH and Municipal agencies in preparing proposals to obtain resources from other projects, such as the World Bank’s project for development infrastructure. The project has met with success in strengthening and improving health services at the facility level, however the planned improvements in management have been less successful. The major USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 44 constraint has been the astonishingly high rate of turnover of key MOH staff since the project’s inception: SEDES directors 5 Cotagaita district directors 4 (The Cotagaita director was changed yet again during the preparation of this report, raising the total to 5.) Municipal hospital directors Cotagaita 2 Municipal hospital directors Vitichi 2 Mayors of Cotagaita 3 The same can be said about most of the health professionals and senior department personnel at both the SEDES and District levels. Most changes are due to political interference, and because Bolivia does not have a stable civil service, all government positions down to the lowest levels are subject to political interference. In addition, similarly to many developing countries, relations between institutions are not as strong as those between the individuals that occupy key positions. The constant turnover of personnel has meant that MCDI must essentially start all over with partnership activities every time there is a change. However, the problem with coordination of activities is not limited only to high turnover. Key MOH staff members are often absent from the zone during meetings or trainings, often traveling with little or no prior notice. In addition, the civil unrest mentioned at the beginning of this report has made travel and coordination with Potosí more difficult. It was observed during the evaluation that most local partners, including the District MOH staff, and the mayors and city councils were only vaguely aware of the project’s goals, activities and achievements. This is largely due to their lack of continuity (because of high turnover) and effective regular coordination and communication mechanisms. It should be noted that, in spite of the limited knowledge of MCDI's project goals and activities, MCDI’s relations with the MOH and Municipal authorities, Area personnel and other entities are excellent. The Municipality of Vitichi is providing free office space to the project and even offered to share their only telephone line. Recommendation: In order to increase awareness of the project, the evaluation team agreed that increasing the volume of written communication, reporting, and general information to all partners, including Municipal governments, SEDES, Districts, Areas, and key staff would be beneficial. This will help to improve awareness of the project as well as maintaining good public relations. Finally, the MCDI Project Manager’s family lives in Tarija, not Potosí, reflecting the partnership with Esperanza/Bolivia at the beginning of the project. The problem with MOH staff turnover will not likely diminish, and ongoing close coordination with SEDES will be necessary for the success and sustainability of the project, even if the partnership focus is shifted to the Municipalities. MCDI has recently promoted a Bolivian physician on the team, to the post of Technical Director. Hopefully, this will give him more authority to coordinate and negotiate with Departmental and District authorities, which will help alleviate the problem. Nevertheless, USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 45 frequent personal contact between the country director and Departmental authorities will continue to be important. After Esperanza/Bolivia left the project MCDI and SEDES mutually agreed that the Coordinating Committee at the level of Potosí was no longer necessary and that MCDI would coordinate directly with the District Director. The other committees, which were going to be created, were not established because they were dependent on the Coordinating Committee. Thus, the formal institutional capacity assessment at the departmental level was shifted to the district and municipal levels. During the year 2001, relations between MCDI and the District Director were strained and there was limited cooperation outside of training and logistical coordination. The District prohibited MCDI from supervising health posts, alleging that supervision is the sole responsibility of the MOH, although “follow-up of the IMCI training” was deemed acceptable for MCDI to perform. Problems were exacerbated when the District refused to provide MCDI with any information from the HIS. The Director became mired in local political problems, and was changed at the end of the year. Relations between MCDI and the District have improved, however, the evaluator was informed that less than four weeks after the evaluation the newly appointed District director was again changed. Unfortunately, the high turnover is likely to continue through the life of the project, as 2002 is an election year. In spite of the constraints mentioned above, there have been successes directly attributable project interventions. The MOH has sponsored courses in EPI II, the new SNIS, and in planning and understanding the use of indicators. MCDI has participated together with District staff. All three courses resulted in successful implementation of the subject. A training session in the new system for management of the rotating drug fund, also sponsored through the MOH, was not successful, as the new system is very complex. The funds for implementation of the new rotating fund were never disbursed and the system has not been executed. Another constraint to capacity-building relates to the various levels of government involved. SEDES and the Districts (MOH personnel) are responsible for technical and policy oversight, but have few resources outside of salaries for personnel and a limited budget for supervision. SEDES, the Districts, and their “Areas” are responsible for supervision of health posts. However, it is the (elected) Municipal governments who manage and supervise the National Health Insurance program, the essential medications program and the rotating drug funds. Health posts report to both the MOH and the Municipal governments and are supervised by both. Joint supervision of health facilities, an important project activity, has only occurred rarely. Aside from strained relations between MCDI and the previous District Director, limited district resources for district and area supervision also make joint supervision difficult. Resources for supervision at the district level come through a UNICEF grant to the MOH, which covers the cost of transportation and per diem. Funds are disbursed irregularly, so district personnel are restricted to carrying out supervision activities only when funds are available. Even when MCDI offers transportation for supervision, in the absence of funds for per diems, district supervisors will not go. This hampers planning of regular supervision. It appears that municipal hospital (“Area”) staff are more willing to accompany the supervision activities, but the shortage of USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 46 personnel precludes their participation. The sole physician in the Cotagaita municipal hospital is also the Area Chief and the hospital administrator. Although she is officially responsible for supervision of Cotagaita Area health posts and is personally willing and interested, she does not have a vehicle and has little time for half-day outings. As a result, most health posts report that they have received one or no visits by district or municipal staff during the previous year. (The upgrading of Cotagaita Hospital, as proposed by the MOH, may improve this situation.) Recommendation: In order to make it possible to implement joint supervision to improve the quality of supervision and help guarantee sustainability, MCDI may want, in coordination with Municipal authorities, to explore options such as providing non-financial incentives for MOH staff in order to guarantee their participation. Another planned activity was MCDI’s participation during the periodic MOH data-analysis workshops. These workshops are held monthly at the health post and area levels, every four months for the Muncipality of Cotagaita, monthly for the Municipality of Vitichi, and twice a year for the District and Department. Data from the SNIS are analyzed and plans are made to improve coverage. MCDI’s participation aims to improve the use of data for decision-making. MCDI has regularly participated at all levels whenever possible. The evaluation team feels that their participation at the health post level has been the most beneficial, the reliability of the data is improving, and the use of data for decision-making has slowly been improving. It is encouraging to observe that at the last Cotagaita district TAI, MCDI was asked to present its data for the first time. However, presentation was not possible, because the MCDI team was not informed about this and thus, was not prepared to present data; this development is promising. The final capacity-building activity described in the DIP is assistance to the District and Municipalities in obtaining funds for infrastructure and other projects through external sources. MCDI helped the Municipality of Vitichi prepare a proposal to the Japanese Embassies program for assistance to community-based projects for infrastructure development. Although the request was not funded a new proposal is now being prepared for equipping the Cotagaita hospital operating room. MCDI’s newest team member, initially hired for the cost-study, is also a grant specialist who should contribute to increasing the project's ability to secure resources for Muncipalities, and secure matching funds for MCDI. He will also train local Municipal and MOH personnel in how to obtain funds as well. MCDI is also pursuing World Bank funding opportunities in cooperation with the municipalities. MCDI has provided some assistance for key infrastructure development for the project as well. MCDI donated two computers to the District that are now being used to process information from the SNIS, improving data use. Project funds were used to help rehabilitate an ailing District vehicle (originally a donation from USAID) to ease the acute shortage of available transportation for supervision. Project funds were also used to rehabilitate a large room and bathrooms in Cotagaita’s District offices to serve as a training center for health workers. Finally, MCDI provided secretarial assistance to the District for nearly a year, allowing them to reorganize their files and restructure some administrative procedures. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 47 Cost study The second major activity for strengthening SEDES and the District(s) was the implementation of a primary health care cost study for the National Health Insurance system. The impetus for this study came from indications that the reimbursement rates for many primary health care activities are below cost. This study would provide the MOH with information that may lead to a more realistic reimbursement table. In addition, by participating in the study, local partners would learn how to carry out such studies in the future. MCDI-USA sponsored a two-day seminar at Núr University attended by staff from Núr’s graduate department, representatives from the Municipality of Santa Cruz, two MCDI staff members and one each from the Municipalities of Cotagaita and Vitichi. The cost-study methodology was introduced, and the data-collection instruments were validated. In addition, there were modifications made in the software. However, data collection did not begin until late in the year, because some trained MCDI team members left the team, and MCDI and Núr were unable to coordinate in arranging graduate students to help with data-collection as planned. MCDI subsequently hired a staff member who trained himself in the data-collection. He is collecting the data with the assistance of administrators from the Municipalities (in each case, the person responsible for administration of the National Health Insurance reimbursement) and the Cotagaita district administrator. MCDI alone is entering the data, which are about 50% complete for Cotagaita district. According to MCDI, the data analysis will occur at MCDI/Washington and the report will be returned for dissemination. Municipal and District administrators expressed satisfaction with their participation in the study, saying that it will help them better determine costs in the future. MCDI is reexamining its capacity-building strategy with SEDES and the District. This is timely, as the agreements with the MOH expire this month and will need to be renegotiated. Several factors have led MCDI to shift its focus away from strengthening SEDES and the Districts, instead focusing on Municipal governments and their role in the health sector: · the problem of high turnover and political manipulation at the level of SEDES and the District will probably not diminish, and may worsen this year due to the elections. Personnel at the Municipal level are more stable and more accountable, as they are elected, not appointed, as are SEDES and District personnel. · civil unrest will likely continue, making frequent travel to Potosí to coordinate with SEDES even more difficult · there appears to be a trend in Bolivia toward strengthening Municipal governments and diminishing the role of vertical entities such as the MOH and SEDES. Some even speak of doing away with the District level altogether. · experience has shown a seemingly greater openness to cooperation at the Municipal level compared with the District level · the National Health Insurance program, which is likely to expand, is being managed by Municipalities, which are asking for assistance in improving their management. This system will likely grow in size and scope. In addition, the rotating medication fund and essential medications are managed through Municipalities. · Municipal governments have sources of income outside central budgets, and so, have access to discretionary funds. This allows a level of flexibility that SEDES and the Districts do not USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 48 have. In addition, many national infrastructure and capacity-building projects are aimed at strengthening Municipalities. MCDI can add its efforts to these, and assist Municipal governments to access these resources. Recommendation: The evaluation team wholeheartedly supports the shift in emphasis in capacity-building away from SEDES and the Districts and toward strengthening Municipalities. This is especially important in strengthening of management, with special attention to management and supervision of the National Health Insurance program, the rotating drug fund, essential medications, and the Municipal Health Committees. In discussions with Municipalities of Cotagaita and Vitichi, officials seem open to the idea of hiring a nurse with Municipal discretionary funds to provide supervision and oversight to auxiliary nurses to improve their technical and administrative skills. Recommendation: MCDI should follow-up on the discussions to encourage the Municipalities to hire a designated nurse supervisor and, if successful, work to strengthen that person’s supervisory skills through joint supervision. Recommendation: MCDI should continue its efforts to support the newly formed Municipal Health Committees (COMUSA) through active participation in their meetings, planning, and evaluation. The COMUSA’s should probably become the chief coordinating bodies between the project and partners. NÚR UNIVERSITY The second partner cited in the DIP is Núr University. Núr, a private university, has its main campus in Santa Cruz, and others in La Paz and Cochabamba. The university has been active in the development sector in Bolivia since its founding as Bolivia’s first private university over 17 years ago. The DIP describes Núr as both a partner to be strengthened and one to provide assistance to the project. The latter role appears more consistently, and, indeed, is the one most likely to be important to the project. Núr’s role is described in the DIP as providing periodic assistance to the HMIS, to provide technical assistance in training CHPs (now CHVs) and TBAs, providing assistance in record-keeping for TBAs, and providing management training for SEDES and the Districts. Núr was also to participate in regular meetings of the project’s Coordinating Committee. To date, Núr/Cochabamba has provided training in HH/C-IMCI, as well as teaching a seminar in Cotagaita on participatory training methods for District and MCDI staff. The project Coordinating Committee was not established, as explained above. TBAs will no longer be a focus for training of the project. MCDI/USA provided technical assistance to the HMIS (see section below on information management). In the future, Núr is likely to be called upon to provide technical assistance to Municipalities in management of the National Health Insurance fund. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 49 In addition to the institutional ties between MCDI and Núr University as outlined in the DIP, the MCDI country director has a close personal relationship with senior staff at the university, facilitating the relationship. Núr has helped MCDI obtain indirect access to PROCOSI, the Bolivian consortium of NGOs (begun as a consortium of Child Survival PVOs). Participation in PROCOSI is important to finding out about technical and financial resources available in Bolivia, and MCDI’s participation mentioned in the DIP. However, MCDI itself cannot join, as PROCOSI has declared a moratorium on accepting new members. The DIP states that, in return, Núr would gain knowledge in cost analysis for primary health care, cost recovery, GIS, and monitoring and evaluation. They would be given computers and technical assistance for distance education on maternal and child health. Members of Núr’s graduate department participated in the workshop on the cost-analysis methodology administered by MCDI in Santa Cruz, but this information has not yet been incorporated into any of Núr’s curricula, probably the best measure of “strengthening” a university. If GIS is implemented, Núr will likely participate as well. The mention of donating computers to Núr probably arose from a misunderstanding. Núr already has computers as it has a strong computer science department, which actually commercializes software in Bolivia as a source of income. The evaluation team was somewhat unclear about how MCDI/Bolivia would be able to meaningfully strengthen Núr University, which is many times larger and has far more resources. In addition, Núr has no ongoing presence in southern Bolivia, so contact between Núr and MCDI/Bolivia is limited. Recommendation: The project should eliminate the capacity-building objectives with respect to Núr University, and concentrate on strengthening the Municipalities. If these objectives remain, however, probably the best way to measure whether exposure to a new methodology or procedure (such as LQAS, GIS, or cost-analysis) is whether the university has incorporated it into its curriculum and taught it to others. (iii) Health Facilities Strengthening Almost all the activities mentioned in this section have been covered in other sections. MCDI performed a facility-based assessment prior to the elaboration of the DIP using an instrument designed by MCDI and Esperanza/Bolivia. It was applied to all health facilities in the Cotagaita district to examine essential equipment, transportation, communication, and infrastructure (including water, electricity, etc.). In addition, the team carried out an inventory of personnel and discussed needs with SEDES, District and Municipal personnel. A similar assessment is planned for Puna district in 2002. The evaluation team identified the following project activities as those most important for strengthening of health facilities: USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 50 Support to health workers in the management of their medication cards for essential medications and for the rotating drug fund. Management of the essential medication program and rotating drug fund, especially at the health post level, was identified as a significant weakness by Municipal authorities, who are responsible for its management. Two systems are involved: the essential medication system provides free medications to health facilities for conditions covered by the National Health Insurance plan, including ORS, antibiotics for pneumonia, vitamins and iron, paracetamol, and others. These are provided to health facilities through a central storehouse in Potosí in quantities based on the number of National Health Insurance consults reported by the facility on the appropriate form. Reports are sent to the Municipality, which pools the information, orders the medication, and distributes it to the health posts. The same system provides information for reimbursement by the National Health Insurance program. Municipalities have long noted that information provided to the Areas and Districts (MOH) through the SNIS was often different than that reported on the NHI forms to Municipalities, with the latter almost always lower than the former. This leads to reimbursement levels that are lower than they should be. Nevertheless, stockouts for the free essential medications are reportedly rare according to the MCDI team, auxiliary nurses, and community members. The second system is the rotating drug fund, which provides medications for conditions not covered by the NHI program, including medications for high blood pressure and diabetes, among others. Health facilities purchase these medications at low cost from the same central warehouse (through the Municipal government) using funds from their rotating drug funds. They are then sold to patients at a price determined by the MOH. In all but a few cases, capital in the health facilities’ funds has been depleted to amounts insufficient to purchase medications for an entire month. Many health posts interviewed had less than US$100 in their total fund at the time of the evaluation. Two main reasons for this depletion were identified. First, medications are often provided “on credit” when a patient cannot pay, and the loan is often not repaid. This problem is difficult to remedy. The second problem is that health post staff have an incentive to set up their own “private” pharmacy fund to generate supplementary income for themselves. Auxiliary nurses are operating many of the rotating funds as "personal pharmacies," which may create incentives to underreport consults. Although stockouts of essential (free) medications are rare, the frequent stockouts of other medications may have a detrimental effect on the credibility of the health facility, which may lead to lower coverage for key primary health care interventions. The rotating drug program can work, however, and is very successful in one health center visited during the evaluation. This health center is fortunate to have an American volunteer missionary physician who sees patients and oversees administration. The physician also trains the center's nurse auxiliary. MCDI will explore the possibility of coordinating with this physician for activities related to training of nurse auxiliaries and the management of the rotating drug fund. The rotating drug fund, well administered and operated strictly according to regulations, totals nearly US$1000, and the pharmacy is reportedly always well stocked. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 51 The MCDI CS team visited eight of the 38 health posts with the specific purpose of providing supervision and in-service training to auxiliary nurses on the rotating fund’s management. After these visits, overall management has reportedly improved, though problems with capitalization still occur. In addition, the MCDI team performed a specific study on medication management, the results of which are still being compiled into a report, which will be presented to the District and Municipalities in the coming months. This report should form the basis for a plan to further improve management of the essential medication system, rotating drug fund, and management of vaccines. Recommendation: MCDI should present the results of the drug management study as soon as possible in order to have time to implement its plan to further improve the system. Support the elaboration of statistical reports for the data-analysis committees (TAIs). MCDI should explore the possibility of coordinating with the American missionary physician for activities related to training of nurse auxiliaries and the management of the rotating drug fund. MCDI has provided specific in-service training to nurse auxiliaries in eight of the 38 Cotagaita district health posts in filling out the monthly SNIS forms in preparation for the TAIs. Training has been provided once in each of the six health posts, and eleven times (monthly) in two health posts. The quality and reliability of the information has reportedly improved from these health facilities. In addition, MCDI staff attend TAIs at the health facility, Municipal, District and Departmental levels whenever possible. Strengthen the training abilities of the nurse auxiliary through supervision during training sessions at the community level. MCDI planned and supported a four-day workshop in February 2001 on methods of participatory education in which all Cotagaita district staff, including nurse auxiliaries, participated. This workshop was taught by Núr University using their own materials. As mentioned in previous sections, twelve nurse auxiliaries, together with MCDI staff, have been teaching the HH/C-IMCI curriculum to women’s groups for the past six to eight months. Through this mechanism, auxiliary nurses are learning to independently teach by following the example of the MCDI trainers. Also previously discussed, MCDI has developed an instrument for measuring the quality of community education activities through direct observation. This form is still being tested, but should prove useful in systematically supervising and monitoring the quality of the nurse auxiliaries’ education activities after this has been formally delegated to them. Also, it has already been recommended that MCDI complete the development of the course in HH/C-IMCI for health workers and administer it as soon as possible. Organization of Women’s Groups to support the work of the health facility MCDI has organized new women’s groups in 15 communities and worked to strengthen another 16 groups. Support has been individualized to the needs of each group, and all are receiving training in HH/C-IMCI. These groups, in turn, help support the work of their respective health facilities through community mobilization, identification of children and pregnant women, case USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 52 finding, community education, and assistance during NIDs. This has also led to increases in demand for services at health facilities and resultant increases in coverage. As with community level training, the team is in the process of progressively turning over follow-up and supervision of the CHVs to the auxiliary nurses. In-service training and supervision of health facility administration, files, and reports The MCDI team has provided on-the-job training and supervision to health workers in 14 health posts and health centers to improve their administrative skills, such as filing and management of forms. The evaluation team and health workers themselves reported that administration had improved in these facilities. Support to logistics, especially in transportation of supplies General logistical support is ongoing, and is most important in transportation of supplies and people to and from health facilities. Many instances were seen during the evaluation, when MCDI delivered MOH forms to health facilities, transported gas bottles for refrigerators, and provided transportation to a health post in Cotagaita. In addition, MCDI has provided pointed logistical support for repair of key equipment (computers and vehicles, for example) by transporting these items to Tarija or Potosí where skilled services are available. This support has been most consistent for 24 of the health facilities in the zone. (iv) Health Worker Performance Most of the points in this section have been discussed previously, including training, supervision, and logistical support. All nurse auxiliaries but one have received training in IMCI, and another eight health professionals still need training (these eight are currently being trained). The MOH (through various grants and sponsors) supported training in EPI II, the new SNIS, the National Health Insurance program, the Municipality/Family program, planning and use of indicators, and management of the rotating drug fund for all staff. In addition, MCDI supported training in participatory training techniques and community organization, and CARE provided training in humanized delivery and quality care for all staff. Supervision with ongoing in-service training is the most important activity in strengthening health worker performance. Unfortunately, as described in the section on capacity-building of local partners, supervision by Departmental, District and Area staff are haphazard and infrequent due to lack of transportation, insufficient funding, and lack of personnel. Municipal governments, which are responsible for oversight of the National Health Insurance program and rotating drug funds, lack technical staff for supervision. Whatever supervision that does exist, is being carried out by Municipalities and the COMUSA is done by administrators for the NHI program. MCDI is using the form designed by PROSIN/BASICS for supervision of IMCI. The team felt that the form may need some modification (to include maternal/neonatal indicators), but it is easy to apply and will likely be useful once systematically in use. MCDI has applied the form in only ten instances to date, but plans to begin systematic use during supervision visits. Health professionals, including doctors and nurses are not yet being supervised, but this is planned. Part USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 53 of the form is to be used during observation of an IMCI consult and examines whether protocols were followed properly. The second part of the form records the completeness of registration on the IMCI forms, and the last part is an inventory of essential medications. The form is clear and concise, and the results of monitoring should prove useful to project management. As already described, MCDI is developing an instrument to assess the quality of community education activities at all levels. This should make it possible to detect problems with the quality of community training and group IEC activities. The team will strive to integrate the instrument with existing supervision tools and make it as simple as possible. Many different forms are in use by MOH personnel for supervision, and according to supervisors, forms are often designed ad hoc prior to each visit. Some of the standard forms are very long and detailed, examining infrastructure, stock, registration, community activities, and theoretical knowledge of the health worker. It is not clear how MOH Departmental, District and Area supervisors are using these data. Joint supervision between MCDI and the Districts/Areas has occurred only rarely, as explained in the section above on capacity-building of local partners. Recommendation: MCDI should renegotiate its agreements at all levels to ensure the implementation of joint systematic supervision of health worker performance. This includes agreeing on standardized instruments, schedules, and how and who will use the information. In addition to auxiliary nurses, supervision of health professionals in hospitals, using integrated IMCI/MNC protocols, should also begin as soon as possible. The information generated by the supervision system could easily be added to the GIS system if it is implemented in order to facilitate analysis. (v) Training See Attachment E on training for a matrix of all training that has been carried out during the project and the assessed impact of each course. It is apparent that there has been a great deal of training—so much so, in fact, that during 2001 the District declared a moratorium on new training courses because staff were away for so much of the time it was beginning to interfere with services. It is also apparent that there are many agencies and programs outside the CS project that are competing for access to health workers’ time. The most important contribution in training made by the CS project has been in supporting training in IMCI, both clinical IMCI and HH/C-IMCI. These two activities have absorbed the most time and effort, and are central to achieving the goals of the project. As stated above, only eight health workers in the Cotagaita district still need IMCI training (are currently being trained), including one new nurse auxiliary and seven other professionals who work either for the District or at lower levels. IMCI has been implemented by those who have received the training. An important constraint to the effectiveness of training has been the high turnover of staff at the District and Municipal levels, and to a lesser extent, within MCDI itself. Much of the investment in training done for MOH staff in 2000 has left the project zone. For MCDI the problem is not so acute, as most of those on staff received training before being hired, while they worked for the USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 54 District in the zone. MCDI’s shift to focus on the Municipal and health post levels, where turnover is less of a problem, seems reasonable if sustainable improvements are to be expected. The MCDI team is testing the HH/C-IMCI training curriculum and materials with women’s groups in 31 communities and has completed about half the training in these communities. The team applies a participatory methodology in one-hour sessions beginning with a discussion of the mothers’ own experiences and then proceeds to the information and methods. The flip charts and laminated picture cards were cited by mothers as very helpful and interesting. There is no formal assessment of the learning, however, but informal discussions are used to assess the effectiveness of the materials. As mentioned above, the team has developed a four-page form to assess group￾training activities, though the form is still being tested. The form will be used by team members to assess each other, to assess auxiliary nurses, and later, by auxiliary nurses to assess CHVs. As mentioned several times above, it may be beneficial to consider wider use of the “Municipality/Family” materials in the training, though perhaps in a more participatory way and in smaller doses, and to take greater advantage of the fact the program has left written materials with families and communities already. In addition, the use of cassette-debates (developed by Esperanza/Bolivia in Quechua) as outlined in the DIP would likely be beneficial. The most important immediate training needs are to finish the development of the course and materials in HH/C-IMCI for nurse auxiliaries so they can begin to assume their role as facilitators. In addition, MCDI will have to see that the remaining District staff are trained in IMCI and begin to schedule those in the Puna district for training as well. The management training for the District and Departmental partners as foreseen in the DIP has not been as effective as the project hoped, largely due to high turnover of MOH staff at the District and Departmental levels. And, again, the shift in focus to the Municipal level should help ameliorate this problem. Needs expressed by Municipal authorities during the evaluation focused primarily on improving reporting of activities related to the National Health Insurance program by health facility staff and management of the rotating medication fund. Núr may prove to be a useful resource in arranging this training. The final important issue is to identify materials and assistance to complete training for health personnel in obstetrical emergencies and protocols, as these topics were not included in CARE’s humanized delivery course. d. Sustainability Strategy The DIP lists the following as EOP expectations: At the end of the cooperative agreement, MCDI anticipates leaving in place in the project area the following: · A network of functioning health facilities providing quality child survival and maternal and newborn health services, with staff competently following SCM protocols (IMCI initial and refresher training and maternal/neonatal health training for health facility staff); documented through the SNIS, project HMIS and supervision records for quality of care USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 55 Half the area of intervention, Cotagaita district, has received training in IMCI. Implementation of protocols appears to good, but systematic supervision of all staff remains to be implemented. It may be possible that protocols are perhaps not being rigorously applied in all cases. Eight health workers in Cotagaita district (are currently being trained) and all staff in Puna district remain untrained (about 40 people). Cotagaita district staff have been trained in supervision of IMCI, but have not yet been performing supervision. It is encouraging that the Tupiza IMCI training center was not operational at the outset of the project, but is now functioning due to the efforts of the project. This will facilitate training of new staff in Cotagaita district when there is turnover and allow for refresher training. This should aid sustainability of IMCI in the zone. · MOH personnel and health facility outreach workers will continue BCC/IEC campaigns at periodic intervals, using materials and methods developed and adapted during the LOP. This has been documented through participation of SEDES and facility personnel in BCC/IEC activities. SEDES has action plans for BCC/IEC activities at the end of the program MOH personnel in Cotagaita district all received training in preparation of IEC materials, however, as noted above HH/C-IMCI activities have already been launched. BCC/IEC activities have only been partially launched in the communities, but nurse auxiliaries are participating in HH/C-IMCI training of groups and are gaining familiarity with the materials. Two district employees also received training in using focus groups, but only MCDI has used them directly. MCDI did support MOH personnel to receive training at Nur University. The adaptation of existing BCC/IEC materials for MNC are in the final stages of development. MCDI has maintained regular contacts with the MNH Project and PROSIN and hopes to receive support by the way of IEC/BCC materials for HH/C-IMCI. · A viable referral and counter-referral system linking community based providers such as trained CHPs and TBAs, health facilities such as the Health Centers and HPs, and higher level facilities in the Department (140 CHPs and 30 TBAs trained in early recognition, referral and simple record keeping); documented through the HP registers, SEDES records, project HMIS, and referral records Since there were a limited number of functional CHPs identified by the project, MCDI decided to change its strategy and focus on the mobilization and training of CHVs ('madres vigilantes"). The project has already begun organizing and training the CHVs in 31 communities. Because the training is still ongoing, there has not yet been an opportunity to evaluate to what extent the referral and counter- referral system is operational. The planned implementation of community￾based surveillance by the District in early 2002 will serve as a first step to support this activity. TBAs are being de-emphasized in the CS project. · There have been positive changes in knowledge and practices of mothers and households arising from the BCC/IEC strategies implemented through program interventions. This has been documented through the KPC and LQAS surveys, and qualitative research USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 56 If CHV training and BCC/IEC activities are implemented as planned, behavior change should occur and be sustainable. There is some indication that increased utilization of health services is already occurring. The largest constraint to success will be in obtaining wide coverage of BCC/IEC activities through delegation of community education activities by auxiliary nurses and CHVs. LQAS has not yet been implemented. See the information management section below for comments. · Strengthened Districts and Municipalities with improved managerial, supervisory, local health planning skills; documented through District and Municipal records, observation, and key informant interviews Some management improvements have occurred though mostly through the MOH’s own efforts (training in SNIS, EPI II, rotating drug fund, planning and use of indicators). Management training for Departmental and District levels is not likely to have sustained impact due to very high staff turnover. Some improvements in administration and management have been noted at the health post level. The project’s planned shift in focus to the Municipal level and focus on the NHI system should help sustainability. · Regular supportive supervision practices of health facility personnel by SEDES personnel following checklists and supervisory protocols; documented through supervision records and project HMIS. Joint supervision has only rarely been possible. The Districts and Area personnel already use checklists, but not systematically, and many different forms are in use. MCDI will encourage Municipalities to hire designated Municipal nurse supervisors, which should help guarantee sustainability of supervision. In addition, renegotiation of the expired agreements between MCDI and MOH/Municipal authorities provides an opportunity to restart joint supervision. · Sound planning and financial management strategies in place for the District teams to support reimbursement of charges from the Municipalities within the program area for activities covered by the NHI program; documented through financial records at the districts and municipalities. The shift in focus to the Municipalities should help make this possible. Municipalities have identified the health post level as the weakest link in the NHI system, and MCDI has already provided assistance to some health posts in improving administration and reporting for NHI. Municipalities have requested assistance in this area, and it should easily be possible in the second half of the project. Núr University is a potential resource for this training and technical assistance. Data gathering for the planned primary health care cost-analysis study is underway with the participation of Municipal and District administrative staff. This study should provide input into more accurate costing and reimbursement, as well as training administrative staff in cost-analysis methods. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 57 As the project is not substituting functions of the MOH, but rather functioning in an advisory role and as trainer, sustainability issues are kept to a minimum. Likewise, no new systems are being implemented other than possibly new supervisory tools (including IMCI) at various levels, though this is being done within MOH policy guidelines. The fact that the anticipated community-based surveillance system to be adopted this year will also be that of the MOH is encouraging, because this, too, maximizes sustainability. The replacement of volunteer CHVs who are either not productive, stop working in the future, or leave the zone is often a problem in projects such as this. MCDI’s more refined selection process, involving self-selection of one or more natural leaders in each community will minimize abandonment. The plan to train auxiliary nurses as HH/C-IMCI facilitators and delegate the training to them should, again, maximize sustainability, as replacement training of CHVs will be possible. Beneficiaries in communities, and especially the women’s groups in training, are naturally somewhat apprehensive about the shift from direct involvement of MCDI staff in training to a role as supervisor, but the fact that this change will occur well before the end of the project should help ease this transition. The fact that nurse auxiliaries are respected and well accepted in the communities for which they are responsible is very positive. Cost-recovery is built into the MOH’s health system, through the NHI plan, fees for services outside those covered by the NHI program and fees from drug sales from the rotating drug fund. MCDI is already assisting Municipalities and health posts to improve the management of these systems, and will intensify this activity with the shift in focus to the Municipality. The MOH itself at the District and Area levels are not greatly involved in cost-recovery activities. One potentially important constraint to the continuity of MCDI/Bolivia’s project is the fact that the project is scheduled to end in 2003. Limited time and funding are constraints to all project interventions and activities. At the present time, the BHR/PVC Child Survival grant is the single largest source of funds for MCDI/Bolivia. While it is recognized that the goal of the CS project is not to establish a long-term presence in Bolivia, one must not ignore the relatively high cost of setting up an initial first-time presence in a country. The cost efficiency of implementing an extension project would certainly improve, as MCDI would be able to take advantage of the existing physical infrastructure, capacity building of staff, country experience and contacts. Recommendation: It is recommended that MCDI should be encouraged to submit an extension application, which will fully incorporate all of the geographical areas of the Linares province (Puna and Caiza D Municipalities). MCDI has recently added a staff member to its team who has specific training in project preparation. His services are planned to be used in helping partners, such as Municipalities, tap into resources available through alternative funds. Recommendation: MCDI/Bolivia must not forget to look after itself as well. It should begin as soon as possible to identify sources of funding for other related projects and sources of matching funds for the Child Survival project. With greater diversity of funding, there will be a better USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 58 chance of sustaining funding. Although MCDI is not a member of PROCOSI, its contact through Núr University provides an opportunity to keep abreast of opportunities for future funding. MCDI should also strengthen its contacts with PROSIN (DIDESCO) in order to procure funds for health facility upgrading in the project area as well as for IMCI and MNC activities. Japanese funding and World Bank funding should also be pursued. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 58 C. PROGRAM MANAGEMENT 1. PLANNING The original project was jointly designed by Esperanza/Bolivia, MCDI/Washington, and SEDES. After Esperanza left the project, the DIP revision was undertaken by the Project Manager and local technical team and MCDI/Washington in consultation with SEDES. The original management structure proposed in the initial project application and subsequently in the revised DIP has undergone some changes due to the termination of the MCDI partnership with Esperanza and the increased integration of project activities with those of the District MOH and municipalities. As a result, the Project Manager and MOH decided that a Project Coordinating Committee that included SEDES Potosi was not necessary. The Project Manager now routinely coordinates with the District Health Director and municipal representatives through the mechanism of the District Health Council. At the District and Municipal levels, project coordination was initially attempted by having the team share offices with the District and share secretarial and administrative personnel. MCDI decided to move to its own offices after attempts at political interference with MCDI hiring practices. The constant personnel changes at the District level and interpersonal problems with one Director during much of 2001 led the team to shift coordination toward the Municipal level. Coordination with the District continues to occur, especially on logistics, but much day-to-day coordination in planning has been carried out by integrating the Project Manager within existing structures in the Municipal governments and mayor's offices, which includes the District Health Council. In Vitichi, the MCDI project nurse works out of an office in the Mayor’s office that was donated to for project use. This facilitates day-to-day coordination and planning. This participation has been fruitful in that it provides feedback to the project, helps to focus attention to the health sector and has contributed to the creation of a supervisory sub-committee for the health sector. In addition, the Project Manager and Technical Director have regular working meetings with SEDES and are regularly invited to participate in planning meetings at the Departmental level, though the high turnover at SEDES has hampered the effectiveness of this coordination somewhat. The project team collaborates with the District Health Council to ensure that its quarterly work plans are consistent with the plans of the Council. To ensure that its annual work plans are consistent with that of the Council, the MCDI annual work plan is discussed with the District Health Council and the District Health Director’s Office. There is an ongoing effort on part of the home office CSST to monitor program activities based on information emanating from the HMIS. Both the District and SEDES have copies of summaries of the project, though it was apparent during the evaluation that understanding of the project is unclear, both at the District and Municipal levels. The full proposal and DIP are available only in the MCDI office, and only in English. Partners’ limited knowledge about the project is due mostly to high turnover among partners’ staff. During the evaluation the team discussed ways to improve partners’ understanding through repeated presentations of the project’s objectives and methods and more frequent written reporting. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 59 The MCDI staff are aware of project objectives and indicators relating to the technical interventions, and they have good technical skills. However, as mentioned earlier, aside from the Project Manager, no one currently on the MCDI staff was involved with the DIP preparation. The current team has been hired gradually over the past six to twelve months. The project’s objectives and indicators have been translated into Spanish, however the full DIP is available only in English. The evaluation meeting was the first time many of the team members became aware of the capacity-building and sustainability objectives and the work plan. These should be translated so the team may use them on a day-to-day basis. At the community level, the evaluation team agreed that nurse auxiliaries are technically well trained, but knowledge about the role of MCDI and the project vis-à-vis the communities should be communicated more thoroughly and more frequently. Evidence of this comes from the frequent requests for equipment and supplies from partners and communities alike, even though the project was not designed to support community infrastructure. The team regularly participates in the TAIs during which SNIS data are analyzed and problems are identified. Planning for the following months based on the data is done jointly. 2. STAFF TRAINING In the beginning of the project, it was agreed that the Project Manager could benefit from an upgrading in CS technical skills. This decision was reached during the orientation in Washington D.C. when it was established that the Project Manager had strong administrative skills and unique local language skills but his CS technical skills were limited. He was encouraged to enroll in the MPH program at Núr University and consequently, started attending courses during the first year of the project. Due to the difficulty of recruiting personnel in the project area, which is extremely remote, additional staff did not start attending courses until the second year of the project. In addition to project staff, MCDI also supported the training of an MOH nurse at Núr University. Dr. Anne Keith, an MCDI Board Member and Professor of Nursing, assessed the skills of project staff during her visit to the project in 2000 and subsequently recommended that they benefit from broad public health training. Since the revision of the project DIP, the following activities have occurred (they are also listed in the training matrix in the appendix): · Training of the Project Manager, Project Technical Director and two nurses as IMCI facilitators; this training was provided by SEDES-Potosí and PROSIN. · The Project Technical Director and two project nurses were trained in methodologies to develop appropriate IEC materials through participation in a workshop organized by CARE in Cotagaita. · The Project Manager, Technical Director, and one of the nurses are currently participating in the MPH program in Núr University. The project is allowing them to take the time to study while the students pay the tuition themselves. · The Technical Director and two nurses were trained in HH/C-IMCI protocols by Núr University in Cochabamba. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 60 · Three MCDI nurses received training in clean birth and safe motherhood provided by CARE in Cotagaita, and in basic sanitation by the Peace Corps. MCDI anticipates identifying home office training needs based on the ISA that is scheduled to take place during the first quarter of 2002, with technical assistance provided by CSTS. At the home office level, the CSST has routinely participated in CORE Group meetings where CS project lessons learned are shared. The CSST's participation in these meetings has proved to be extremely beneficial in that home office staff are kept technically abreast of developments related to child survival intervention activities. In January 2001, a representative from MCDI CSST participated in the HH/C-IMCI workshop in Baltimore, Maryland that produced the new HH/C-IMCI framework; this framework has been sent to all child survival field team personnel. This framework has been shared with PROSIN and the UNAP (the “Unit for Care to People”), the department within the MOH, responsible for implementing HH/C-IMCI activities. The project has sufficient resources in the budget allocated for staff training and there is a continuous effort in the field to provide time and opportunities for the project staff to have access to training in salient child survival program areas. English language training for the Technical Director is programmed for 2002. The primary future training needs identified by the MCDI field team include further training and/or technical assistance in formalizing the BCC/IEC plan and monitoring the activities, assistance in implementing LQAS, instruction on using GIS if it is to be implemented, and instruction on how to operate the cost-study computer program in order to be able to extract the data. Partner Training Nearly all facility-based personnel in the Cotagaita district have been trained in the utilization of the IMCI protocols for case management, and two District staff members were trained in IMCI supervision. A total of seven MOH staff were trained as IMCI facilitators including four from Tupiza and three from Cotagaita. Nearly all facility-based personnel in the Cotagaita district have been trained in adult education techniques. MCDI has provided support for one nurse licenciada to participate in training workshops as part of Nur University's MPH program. Núr graduate students in the School of Public Health have been trained in methodologies for undertaking cost analysis of health services as part of the health care financing study. The MCDI Health Economist from the home office carried out this training. The MCDI field team is small, and staff skills are monitored informally by the Project Manager and Technical Director. 3. SUPERVISION OF PROGRAM STAFF All field staff, with the exception of one nurse, works directly with the Project Manager and reports to him on a daily basis. One nurse works in Vitichi and meets with the Project Manager USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 61 on a weekly basis (the town of Vitichi has only two telephone lines, one is public, and the other is in the Mayor’s office). These meetings allow the Project Manager and field staff to exchange ideas about the progress of activities, identify constraints and recommend solutions. Ninety-day probationary periods are used for new-hires, and have resulted in occasional staff changes. The recent promotion of Dr. Ismael Gonzales to the role of Technical Director should provide an opportunity for better technical supervision of staff. Due to the she small size of the project team (Project Manager, Technical Director, four technical team members, an administrator and a driver) the work-style is generally one of cooperation, sharing, mutual support and consensus. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 62 Roles and Workloads The original DIP called for a staffing pattern that consists of a Field Technical Director, Training Coordinator, Administrator, Secretary, and a Driver. This staffing pattern has been revised and includes: · the Project Manager who oversees all project activities and is the MCDI representative in Bolivia; · the Project Technical Advisor (recently redefined as a Technical Director) who is a Bolivian MD and oversees all the activities with the technical staff, supports the Project Manager and is responsible for the quality of IMCI and MNC activities; · an IEC/Training specialist who is an RN and acts as a community mobilizer; · three community mobilizers who are responsible for implementing and coordinating the community mobilization of volunteers, e.g., vigilant mothers and community groups, e.g., mothers groups. The team also consists of a driver and an administrative assistant. Although there is currently a division of job titles between mobilizers and the IEC/Training specialist, all are either RNs or physicians. However, hands-on experience with formal BCC/IEC activities is limited. In practice, all team members are currently performing very similar functions, and there seems to be no reason to change this, especially as the geographic area expands. Eventually, it should be possible to assign one to each of the four Municipalities in the entire zone, with the Technical Director providing supervisory support and coordination between all of them. The current size of the team appears adequate to achieve the project’s objectives if their roles are somewhat modified. At the present time, team members are spending the majority of their time in communities training women’s groups in preparation for CHV selection and training. The team felt that this was important in order to develop methodologies and test training materials. Another large percentage of time is dedicated to supervision of nurse auxiliaries’ IMCI activities in 14 health facilities. However, the team is currently working with women’s groups in 31 out of 140 target communities, and in 14 of nearly 38 health facilities in the project zone. If the project is to achieve the desired impact on the proposed population of over 100,000, it is clear that activities will soon have to be scaled up. Roles will necessarily have to change. Once the team is active in the entire project zone, each member will be responsible for supervision of about 20 facilities per month. If facilities can be reached on average in one-half day (some are closer, some farther), supervision should occupy about half the time of team members, a workload that would seem reasonable. The rest of the time will be occupied by other activities, including planning and coordination, training, BCC/IEC, monitoring, and evaluation. Recommendation: The MCDI CS team should now be shifted upward, from direct trainers at the community level to training of trainers (nurse auxiliaries), supervision of health facilities and community activities, finalize materials, monitoring, and evaluation. In this way, project activities can be scaled up to cover the entire four-Municipality zone as proposed. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 63 Another recent change from the DIP was hiring of a person to collect the data for the cost study. It was originally hoped that Núr University graduate students would be able to carry this out, but this was not possible. The person hired is also specialized in health economics and has experience in project preparation and proposals, which should strengthen the capacity-building aspects of the project significantly. As presented in the DIP, the project staff have directly supported and actively participated in the TAIs as well as in the supervisory visits of auxiliary nurses. The CSST is responsible for providing backstopping and direct supervision to the field. The acting CS Coordinator has made three supervisory visits to the project. In 1999, the CS Coordinator made his first project visit during the initial start-up period of project implementation. In May 2000, the CS Coordinator made a two week supervision visit to discuss the roles and relationships of project partners. This visit was instrumental in developing contacts with the PROSIN and UNAP personnel in the MOH in La Paz and with Núr University. Finally, a third supervision visit was made in June 2001 so that the CS Coordinator could assist in launching the IMCI and HH/C-IMCI training plan, providing guidance in IEC materials production, and strengthening the partnership with Núr University. Other technical and supervisory visits have been made by MCDI personnel to include Dr. Anne Keith, MCDI Board Member and Professor of Nursing and Dr. Jane Gardner, Professor of Maternal and Child Health from the Harvard School of Public Health. The MCDI Health Economist, Dr. Schwabe, has also provided supervisory support to the field team in carrying out health care cost study. Finally, MCDI headquarters has weekly telephone contact with both the Project Manager and Technical Director. 4. HUMAN RESOURCES AND STAFF MANAGEMENT Due to the remoteness of the project area, the CS Project was initially challenged with high staff turnover. MCDI has overcome this constraint by mobilizing personnel in the District and Department of Potosí. Most project team members are originally from Cotagaita or the surrounding area, and all are fluent in Quechua, the local language. All technical team members have previously worked for the MOH either in the project zone or elsewhere, and are therefore knowledgeable about its policies and protocols. The project utilizes the MCDI Policy Manual as a general guideline for personnel polices and takes into consideration the Bolivian labor laws and regulations. All personnel have a formal contract and TOR’s that specify their functions and responsibilities, and they are aware of these responsibilities. In addition to the daily contact between staff members and the Project Manager, weekly staff meetings take place to review activities and schedules, to solve problems identified by staff members and to identify the main activities for the following week. Once a month, a more formal staff planning and review meeting takes place. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 64 Staff morale is high, and their close cooperation and dedication to the project were evident throughout the period of the evaluation. In addition, the team’s respect for the leadership and guidance provided by the Project Manager was obvious. MCDI recognizes that regular formal evaluations of project personnel should be instituted to help redirect staff members, identify areas in which additional training should be provided, and adapt tasks to revised work plans. Each member of the technical staff are from the region and previously worked for the MOH in some capacity. This should greatly ease their transition back into the workforce after project termination. Indeed, the most important factor for a successful transition seems to be more dependent on their political party affiliation and the outcome of upcoming elections than anything that MCDI could do to help them. Their experience with the project, willingness to work in this relatively isolated zone, and training received throughout the project will doubtless make it easier for them to find employment after the project ends. 5. FINANCIAL MANAGEMENT The financial management activities carried out by the Child Survival Project are consistent with the activities outlined in the DIP. At the home office, MCDI has a financial management system that is overseen by our Chief Financial Officer and is regularly audited. The financial management of the project funds is the shared responsibility of the field team and home office staff. In the field, the Administrator/Financial Manager has the responsibility for implementing MCDI management protocols for field accounts. This protocol consists of a manual that outlines standard procedures for tracking all field expenditures and income fund transfers authorized by the Project Manager and the Director of the Division. All program expenditures will be entered into an Excel spreadsheet and aggregated monthly. Monthly financial reports summarizing the results will be submitted by the field-based CS project Administrator to the MCDI Administrator in the Washington DC Headquarters. The CSST in Washington will review and forward the monthly reports to the office of the CFO in Augusta, Maine. Field expenditures are tracked using a coded chart of accounts that corresponds to project grant line items. All accounts are reviewed and entered into an automated system and later merged with the main corporate system in Maine. In the field office, MCDI operates a local expenditure account and only the Project Manager can authorize payments. Comprehensive financial reports are routinely compiled and forwarded from the field office to the Washington office where they are reviewed. If necessary, queries are sent to the field for clarification. The Financial Management Information System is an integral part of the HMIS at the field level. The Project Accountant and Project Manager collect information on a monthly basis. This information includes project expenditures, supporting receipts and documentation, and the monthly request for funds. At the field level, financial logistics are complicated by the fact that there are no banks in any of the four Municipalities, though the field team has adapted to this reality well. One project constraint already described above in the sustainability section is the fact that all activities are largely dependent on a single source of funding, the USAID CS grant. Furthermore, MCDI USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 65 headquarters has few resources from non-restricted sources. A significant proportion of the matching funds come from accounting for expenditures of local counterparts, including the MOH. This limits MCDI’s flexibility to meet unplanned or unexpected resource needs with supplemental funds from other sources. This is particularly difficult for a first-round project. A recommendation for the field staff to work to secure supplemental funding from sources available in Bolivia appears above in the sustainability section. The project appears to be approximately on-target with expenditures, neither over nor under budget. The unanticipated need for more staff, offices, and vehicles than originally planned may strain the project's budget. Consequently any plans that would generate increased expenditures would have to be weighed carefully. 6. LOGISTICS The logistical challenges to the project cannot be underestimated. Cotagaita is at least eight hours from Tarija and four to six hours from Potosí, all on Andean dirt mountain roads. Some health facilities are as much as eight hours from the Municipal government seat, and some villages as much as five hours walk from the nearest health facility. Electricity is available only during office hours in Cotagaita (and then, only in limited offices). The nighttime generator burned out shortly before the evaluation and will not likely be repaired. This poses a major challenge to routine administrative and managerial procedures that rely on computers. The support of the district authorities has helped to mitigate the problem but it is still a significant constraint. The acquisition of a small generator is being pursued and is in the stage of procurement. Cotagaita is scheduled to be linked with the national electricity grid later this year. The other three Municipalities have 24-hour electricity. Telephones are available in Cotagaita and Puna, but Vitichi has only two lines, one public and one in the Municipal offices. Tenuous connection to the Internet via long-distance calling is possible from Cotagaita. It appears that at least half of the communities with health facilities have at least one telephone line, and more are being installed each year. Vehicles The project acquired a four-wheel drive Toyota at the project start-up. Due to the poor road conditions, the performance of the vehicle has been seriously affected and is beyond the point of repair. The Project Manager and CSST are working to procure another vehicle and one motorbike in order to meet the pressing needs for community outreach activities, and the Project Manager has been frequently lending his personal vehicle for project activities. In addition, the project has assisted the district health authorities with maintenance repairs for their vehicle, though this effort has not been very fruitful, as the vehicle (originally a USAID donation) is very old and in very poor condition. In Vitichi, the mayor authorized the MCDI nurse to utilize a motorbike to visit communities that would otherwise be difficult to reach, though it is available only intermittently. The District has a single working vehicle that also serves as an ambulance and is available only intermittently for outreach activities. Area supervisors have no vehicles, and each Municipality has a single vehicle. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 66 If the CS project is to have the desired impact, the need for reliable transportation cannot be stressed enough. For example, during the eleven days spent in the field for the evaluation, the evaluator spent a total of 40 hours in vehicles in transit. The project has five technical staff members plus the administrator all engaged in regular outreach activities, in addition to the need for the Project Manager to frequently visit Potosí for coordination meetings, TAIs, banking, purchases, etc. The project’s single vehicle is simply insufficient. The team related that on numerous occasions they have had to resort to high-cost taxi rental to reach communities, and that scheduled community activities have been cancelled due to problems scheduling transportation. This has reportedly led to a loss of credibility of MCDI and its team at the community level in some instances. Whereas motorcycles are a low-cost solution, they are not appropriate for long distances and cannot be used for joint supervision, an activity that will become increasingly important for institutional strengthening. Motorcycles are also of limited usefulness for transportation of equipment and supplies. As the project begins its planned expansion into the second half of the zone, more vehicles will be needed. Recommendation: MCDI should make every effort to provide the field with one more vehicle. The minimum requirement would appear to include at least one more reliable vehicle plus one motorcycle. This, together with very detailed programming and close coordination with Municipal and District vehicles should allow the project to fulfill its objectives. Computer equipment The project has received a laptop and five desktop computers from the home office. One of the computers was given to the district health authorities in Cotagaita to support their effort to have their HIS installed in a computer, which has occurred. Due to energy fluctuations one of the remaining computers was damaged beyond repair. MCDI will replace this computer, and may need to supply one or more units as the project expands into the larger zone and opens more offices. Offices and location Expansion of project activities will have to be tempered by the availability of project resources to support the expansion. If the expansion into Puna health district is mainly for training and limited community interventions will only occur in the municipality of CaizaD, then a new office in Puna will not be needed. Whereas Cotagaita has the advantage of having telephone lines, problems with electricity and the relative isolation are serious disadvantages (Cotagaita is the farthest Municipality in the zone from Potosí). In addition, the Project Manager’s family resides in Tarija, a result of the original proposed arrangement with Esperanza/Bolivia (whose headquarters is in Tarija). Cotagaita is the closest Municipality in the zone to Tarija, but still eight hours distant, and the Project Manager spends a significant amount of time going back and forth. This not only cuts down his time with the team and in the zone coordinating with counterparts, but also reduces time spent in coordination with Potosí, four to six hours distant in the opposite direction from Tarija. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 67 7. INFORMATION MANAGEMENT The DIP outlines the HIS to be used in the project and stresses the fact that MCDI will not introduce any new systems, but rather, will work to strengthen existing systems. This decision is appropriate, as it makes the project more sustainable. Nonetheless, some new systems are inevitable, and are actually specified in the DIP, including the possible use of LQAS and GIS for monitoring purposes. In addition, as a new level of health worker is being introduced (the CHV) and new activities and responsibilities are being delegated to auxiliary nurses (community education and supervision of CHVs), some form of monitoring and evaluation will be needed, and the introduction of some new systems seems inevitable. Health facilities and SNIS The Bolivian MOH’s SNIS is fortunately an excellent HIS. It was designed with the help of considerable outside technical assistance and has been in place for over ten years with only minor modifications. The system is relatively agile and, unlike systems in most developing countries, it actually has provisions and policies that encourage local analysis of data at each level and regular and timely feedback from higher levels back down to lower levels. Among the indicators tracked by the SNIS, the following are particularly relevant to the MCDI/CS project: Weekly epidemiologic surveillance form Cases of vaccine preventable diseases (<1y, 1-4y) · rashes (suspected measles) · pertussis · neonatal tetanus · diphtheria · flaccid paralysis Diseases treated classified by system: · Respiratory diseases < 1 year, 1-4 years · Digestive diseases < 1 year, 1-4 years Obstetrics: · Hemorrhage during first half of pregnancy · Birth in adolescent (<20 years) · Birth in older woman (>35 years) · Delivery in multiparous woman with previous death of child · Delivery in multiparous woman without previous death of child Nutritional status in children < 2 months and 2 months to 5 years by classification (AB, C, D, E, F) Mortality in the facility: maternal, newborn <7 days, < 5 years due to diarrhea, < 5 years due to pneumonia, < 5years from other causes, malaria, > 5 years all other causes Mortality outside the facility: maternal death, others Suspect of reportable diseases by age (<1y, 1-4y, 5-14y, 15-39y, >=60y) Measles, rubella, pertussis, diphtheria, polio, meningitis due to H. influenza USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 68 Diarrhea <1y, 1-4 years Pneumonia <1y, 1-4 years Monthly activity report Number of ambulatory consults (by sex): New < 5 years, return visits < 5 years Total number of patients referred to other facilities Prenatal consults: new before 5th month, new after 5th month, return visits, women with 4th or higher consult, high-risk detected, high-risk referred New child weighings < 2m and 2m-5y, return weighings < 2m and 2m-5y Pregnant women receiving 90 tablets of iron Post-partum women receiving 90 tablets of iron Children < 5y with 1st dose of iron Children < 5y with 3rd dose of iron Post-partum women receiving vitamin A Children 1-4y with 1st dose of vitamin A Children 1-4y with 2nd dose of vitamin A Number of supervision visits received Number of training sessions attended Number of community activities carried out Number of CHVs, TBAs, promoters active Number of meetings of TAI (by level: local, municipal, district) Deliveries in the facility: vaginal, cesarean, newborns, stillbirths, newborns by weight (<2500g, >2500g), women with first post-partum visit Deliveries at home: attended by health personnel, newborns attended by health personnel, stillbirths attended by health personnel, deliveries attended by trained TBA or other trained person, newborns attended by trained TBA or other trained person, stillbirths attended by trained TBA or other trained person Discharge diagnoses in children < 5y: diarrhea, pneumonia, other Immunizations <1y by dose: pentavalent 1st dose, 2nd dose, 3rd dose; polio doses 0, 1, 2, 3, BCG Immunizations in children 12m-5y: DPT (all doses combined), polio (all doses combined) MMR doses (12-23m); measles doses 24m-5y Tetanus toxoid in women 15-49 by dose (1st to 5th) The most important indicators that are not included in the SNIS relate to diarrhea by stage of dehydration or treatment plan, and cases of dysentery. The absence of these indicators may be partly compensated by using the NHI forms together with essential medication order forms to be used to cross check and supplement the information from the indicators above. Health workers in each facility keep records in a series of books and registries, which are compiled weekly (for the surveillance form) and monthly (for the activity form). Key indicators are also graphed at each health facility according to planning done to estimate desired year-end coverage based on planning by population. Forms are compiled at each progressive level and the USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 69 aggregates are again graphed based on population-based planning. The graphs use the standard “progressive coverage” format most commonly used to monitor progress toward coverage in immunizations. At the District level, data are fed into a computer, which permits more sophisticated analysis. The computer database is sent to the Departmental SEDES for aggregation and analysis, and from there it is sent electronically to the MOH in La Paz. The SEDES prepares printouts of the computerized analysis for the aggregate data as well as for each health facility and sends it back down the chain. With some brief exceptions, the District has regularly shared the SNIS data files with MCDI who has the SNIS system installed on their own office computers. During supervision visits to health posts MCDI staff assist auxiliary nurses in filling out the SNIS forms and prepare for the TAIs. MCDI staff have participated in TAIs as often as possible. At these meetings, coverage rates and trends are discussed, problems identified, and solutions proposed. The frequency of TAIs varies by level, as follows: TAI for health facilities and Areas: monthly X 1 day TAI Municipal level Vitichi: monthly X 1 day TAI Municipal level Cotagaita: monthly X 1 day TAI District level every 6 months X 2 days TAI Departmental level every 6 months X 2 days At the Municipal TAIs, the NHI system data and essential drugs/rotating fund data are presented, crosschecked with the SNIS, and analyzed as well. MCDI is helping supervise the quality of the data in the 14 health posts in which it is currently active, though MCDI is not collecting or monitoring NHI data at their office. Several problems have been identified with the TAIs. As discussed earlier, there are problems with consistent overestimation of the size of the target population, which leads to artificially low coverage estimates for most indicators. Unfortunately, this also leads to a general decrease in the perceived reliability of the information, although trends should be more reliable. This problem should be ameliorated by new census data due out very soon. Another problem is the attitude of supervisory staff during the TAIs, who reportedly often use them more as an opportunity for criticism than for problem solving. Through their participation, MCDI is working to help guide the TAIs toward a more constructive mode. Another problem with the system is the reliability of the data. The HIS at the level of the health post is quite complex and requires a great deal of the health workers’ time. Health workers are sometimes required to register a single consult in as many as five or more different registration books. Underreporting and misclassification inevitably result. Evidence of problems is best seen in the lack of coherence between data gathered through the SNIS and that through the NHI system, with the former always being higher. This problem was discussed above, and is attributed partly to the fact that there are incentives to underreport on the NHI system as “off-the￾books” drug sales provide an opportunity for n nurse auxiliaries to supplement their income. Tighter and more consistent supervision as well as greater involvement at the Municipal level should lessen this source of error. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 70 During 2001, the MCDI/Bolivia team, at the recommendation of their headquarters office and with its technical assistance, introduced a supplementary HIS form to gather data for indicators specific to the project that are not included in the SNIS. The indicators include cases of diarrhea and pneumonia in children under two years of age, as per project indicators, instead of the 0-1y and 1-4y age groups defined in the SNIS. The HMIS instructions indicate that the forms are to be filled out by nurse auxiliaries and collected by the team during supervisory visits. Data are to be entered into spreadsheets in the MCDI office, tabulated monthly and summary sheets sent to Municipal and District authorities as well as to MCDI headquarters. The internal system for tracking these process indicators has been in place for just over one year. Unfortunately, a number of constraints impede its full use. Auxiliary nurses already have a very full workload filling out forms for the MOH and NHI, and both they and the District were not favorable to adding yet another form that would collect data so similar to the SNIS. In addition, even the raw information in the registration books does not contain the age of the children attended, so it was impossible to collect data on children zero to two years of age without modifying the registration books in all the health posts. MCDI staff are obliged to collect the data themselves during monthly supervisory visits, having to extract it from the raw information in to the original consult registration books. The team is collecting data only from the 14 health facilities receiving regular supervision, and so coverage does not reflect true coverage in the entire project zone, which has 38 facilities in Cotagaita district alone, and nearly 70 in the entire zone. The MCDI team stated that it takes them several hours during each visit to collect the data, and that they spend about 10 days per month collecting and entering the data. If the system is already taxing the resources of the MCDI team, it seems unlikely that the team, with limited resources, will be able to scale this system up as the intervention area expands and more HIS forms and systems are implemented (supervision forms, monitoring of community-based activities, CHV forms, and community-based surveillance forms). One last constraint to the successful use of the system relates to problems with using computers. During the evaluation, electricity in Cotagaita was limited to office-hours only (Vitichi, Puna, and Caiza “D” reportedly have 24-hour electricity). During the evaluation, two days passed between requesting the data from the HIS and the team finding time during the day to extract it from the computer for analysis. More importantly, it became evident that no one on the team had been following the information regularly—the evaluation was the first time they had sat down to analyze it. Although page 66 of the DIP specifically states that the project will not use census-based monitoring, the team has expended considerable effort to collect and maintain data-lists of children by name from the catchment areas of the 14 health posts currently being supervised by MCDI. Although this may allow the team to accurately monitor the status of these children, this system cannot be scaled up and is taking an inordinate amount of time to maintain, time that would be better spent on systematic supervision and training. This effort should probably be discontinued. (The field team independently made this decision.) The data in the SNIS, while not perfect, are probably “good enough” for monitoring purposes, especially when combined with other official sources of information, such as the NHI system, essential medications, rotating drug fund, supervision forms, and the soon-to-be-introduced USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 71 community-surveillance forms. For example, trends in coverage for the 0-1y age group and 1-4y age groups included in the SNIS are probably very similar to those for the 0-2y age group being tracked by the MCDI HIS. Furthermore, the SNIS comes with a built-in forum for regular analysis: the TAIs. Recommendation: MCDI should reconsider the current HIS methodology for capturing information in order to spend more time supervising the existing MOH and NHI information systems to improve their reliability, as well as spend more time analyzing the data regularly. MCDI may wish to consider requesting access to the NHI program data, essential medication data, and rotating drug fund data from the Municipalities for internal analysis and reporting. CHVs PROSIN/BASICS has introduced a registration form for HH/C-IMCI activities carried out by CHVs. MCDI is considering introducing this form. The three-page form is mostly pictorial and serves as a guide for contact between the CHV and the caretaker, with simple flow-charts to guide actions and the content of orientation. It includes places to register immunizations, love and attention, breastfeeding and complementary feeding, prevention of illness (hygiene, smoke, garbage), danger signs for under 2 months and over 2 months, ear problems, malnutrition, and cutoffs for respiratory rate and fever by age. Though the forms are simple and have been field-tested in Cochabamba, a number of issues arise regarding their implementation in the project. For example, a tremendous amount of resources will be required if CHVs fill out a report for each contact or visit. In collaboration with health authorities, the project should also establish where these forms will be sent, who will be responsible for compiling and analyzing the information and who will be accountable for producing the forms every month. It may be advisable to consider using the forms simply as a guide for contact, and have the auxiliary nurse and MCDI team adapt it for use as a supervision form. Coverage of the person￾to-person BCC/IEC activity could be measured through verbal reporting by CHVs to auxiliary nurses, which could then be incorporated into the SNIS data. This approach would be sustainable, though a little less precise. The forms themselves will also likely require minor modification to include maternal/neonatal care activities and orientation and danger signs as well. Supervision The MCDI team is in the final stages of testing a supervision form for IMCI, as mentioned above. The form was designed by PROSIN/BASICS and has been adopted as MOH policy. It is simple to apply and easy to tabulate. To date, it has not been systematically applied to all health workers, but MCDI plans to begin shortly. MCDI has been using a standardized IMCI instrument for auxiliary health workers and is currently preparing an integrated instrument that will be used by the project and the MOH for the supervision of auxiliary nurses (and other personnel). The implementation of joint supervision, either with the Districts, Areas, or Municipalities should provide an opportunity to delegate responsibility for compilation to MOH or Municipal USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 72 staff so the activity can be sustained in the long-term. Data should be presented at the TAIs together with that of the SNIS. Supervision of other activities outside IMCI by Municipal, District, Departmental, and Area staff are infrequent and haphazard. Many forms are in use (by the MOH personnel), some very long and complex (one is 14 pages long), and often the supervisor designs the forms ad hoc just before they are used. The data are not being analyzed or presented at the TAIs. Recommendation: MCDI should take the lead to coordinate agreements on integrated standardized supervision instruments for joint supervision at the various levels and work toward their systematic use, compilation, and analysis in the TAIs. MCDI should also explore the possibility of developing job aids based on self-assessment methodologies that could be utilized by auxiliary nurses to evaluate/supervise their activities at the health facility level. A similar model can be prepared for the use by community-based organizations. The utilization of self￾assessment tools will help to overcome the difficulties in maintaining the regular face-to-face supervision. BCC/IEC BCC/IEC activities have not been fully launched so systematic monitoring of BCC activities is not occurring yet. As mentioned earlier in this report, MCDI is testing an instrument for supervising group education activities. The team will apply this instrument to auxiliary nurses who are training CHVs as well as to themselves. (MCDI will strive to integrate this instrument with other supervision instruments and will make it as simple as possible. MCDI will explore the possibility of simplifying this instrument to be used as a job aid for nurse auxiliaries.) The results should prove very useful in counseling and guiding individual workers to improve the quality of their group education activities, but probably would have little use if compiled. Monitoring progress toward change in knowledge and behaviors of caretakers and communities will be even more important. The team is only beginning to tackle this issue and may need outside technical assistance, perhaps as part of the recommended BCC/IEC planning workshop mentioned in the BCC/IEC section above. The team should choose a small number of key indicators according to the projects behavior￾change objectives, perhaps one indicator which reflects knowledge and another reflecting behavior change from each of the four interventions (for a total of about eight questions). A representative set could include card-coverage for measles vaccine in children 12-23 months of age, maternal knowledge about the proper age to vaccinate against measles, continued feeding during the previous episode of diarrhea, danger signs for dehydration, danger signs for pneumonia, card-coverage for vitamin A, care-seeking during the last episode of difficult breathing, documented prenatal care for pregnant women, and danger signs during labor and delivery. Such an instrument would take little time to fill out by randomly interviewing caretakers, whether by a nurse auxiliary or a supervisor. A quarterly sampling and analysis period would be reasonable, as knowledge and behaviors are not expected to change much more rapidly than that. A set of nineteen randomly selected interviews from the catchment area of each health facility every quarter would satisfy the data USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 73 needs for analysis using LQAS if it is implemented in the future. This would mean that each health facility would need to perform only one random short interview per week. This should not present an undue burden on staff, yet should be sufficient to rapidly indicate problems in achieving impact on behaviors and knowledge by intervention and by geographic area. The data could easily be adapted to the GIS if it is implemented in the future. This may seem like a very small sample, but once scaled up to the entire project zone, it would generate over 800 forms per quarter, over twice the sample size of the original baseline survey. If even this small sample proves too cumbersome, the team could opt for performing quarterly short rapid surveys with a small subset of knowledge and behavior indicators, or use of sentinel sites. These strategies would be less burdensome, but would also eliminate the ability to pinpoint specific health posts or geographic areas for targeted intervention when problems arise. Recommendation: A very simple system for monitoring changes in caretaker knowledge and behaviors should be designed and deployed as early as possible. The system should be put in place with the permission and participation of the MOH and/or Municipalities, and data should be analyzed during TAIs in order to help institutionalize the process. The renegotiation of MCDI’s agreements with the MOH and Municipalities would provide an opportunity to present this idea. Suggestions are made in the report regarding how such a system may operate. LQAS The DIP mentions using LQAS for monitoring purposes. This sampling method would prove useful especially in monitoring the BCC/IEC activities as described above, though it could also be applied to monitoring of activities of CHVs, whose large numbers would make any reporting system that attempts to monitor all of their activities too cumbersome to be scaled up and sustainable. MCDI will need external technical assistance to implement LQAS, though perhaps this could probably be obtained through PROCOSI/Núr in-country and need not take long. GIS The DIP also describes the geographic information system as a useful tool for monitoring and evaluation, and highlights MCDI’s particular expertise in its use. The project has begun to collect GPS information on the location of health facilities in Cotagaita District, which the headquarters CS support team has entered on a map using GIS software. This map was instrumental in the DIP presentation to USAID BHR/PVC. The new CS field team has not yet been trained in the use of the GIS. This technology may prove itself useful for fine-tuning monitoring, and the data from the SNIS and other aspects of the HIS that are already computerized could probably easily be adapted to the system. However, the team has many other higher-priority activities to complete, including getting the joint systematic supervision underway, adapting BCC/IEC materials implementing and monitoring activities and scaling up the geographic scope of the project. If GIS is to be used, it should probably be tried as an experimental method during the last year of the project. Special assessments and studies Cost study The primary health care cost-study has been discussed above under the section on capacity￾building. The study is expected to yield two benefits. It should provide input toward more USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 74 realistic reimbursement rates for primary health care activities by the NHI system. In addition, municipal and district personnel will be trained in how to determine costs in the future by participating in data-collection for the study. Qualitative studies The CS team carried out a series of qualitative studies using focus-group methodology as part of the baseline assessment. The results of these studies were described in the DIP, and allowed the team to focus on the interventions. The results, for example, were partly responsible for changing the intervention mix to include EPI and reduce the nutrition activities. The studies were also used to define key messages for BCC/IEC and training activities. Assessment of logistics and management of supplies, essential medications and rotating drug fund This study has also been discussed above. Data were collected from 15 health facilities during the second half of 2001, and are still being processed. The key results were: · identification of problems with the medication system, especially with respect to ordering · errors in processing orders at the central level · errors in managing the cards at the health facilities · nurse auxiliaries have incentives to mismanage the rotating drug fund and establishing personal pharmacies to supplement their income · problems with financial management of the NHI at the Municipal level Based on the points above, MCDI has elaborated a training, supervision, and technical assistance plan to help improve medication and supply management. The results of the study and resulting plan have yet to be presented to relevant authorities. Assessment of EPI logistics, cards and supplies The team performed a formal assessment in 15 health facilities. They found equipment lacking, especially with regard to the cold chain and its management (no thermometers, for example), health workers are not using cards to manage their stock, logistics problems, especially with supplies of gas for refrigerators (many health posts are given only one gas bottle per month, which is not enough to keep a refrigerator working), and problems with transportation of vaccines. This report is being used as a basis for planning logistical support activities, and should also provide input to help Municipalities prioritize purchase of essential equipment using surplus NHI program funds. The formal report from this study is still being prepared and has yet to be presented to the relevant authorities. 8. TECHNICAL AND ADMINISTRATIVE SUPPORT MCDI does not have a regional office in South America to support the project in Bolivia. However, the Home Office Child Survival Support Team (CSST), comprised of the Project Manager and other technical support staff provided on-going technical backstopping to the project. The project receives regular backstopping and technical support and supervision from the CSST. The CSST convenes at least monthly to discuss project activities and provide feedback to the field. In addition, the Project Manager maintains weekly telephone and email contact with USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 75 project staff. These contacts are useful in providing guidance and recommendations related to issues raised by the field staff; if warranted, memoranda are prepared to clearly delineate home office recommendations. Home office technical support is provided by a host of channels to include field visits by home office technical staff as described above, forwarding technical updates from various sources, e.g., CORE, BASICS, PAHO, UNICEF, etc., to the field staff and securing technical guidance from the CSTS. Regular updates of website contacts are provided as well. The headquarters office has mobilized technical assistance from Dr. Anne Keith, an MCDI Board Member and a Professor of Nursing and Dr. Jane Gardner, a Maternal and Child Health Professor at the Harvard School of Public Health. Dr. Keith visited the project in October 2000 to provide support in the assessment; design and implementation of training strategies to better achieve the outcomes stated in the DIP. The recommendations provided by Dr. Keith focused on the needs of the trainees, the curricula, the staffing model for training and the role of Núr University in the training. One of the most important recommendations was the need to redefine the community agent role and as a result the project has redesigned his approach to community volunteers, such as “vigilant mothers” and mothers groups instead of maintaining work with promoters that no longer were really active in Cotagaita. Substantial technical assistance was provided by the MCDI Health Economist, Dr. Chris Schwabe who prepared the research protocol for the NHI program cost study and made a follow￾up supervisory visit to launch the study, in February 2001. During this visit the Health Economist has also conducted a training workshop in Santa Cruz as described in the section on Training. In addition, the headquarters Project Manager has made three field visits thus far for supervision and technical assistance as outlined above in the section on supervision of program staff. On an ad hoc basis the MCDI headquarters CS technical team receives support from an expert in Maternal Health, Ms Mona More. MCDI has also received technical guidance from the American College of Nurse Midwives that is expected to impact on the maternal and neonatal health intervention. Finally, the project received a short visit from Dr. Jude Pansine from MCDI/Washington in December 2001 with the aim of improving field reporting and to prepare for the mid-term evaluation. The evaluation team identified the following needs for future technical assistance: -preparation of the BCC/IEC plan and monitoring tool (may include LQAS) -GIS -community-based surveillance (assistance will probably come through SEDES) -project design and proposal writing USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 76 D. OTHER ISSUES None were identified that have not already been discussed. E. CONCLUSIONS AND RECOMMENDATIONS The design of MCDI’s Central Potosí Child Survival project is well thought out. The intervention mix appears to be appropriate to the priority needs of the region and the partners. The initial preparation and assessment phases were careful and complete, setting the stage for the design and modification of plans, systems and materials that address the specific problems identified. The current CS technical and administrative field team, while new, appears to be well￾trained and experienced, cohesive and enthusiastic. Team members are well-informed about both technical issues related to project implementation, the goals, objectives and activities of the project itself, and local cultural characteristics, strengths and barriers. MCDI has formed productive relationships with government entities and communities, though the relationship specifically at the District level has, at times, been strained. The relationship at the critical Municipal level, however, has been especially productive. Finally, MCDI has been successful in identifying and harnessing existing resources to benefit the project, including PROSIN’s clinical IMCI course, PROSIN/Núr’s CC/H-IMCI course, CARE for training in humanized delivery, and MOH training in EPI, SNIS, NHI, and essential medications. The early stages of the project were plagued by an unusual number of unexpected setbacks, and MCDI has proven itself flexible and resourceful in its efforts to compensate for them. The original partnership with Esperanza/Bolivia was dissolved and the partnership arrangement was redefined to focus on SEDES and the Districts. When high turnover led to unproductive relationships, MCDI was obliged to rethink its partnership arrangement again, shifting focus to strengthening the Municipalities. The project has also been plagued by civil unrest, which has impeded access to communities. Several potential partners and resources outlined in the DIP have not been as helpful as originally anticipated or have not been able to provide assistance at all. This has placed an unexpected burden on the CS team to design and execute the activities themselves. The number of community health promoters was far lower than anticipated, prompting a shift in strategy toward working directly with women’s groups. Whereas this change in strategy will likely increase the effectiveness and sustainability of the project in the long run, it has greatly increased the time needed for community level training several times. This has prompted the need to expand the size of the team from the planned two technical staff members to five technical staff members. After the relationship with Esperanza was dissolved, MCDI also faced difficulties recruiting a competent and stable team in this isolated region. The project is significantly delayed in project implementation as outlined in the revised DIP, though delays are not uniform. Some activities are almost on schedule, including training in clinical IMCI, which is nearly complete in the Cotagaita district and could be completed in Puna this year. IMCI protocols are being applied in most health facilities in Cotagaita. In addition, health worker training in maternal/neonatal care is nearly complete, lacking only emergency obstetrical care for health providers. However, training of community health volunteers is only about 25% complete for the Cotagaita district alone. Although the curriculum for training USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 77 auxiliaries in CC/H-IMCI is nearly complete (lacking only integration of content on maternal/neonatal care), training materials for this course are still in development. The BCC/IEC plan needs refinement, and some materials are still being identified. To date, systematic BCC/IEC activities are limited to counseling during consults and health fairs. Joint supervision of clinical IMCI is still not in place. There is some evidence for improved quality of health services and increased utilization. Immunization coverage is probably higher than originally estimated. Essential medication stocks are regular, and IMCI protocols are being applied in most consults where workers have been trained. The maternal/neonatal care intervention has been implemented inasmuch as humanized delivery has been implemented at health facilities and demand is up. However, the staff have not yet been trained in emergency obstetrical protocols. Administration at health facilities under MCDI supervision has reportedly improved. MCDI has been assisting health workers to improve the reliability of data in the HIS and has participated in MOH data-analysis workshops. In light of the change of project direction, to focus capacity building at the district and municipal level, it is understandable that little focus has been placed in the capacity building of SEDES and MOH. It is important to highlight that the NHI cost-study is well underway. In addition, overall project management and the current staffing levels appear to be adequate. However, the number of vehicles available is not sufficient to allow the staff to fulfill the objectives. The CS project should be able to fulfill its objectives, but only by making some modifications in the way the team is currently working. Most importantly, the team must work quickly to finalize several project activities (BCC/IEC, HH/C-IMCI training for auxiliary nurses, obstetrical emergency course, supervision systems). These must then be implemented and scaled up in Cotagaita and prioritized areas of Puna. The following are the most important recommendations as determined by the evaluation team, grouped by themes: ÿ MCDI should renegotiate its agreements with SEDES, Districts, and Municipalities as soon as possible. The team should strive to include the following terms: · regular and consistent access to information from all health information systems and the NHI program (and regular sharing of MCDI information with MOH and Municipal authorities) · a commitment to regular standardized joint supervision of health facilities using mutually agreed-upon instruments · commitment from SEDES to assent to the wider role of the nurse auxiliaries in training and supervising community health volunteers · actively pursue contacts with the PROSIN community health division (DIDESCO), as well as CIELO at the SEDES level for activities related to MNC, HH/C-IMCI and IMCI. ÿ The MCDI team should strive to complete the adaptation of the following critical materials and systems as soon as possible: USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 78 · the BCC/IEC plan, support materials, and a simple monitoring system. External technical assistance in BCC/IEC will likely be necessary for this step. The monitoring system may require assistance from MCDI/Washington if LQAS is to be used as a sampling method. · the training program and accompanying materials for training of auxiliary nurses and other staff as facilitators for HH/C-IMCI and maternal/neonatal health (with assistance from MNH and an external consultant). · integrated supervision instruments for clinical IMCI and maternal/neonatal health. · define the exact role, responsibilities and estimated time commitment of the CHVs and community health committees; reporting requirements and monitoring/supervision system. Suggestions are presented in the body of the report. In order to accomplish the above, the team may have to temporarily halt community-training activities in order to concentrate on finalization of these materials and systems. Completing all of the above should not take much more than a month if the team works together. ÿ Complete training at the following levels immediately or as soon as soon as materials are ready · clinical IMCI for the remaining health workers in Cotagaita. (This is in the process of being done in March-April 2002) · training as HH/C-IMCI facilitators for all health workers as soon as materials are finalized · locate appropriate resources for training of all health workers in Cotagaita in protocols for management of obstetrical emergencies, including prenatal, delivery, post-partum, and neonatal emergencies. (MCDI has already established contact with MNH and MOH to begin TOT training on emergency obstetric care in Tupiza in April/May 2002.) If appropriate materials are available, all of the above training should be completed in Cotagaita in less than two months. ÿ Begin joint systematic supervision of all health facilities in Cotagaita as soon as renewed agreements are in place and mutually agreed-upon instruments are available · clinical IMCI and maternal/newborn health · NHI system, essential medications, and rotating drug fund · community-based education activities (team to team and team to auxiliary nurses) · community-based epidemiologic surveillance system (once it is in place later in 2002) · results of supervision should be presented in the TAIs This activity should occupy only about one-fourth of the team’s time for all of Cotagaita, and one-half of the team’s time once Puna is also included. ÿ Through a phased approach delegate training for women’s groups, community groups and CHVs and supervision of CHVs to auxiliary nurses once they have received HH/C-IMCI facilitator training. MCDI will be maintaining initially a close support to the outreach activities of the nurses consolidating the results in organizing and training the community volunteers and implementing the HH/C-IMCI interventions and progressively moving for new communities. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 79 This could begin immediately after completion of HH/C-IMCI facilitator training within the next 60 days. All communities in Cotagaita district could complete their community level HH/C￾IMCI training within 12 months from then. ÿ Develop a realistic joint capacity-building plan with Municipal authorities as soon as possible after renewal of agreements · present the results of the medication and supply studies as soon as possible and use it as the basis for planning · emphasize improving the functioning of the SNIS, NHI program, and essential stock, medication management · develop some simple indicators and a means to measure progress ÿ Improve and expand logistical and equipment support to allow expansion and scaling up of the project · increase the number of vehicles. The project will likely need at least one additional vehicles and one motorcycle if it is to be able to cover the vast area. · establish a liaison office in Potosí · purchase basic safe birthing kits for home-based care use once the MNC training for obstetrical emergency care has been finalized; continue to pursue funding from the Japanese Embassy for hospital equipment for the ObGyn department of Cotagaita Hospital · supplement immunization and maternal health cards as needed ÿ Initiate contacts and activities with Puna and Caiza “D” · establish agreements with the Districts, Municipalities, and local PVO partners · schedule training in clinical IMCI for health workers as soon as possible. This could begin almost immediately after agreements are signed. · evaluate the feasibility to relocate two staff members and appropriate resources for their support from Cotaigaita district to Puna district some time in the next nine to twelve months to begin replication of all activities If training of selected health workers as HH/CC-IMCI facilitators can be completed in Caiza “D” by the end of this year, it will be possible during the next year to develop community interventions in Caiza “D” Municipality. In this way, it would be possible to achieve the project’s goals and objectives in the time remaining. A list of all the recommendations in the order that they appear in the body of the report appears as follows: IMMUNIZATIONS Recommendations: Two important recommendations resulted from this evaluation: The team should make every effort to verify true immunization coverage as soon as possible. This could be done rapidly by performing home visits during regular supervision visits, taking USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 80 the health post’s copy of the cards to compare with the copy in the home. It should soon become apparent whether discrepancies in record keeping between the two copies are common or not. MCDI should go ahead with its plan to supplement maternal and child cards to guarantee their regular supply. As long as supplies are irregular, it is impossible to obtain good record keeping, and it will be impossible to create a “culture” of valuing the card among both health workers and caretakers. The cards empower caretakers and communities. The issue of reduced sustainability is less important than the benefits gained by improved monitoring, and helping families and health workers understand the cards’ value. MCDI can work to normalization MOH supplies in the meantime. Recommendation: The MCDI team should not forget to develop at least one message (and supporting visual aids) each regarding vitamin A and tetanus toxoid immunization to supplement the materials lacking in the existing community IMCI materials. DIARRHEA Recommendation: The team should keep in mind that, in the event of intermittent short supply of ORS, priority should go to health facilities equipped to rehydrate dehydrated children (plan B) rather than distribution to mothers to use to prevent dehydration (plan A). As the first step to prevent complications and to overcome ORS shortages the team should maintain the education of CHVs in the use of home-based rehydration liquids. PNEUMONIA Recommendation: MCDI should work with the auxiliary nurses to develop community outreach schedules for two reasons: 1) to maximize the effectiveness of their outreach activities and 2) to minimize the negative impact of their absence from the health posts. MNC Recommendation: MCDI should carefully review and evaluate the feasibility of adapting the Municipality/Family educational material for training CHVs as well as for BCC/IEC activities. Recommendation: De-emphasizing TBAs seems a reasonable decision Nevertheless, where TBAs are active and accepted by mothers, MCDI could explore the possibility of training them in HH/C-IMCI including the MNC component. Recommendation: MCDI should move quickly to coordinate with MNH and the MOH to arrange plans and schedules for technical training of a TOT team in obstetrical and neonatal emergencies. The Municipality of Vitichi included funds in its 2002 budget to support reproductive health training for health workers with PROSIN. MCDI should urgently investigate this possibility as well. Recommendation: In all but the most inaccessible areas, MCDI should target messages and training toward encouragement of deliveries by MOH health personnel, at a facility if possible, or in the home. MCDI may need to tailor its community MNC training and BCC/IEC to the USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 81 situation/problem of each individual community, while encouraging institutional deliveries in the majority of communities who have reasonable access to facilities. In communities where access is limited, community awareness and birth preparedness should be stressed. IMCI Recommendation: In order to not jeopardize the valuable investment in IMCI training to date, MCDI should begin applying the systematic IMCI supervision tool to all trained personnel in all facilities. The team should also strive to engage MOH personnel at the Muncipal and/or District levels whenever possible to participate in the supervision. The recent suggestion by MCDI to encourage the municipalities to support a full-time municipal nurse supervisor with their own resources seems like a reasonable approach. Recommendation: MCDI should encourage the final training of the remaining eight health workers in Cotagaita district in clinical IMCI as soon as possible (these 8 health workers are in the process of being trained). Recommendation: As the project does not have the luxury of waiting any longer for Núr University to finalize and release the materials, MCDI should work with PROCOSI/CORE HH/C-IMCI initiative to complete the development of the HH/C-IMCI course for facilitators and health workers and carry it out as soon as possible. These steps should be taken so that the project can move forward and be scaled up to achieve coverage. (MCDI has already begun to address this limitation by contracting a consultant who will assist the project team in finalizing this curriculum.) Recommendation: It would seem reasonable to integrate maternal/newborn topics into the course for community IMCI (both the courses for heath workers and community members) rather than trying to organize a separate training course. Recommendation: Both the team and mothers agree that the training of mothers’ groups and CHVs will be more effective if some sort of material can be left behind in the community. The project is expecting CHVs and caretakers to memorize a long list of danger signs, yet no memory aids have yet been developed for use by CHVs and/or mothers. One good start would be to look carefully at the Municipality/Family materials, which are already present in communities. Recommendation: A brief baseline assessment of the degree of application of appropriate IMCI and MNC protocols by health workers in health facilities before training health workers in the expanded zone would provide better documentation of the project’s direct impact on the quality of services. (This should be done in collaboration with the MOH). Recommendation: MCDI needs to finalize the HH/C-IMCI materials initially developed by Nur/PROSIN and should be done through technical assistance because the project cannot wait for Nur to finalize this. The evaluation team was unanimous in their feeling that it will be possible for the Núr/PROSIN HH/C-IMCI methodology to be scaled up and made more cost￾effective by adapting the methodology to the reality and constraints of the region and the MCDI/CS project. This will involve immediately training nurse auxiliaries as HH/C-IMCI USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 82 facilitators and delegating training responsibilities to them, while simultaneously developing and implementing an intensive system of field supervision, which will allow the team to support this change. In this way, CHVs and community groups in many communities can be trained simultaneously with minimal degradation in quality. COMMUNITY MOBILIZATION Community organization/women’s groups Recommendation: The MCDI team should develop some simple criteria for measuring the level of community organization that they wish to achieve and include these goals and methodologies in the training course of HH/C-IMCI for nurse auxiliaries. Minimum criteria may include the formation of an active women’s group and health committee in each community. In addition, as it will be increasingly difficult for the team to informally monitor the success of community organization and mobilization activities as the project scales up to reach more villages, development of simple objective criteria for measuring the level of community participation and organization would be useful. In the future these could be mapped into the planned GIS if MCDI so desired. In this way, team and Municipal/District can target their supervision to those communities requiring the most attention. Recommendation: As the establishment of health committees appears to be a strategy that the team is interested in pursuing, MCDI should also work with those communities with the most active and organized committees (such as Tocla). MCDI can draw on their experience and jointly develop a description of specific responsibilities and activities for health committees so these can be included in training for auxiliary nurses. Decisions should be made about the committee’s composition, whether health committees will file reports, indicators on those reports, and to whom they will file them. This must be done as soon as possible so auxiliary nurses can be trained and community activities can be delegated to them. CHVs Recommendation: In collaboration with nurse auxiliaries and municipalities, the role of CHVs needs to be clearly defined to include their reporting requirements, supervisory mechanisms and relationship with nurse auxiliaries.. Recommendation: MCDI should make every effort to coordinate municipal efforts to support the training of 50 "health promoters" with the CS project. Community-based epidemiologic surveillance Recommendation: The MCDI team should investigate the community-based epidemiologic surveillance system to be implemented by SEDES this year and support this effort. Selection of community members as candidates for training has already begun. As much as possible, MCDI should immediately begin a dialogue with the district to coordinate the selection process for trainees with their own efforts to select and train CHVs. COMMUNICATION FOR BEHAVIOR CHANGE USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 83 Recommendation: The team should continue to give priority to targeting messages to health workers, CHVs, caretakers, and communities as a whole. Other targets (traditional healers, rural cooperatives, TBAs) may receive attention according to individual need and opportunity. For example, TBAs who are operational in the project area may be provided with MNC training based on HH/C-IMCI strategy. Recommendation: MCDI should consider reproducing laminated cards and flipcharts for each community in order to enhance the effectiveness of the CHVs’ interpersonal communication activities. In addition, it would be advisable for MCDI to begin developing a supervision instrument for interpersonal communication that can be applied by MCDI staff and nurse auxiliaries. Use of this instrument could be included in the HH/C-IMCI course for nurse auxiliaries so that they can begin to practice its use with CHVs. MCDI needs to work with the HH/C-IMCI working group and with nurse auxiliaries and mother groups to prioritize the BCC campaign focus on those behavioral change interventions that will have the greatest impact. Recommendation: For the purpose of quality assurance, the team should be encouraged to implement the group education supervision instrument as quickly and widely as possible. The current four-page form is somewhat lengthy and may eventually require some simplification. The possibility of using job aids for this activity should also be explored. Monitoring Recommendation: The team should not wait, but should move ahead to develop and implement a streamlined tool to monitor caretakers’ knowledge and practices. A very simple tool and a very small but consistent and regular sample should provide useful results. For example, a nurse auxiliary could apply a form with eight or ten key questions adapted from the KPC to one randomly selected mother at each health post once each week. If tabulated and monitored, this would provide very useful information. Adaptation to LQAS is possible later, if technical assistance can be arranged. Recommendation: After the gap analyses, key messages were prioritized however there still appears to be an excessive amount of messages to be memorized by caregivers. Therefore, key messages should be reassessed and reprioritized. The evaluation team unanimously felt that outside technical assistance would be beneficial to assist the team in refining a detailed written IEC plan. This would probably best take the form of a workshop focusing on the project’s specific plan rather than IEC in general, and could draw upon in-country resources discussed during the evaluation. (MCDI is in the process of addressing these issues by contracting a consultant who will support the team in improving the BCC strategy and assist in prioritizing key messages using the BEHAVE model as well as resources from MNH, MotherCare and PROSIN) CAPACITY-BUILDING LOCAL PARTNERS Strengthening the Local Partner Recommendation: In order to increase awareness of the project, the evaluation team agreed that increasing the volume of written communication, reporting, and general information to all partners, including Municipal governments, SEDES, Districts, Areas, and key staff would be USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 84 beneficial. This will help to improve awareness of the project as well as maintaining good public relations. Recommendation: In order to make it possible to implement joint supervision to improve the quality of supervision and help guarantee sustainability, MCDI may want to explore options, such as providing non-financial incentives for MOH staff in order to guarantee their participation. Recommendation: The evaluation team wholeheartedly supports the shift in emphasis in capacity-building away from SEDES and the Districts and toward strengthening Municipalities. This is especially important in strengthening of management, with special attention to management and supervision of the National Health Insurance program, the rotating drug fund, essential medications, and the Municipal Health Committees. Recommendation: MCDI should follow-up on the discussions to encourage the Municipalities to hire a designated nurse supervisor and, if successful, work to strengthen that person’s supervisory skills through joint supervision. Recommendation: MCDI should continue its efforts to support the newly formed Municipal Health Committees (COMUSA) through active participation in their meetings, planning, and evaluation. The COMUSA’s should probably become the chief coordinating bodies between the project and partners. Recommendation: The project should eliminate the capacity-building objectives with respect to Núr University, and concentrate on strengthening the Municipalities. If these objectives remain, however, probably the best way to measure whether exposure to a new methodology or procedure (such as LQAS, GIS, or cost-analysis) is whether the university has incorporated it into its curriculum and taught it to others. Strengthening Health Facilities Recommendation: MCDI should present the results of the drug management study as soon as possible in order to have time to implement its plan to further improve the system. Support the elaboration of statistical reports for the data-analysis committees (TAIs). MCDI should explore the possibility of coordinating with the American missionary physician for activities related to training of nurse auxiliaries and the management of the rotating drug fund. Strengthening Health Worker Performance Recommendation: MCDI should renegotiate its agreements at all levels to ensure the implementation of joint systematic supervision of health worker performance. This includes agreeing on standardized instruments, schedules, and how and who will use the information. In addition to auxiliary nurses, supervision of health professionals in hospitals, using integrated IMCI/MNC protocols, should also begin as soon as possible. SUSTAINABILITY USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 85 Recommendation: It is recommended that MCDI should be encouraged to submit an extension application, which will fully incorporate all of the geographical areas of the Linares province (Puna and Caiza D Municipalities). Recommendation: MCDI/Bolivia must not forget to look after itself as well. It should begin as soon as possible to identify sources of funding for other related projects and sources of matching funds for the Child Survival project. With greater diversity of funding, there will be a better chance of sustaining funding. Although MCDI is not a member of PROCOSI, its contact through Núr University provides an opportunity to keep abreast of opportunities for future funding. MCDI should also strengthen its contacts with PROSIN (DIDESCO) in order to procure funds for health facility upgrading in the project area as well as for IMCI and MNC activities. Japanese funding and World Bank funding should also be pursued. SUPERVISION OF PROGRAM STAFF Recommendation: The MCDI CS team should start a gradual shift upward, from direct trainers at the community level to training of trainers (nurse auxiliaries), supervision of health facilities and community activities, finalize materials, monitoring, and evaluation. In this way, project activities can be scaled up to cover the entire four-Municipality zone as proposed (change language not going into whole zone). LOGISTICS Recommendation: MCDI should make every effort to provide the field with one more vehicle. The minimum requirement would appear to include at least one more reliable vehicle plus one motorcycle. This, together with very detailed programming and close coordination with Municipal and District vehicles should allow the project to fulfill its objectives. INFORMATION MANAGEMENT Recommendation: MCDI should reconsider the current HIS methodology for capturing information in order to spend more time supervising the existing MOH and NHI information systems to improve their reliability, as well as spend more time analyzing the data regularly. MCDI may wish to consider requesting access to the NHI program data, essential medication data, and rotating drug fund data from the Municipalities for internal analysis and reporting. Supervision Recommendation: MCDI should take the lead to coordinate agreements on integrated standardized supervision instruments for joint supervision at the various levels and work toward their systematic use, compilation, and analysis in the TAIs. MCDI should also explore the possibility of developing job aids based on self-assessment methodologies that could be utilized by auxiliary nurses to evaluate/supervise their activities at the health facility level. A similar model can be prepared for the use by community-based organizations. The utilization of self￾assessment tools will help to overcome the difficulties in maintaining the regular face-to-face supervision. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 86 BCC/IEC Recommendation: A very simple system for monitoring changes in caretaker knowledge and behaviors should be designed and deployed as early as possible. The system should be put in place with the permission and participation of the MOH and/or Municipalities, and data should be analyzed during TAIs in order to help institutionalize the process. The renegotiation of MCDI’s agreements with the MOH and Municipalities would provide an opportunity to present this idea. Suggestions are made in the report regarding how such a system may operate. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 87 F. RESULTS HIGHLIGHT The design of MCDI's Central Potosi Child Survival Project is well thought out. MCDI is the first organization to have worked at the community level in most of these villages, and therefore the negative effects of previous programs focused on “give-aways”, so commonly encountered in many zones, are largely absent. Clinical IMCI was initiated early in the project, as the training center in Potosí was already operational when the project began however, the center in Tupiza lacked trainers. In October 2000, MCDI supported the six-day training of eight people as IMCI facilitators, including four from the Tupiza district, three from Cotagaita district and one from MCDI. CARE supported two others from Villazón district. This allowed the Tupiza IMCI training center to become fully operational, which in turn facilitated training of personnel from Cotagaita and Vitichi. A total of 48 health workers have since been trained in the six-day course, either in Tupiza or Potosí. Materials developed by the bilateral USAID-funded PROSIN project were used and currently, all 38 health facilities in Cotagaita are utilizing IMCI protocols for case management. The HH/C-IMCI materials developed by Nur Univerisity have been field-tested with community groups, and MCDI has developed a draft training plan for HH/C-IMCI for use at the community level. Based on the HH/C-IMCI materials, MCDI has developed a written curriculum for a six￾day course to train health workers (especially nurse auxiliaries) as facilitators. These training plans have remained restricted to topics contained in Núr’s HH/C-IMCI course only, however, MCDI is in the process of including subjects related to the maternal and newborn care interventions. The maternal and newborn interventions have required significant coordination with MOH personnel and a close partnership with CARE who has provided training workshops on humanized delivery and family planning to all auxiliary nurses. Furthermore, the MNH Project is the primary force for upgrading the EOC skills of facility and community based providers). Since the beginning of the project, utilization of the maternal services in the project area has improved and the number of deliveries attended by health professionals has increased. For example, 48% of deliveries were performed by MOH health providers (either in the home or at a health facility) in 2000 and 68% were performed by MOH health workers in 2001, notably, after they received training in humanized delivery. This contrasts with the baseline KPC, where 80% of deliveries were taking place at home, with 65% of these deliveries attended by an untrained person (husbands, relatives and TBAs). In the 31 communities where MCDI is working, many have formed health committees, which meet regularly to discuss health-related problems and solutions. Furthermore, MCDI has developed a strong partnership with the municipalities to include the mayors and municipal health directors. One activity that is well underway is the cost study covering all the health facilities in Cotagaita and Vitichi. This study is expected to have a positive impact on the way that the municipal authorities manage the SBS recovery system. USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 88 III. ACTION PLAN A timeline of proposed activities are presented in the Action Plan below. During the evaluator's meetings in the field, the action plan was formulated in collaboration with the MOH partners at Municipal and District levels, and with other local stakeholders. At this time, general recommendations and ideas about each intervention area were discussed. Unfortunately due to the risk of a general strike in the country, the final meeting with SEDES and with local NGOs (CARE, PROCOSI, etc.) in Potosi was not carried out. Due to this unexpected occurrence, the following action plan is a draft that will require further discussion. The project and our MOH partners are in agreement that this Draft Action Plan would be jointly reviewed so that the final action plan can be prepared and completed. 89 RECOMMENDATIONS/ STEPS ACTION RESPOSIBLE PERIOD 1. Renegotiate Agreements with MOH – SEDES and Municipalities · Negotiate agreements to include: · Commitment to regular standardized joint supervisions. · Commitment to assent on a wider role of the auxiliary nurses in training and supervising CHVs. · Regular access to all information systems (SNIS/ SBS) Project Manager and Technical Director MOH SEDES And Municipalities First trimester of 2002 2. Complete the development and testing of critical training materials for HH/C-IMCI · Contract HH/C-IMCI technical assistance · The field team should Revise the existing materials and introduce the information lacking (as MNC). · Prepare materials for Auxiliary nurse training and for CHV’s training. · Test the final materials · Reproduce and disseminate training materials. Technical Director. Field Team, H.O. Consultant (Dr. Castillo) MNH Project, PROSIN April – May 2002 May – June 2002 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 90 3. Finalize training of clinical IMCI in Cotagita · Train 8 remaining health professionals (These 8 are in the process of being trained.) SEDES TOT and Field team March-April 2002 4. Complete the developments of BCC/IEC plan, support materials and monitoring system. · Contract consultant in BCC/IEC · Organize workshop to revise and adapt existing materials and identify new materials needed for IEC/BCC for HH/C-IMCI based in 16 Key family Practices. · Evaluate feasibility to adapt materials from the existing UNICEF Municipality/Family education Materials. · Adapt IEC messages and visual aids for Vitamin A and TT. · Design with consultant support monitoring system for IEC activities. · Reproduce and disseminate materials · Explore with Esperanza￾Bolivia the possibility to train MCDI team members in the use of cassette debate methodology. Technical Director, Project Manager, Field team and Consultant April –2002 April-May 2002 April-May 2002 April-May 2002 April-May 2002 May- August 2002 August 2002 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 91 5. Organize and launch MNC/emergency obstetrical care training with MNH/MOH (PROSIN/CIELO) · Coordinate training with MNH/MOH (PROSIN￾CIELO) · Launch TOT training activities in Tupiza · With assistance from consultant, prepare training plan for nurse auxiliaries in Health Posts · Launch training of health staff in Cotagaita, Vitichi and Puna hospitals · Launch short training course on MNC/emergency obstetrical care for nurse auxiliaries in Cotagaita and selected areas in Puna. Technical Director, Project Manager, Field team, MNH/MOH (PROSIN/CIELO), and Consultant (Dr. Castrillo) April 2002 April-May 2002 May 2002 July - November 2002 July 2002 - July 2003 6. Complete the development of instruments for integrated IMCI and MNC supervision · Finalize the draft integrated supervision instrument. (Make sure that existing MNC supervision protocols are used as reference) · Present to Municipal health authorities, analyze, redesign as needed draft document. · Approve document with SEDES. Technical Director, Field team and Municipal Heath directors April-May 2002 May 2002 May-June 2002 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 92 7. Finalize the essential drugs and EPI management studies and, present results to municipal and define plan to overcome constraints identified in the study · Finalize report and sent to HO for revision and approval. · Present report in a workshop with Municipal health teams. · Use the workshop to prepare an action plan to address the constraints identified in the study. Technical Director, Project Manager, CSST-HO. April -May 2002 May –June 2002 8. Finalize SBS cost recovery study, prepare report and present results to Municipal Authorities · Finalize data collection and data entry. · Prepare report based on data collected. · Workshop to present results to Local partners to include Municipalities and SEDES N. Agramonte (Economist and study assistant), Project Manager, Dr. Chris Schwabe (Health Economist), and Technical Director End March 2002-April 2002 April – May 2002 August 2002 9. Adapt the project HMIS to accommodate the SNIS data reporting system · Analyze with HO (CSST) support more efficient ways of collecting and analyzing data using the SNIS forms. Technical Director, Project Manager, CSST. May - June 2002 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 93 10. Define the role, responsibilities, time commitment, reporting requirements and monitoring/ supervision system for CHVs · Prepare in collaboration with Municipal health authorities strategy document outlining the role, responsibilities, time commitment, reporting requirements and monitoring/supervision system for CHVs and CHCs · Discuss and finalize draft with auxiliary nurses in TAI’s. · Approve document with Municipalities Project Manager, Health Educator, Technical Director and Municipal health teams. May 2002 May-June 2002 July 2002 11. Continue TOT on HH/C-IMCI with auxiliary nurses in Cotagaita) · Prepare plan/schedule of training and organize training materials. · Do the training Technical Director and Training IEC supervisor, Municipal/district health authorities May – June 2002 July - October 2002 12. Initiate routine joint supervisions with Municipal health authorities to the health post and with Auxiliary nurses to the communities. · Prepare draft, discuss and approve jointly with Municipal health director the supervision plan on a quarterly base. · Distribute supervision plan to all project team, to district and municipal health directors and to Auxiliary nurses. · Provide logistical support to supervision teams. · Execute supervision and present results in TAIs Project Manager, technical director, municipal and district health teams and field team End of April 2002 April-May 2002 May 2002 to the end of project in 2003 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 94 13. Strengthen municipal capacities to carry out supervision visits to health facilities. · Develop advocacy with Municipal Mayors to contract supervision nurses. · Help Municipalities to define job description for these nurses. · Analyze with municipal health teams and district health team possible incentives for members of supervision team. Project Manager, Technical director, Municipal Mayors and health teams May 2002 May 2002 May 2002 14. Initiate training of CHVs on HH/C-IMCI in Cotagaita (140 CHVs) · Continue identification mobilization of mothers groups. · Chose CHVs for training. · Direct support to the Auxiliary nurses to implement CHVs training in prioritized communities. · Continue training of CHVs delegating more responsibility to the auxiliary nurses June 2002 July 2002 August - December 2002 December 2002 to the end of project 15. Initiate process to expand activities to Linares district and prioritize areas of intervention · Discuss with District and Municipalities strategy for scaling up and agree in plan to scale up. · Define specific schedule of training., supervision and participation in TAIs for new areas. Project Manager, District and Municipal health directors of Linares and SEDES End May 2002 End May 2002 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 95 16. Increase awareness of the project within partners and stakeholders. · Increase the volume of written materials as performance reports and study reports. · Disseminate reports among partners and stakeholders as Municipal governments, SEDES, District areas. Project Manager, technical director, field team April 2002 to the end of the project 17. Support the COMUSA (Municipal Health Committees) and use them as the main coordination mechanism between the project and the health authorities. · Regularly participate in COMUSA meetings. · Present to COMUSA updates on project activities and coordinate interventions related to the project. Project Manager, Technical director May 2002 to the end of the project USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 96 18. Introduce the MOH Community Based Surveillance System (CBSS) · Contact SEDES to identify the materials, methodology and schedule to implement the CBSS. · Define with SEDES, DISTRIC and Municipalities the collaboration process and schedules to implement the CBSS. · Define with Health Authorities criteria to mobilize and train the CHVs that will do the surveillance. · Receive from SEDES training materials and initiate training of selected CHVs on CBSS. · Support Municipalities/Health posts in collecting information from the CBSS and use for management proposes. Technical Director, Project Manager, field team, SEDES and District/Municipal health authorities May – June 2002 ( depending on SEDES accessibility) May-June 2002 June-July 2002 September 2002 October 2002 to the end of project USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 97 19. Initiate training on clinical IMCI in Linares · Define, with Municipal health director and SEDES/Tupiza – IMCI TOT team, plan and schedule to begin training for health staff in Linares District. · Define costs of training for IMCI in Linares. · Initiate Training on Clinical IMCI for health staff in Linares Technical Director, Project Manager, Municipal health authorities and SEDES/Tupiza IMCI TOT team. May 2002 June 2002 July - October 2002 20. Initiate TOT on HH/C-IMCI with auxiliary nurses in Linares (Caiza D) · Prepare plan/schedule of training and organize training materials. · Do the training Technical Director and Training IEC supervisor, Municipal/district health authorities December 2002 First Quarter 2003 21. Initiate training of CHVs on HH/C-IMCI in selected communities of Caiza D · Implement identification mobilization of mothers groups. · Chose CHVs for training. · Direct support to the Auxiliary nurses to implement CHVs training in prioritized communities in Caiza D. · Continue training of CHVs delegating more responsibility to the auxiliary nurses January 2002 February 2002 March-May 2003 May 2003 to the end of project USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 98 22. Design and implement in collaboration with the Municipal MOH a strategy to verify true immunization coverage · Design small survey instrument, define sampling frame, chose sample, train enumerators, implement the survey. · Analyze results and disseminate among Municipal health teams and in TAIs. Technical director, MCDI H.O., Project Manager, field team. June 2002 July 2002 23. Support Municipalities and health facilities with a regular supply of maternal and child cards. · Identify needs, sources of card printing and costs. · Coordinate acquisition with MCDI – HO and municipal health team. · Buy, distribute and help in controlling distribution. Technical director, Project Manager, MCDI-HO, Municipal health team. May 2002 June 2002-June 2003 24. Monitor caretakers knowledge and practice · Design simple survey instrument based on KPC questions. · Define small, regular sample and methods to survey the mothers in the health facility. · Study possibility to use LQAS methodology to develop later the monitoring activities Technical Director, Training/IEC supervisor, Project Manager, MCDI-HO and municipal health teams/SEDES, June – July 2002 USAID/BHR/PVC Child Survival Project - MCDI/Bolivia Mid-Term Evaluation Report 99 25. Implement sustainability strategy · Prepare Japan grant proposal. · Prepare WB grant proposal for local projects. · Prepare CS Project Cost extension Project Manager, technical Director, Field team, CSST, Consultant March 2002 July 2002 August-November 2002 26. Prepare and implement final evaluation · Procure and secure external evaluator. · Prepare and implement final KPC. · Implement Final Evaluation Project Manager, technical Director, Field team, CSST, Consultant January 2003 June-July 2003 September 2003 100 IV. ATTACHMENTS ATTACHMENT A: BASELINE INFORMATION FROM THE DIP 1. Field Program Summary ESTIMATED PROGRAM EFFORT AND USAID FUNDING BY INTERVENTION Intervention % of Total Effort (a) USAID Funds in $ (b) Immunization 25% $249,825 General Nutrition % $ Vitamin A % $ Micronutrients (other than Vitamin A) % $ Breastfeeding Promotion % $ Control of Diarrheal Disease 25% $249,825 Pneumonia Case Management 25% $249,825 Control of Malaria 0% $0.00 Maternal and Newborn Care 25% $249,825 Child Spacing % $ STI/HIV/AIDS Prevention % $ Others (specify) 0% $0.00 Total 100% $999,300 (a) Estimate the percentage of total effort (from USAID and PVO match funding) the program will devote to each intervention to be implemented. Estimate in US dollars (not in percent) the amount of USAID funding (excluding PVO match funds) the program will devote to each intervention. 2. Program Goals and Objectives Project Goal: To reduce child mortality and morbidity, and to improve the health of women of reproductive age in the project area through interventions in immunizations, diarrheal disease control, pneumonia case management, and maternal and newborn care. Project Objectives Measurement Method Major Activities Process Indicators Source of Data Immunizations: 50% of children aged 12-23 months fully immunized by card 60% of mothers have received 2 or more doses of Tetanus Toxoid vaccine before the birth of their youngest child 50% of 6-11 month old children have received one dose of Vitamin A supplementation 50% of 1-5 year olds have received 2 doses of Vitamin A per year KPC Final Survey KPC Final Survey KPC Final Survey KPC Final Survey Review and establish protocol for capturing children 12-23 months who do not have a complete schedule Upgrade the cold chain and enhance its functionality at 35 health facilities in coordination with SEDES Provide ongoing initial and refresher training to 48 health workers on technical protocols, record keeping, avoiding missed opportunities, and maintenance of the cold chain Provide training to 140 CHPs in immunizations promotion, village monitoring and surveillance, and follow-up of defaulters through the Family Card Establish effective referral system between the CHPs and health facilities Review and adapt culturally appropriate strategies for social mobilization and BCC/IEC in close collaboration with the SEDES and key health programs, such as PROSIN/BASICS-II Enhance district supervision capabilities to permit frequent follow￾up and coaching to improve the quality of immunization services Protocols for capturing children 12-23 months who do not have a complete schedule established Percent of health facilities with upgraded and functioning cold chain Percent of health workers provided initial and refresher training on technical protocols, record keeping, avoiding missed opportunities, and maintenance of the cold chain Percent of CHPs provided training in immunizations promotion, village monitoring and surveillance, and follow-up of defaulters through the Family Health Card Effective referral system between the CHPs and health facilities established Culturally appropriate social mobilization and BCC/IEC strategies adapted and implemented Percent of health facilities receiving a supportive supervision visit in the past 3 months Project participation in NIDs/ campaigns during the year Percent of health workers trained in recognition and treatment of VAD Percent of health facilities with adequate stock of Vitamin A supplements Health Facility Records District HMIS Project HMIS Training Records Supervision records. Pre and Post Tests Prenatal Card Monthly CHP Meetings Records Focus Group Discussions Key Informant Interviews Direct Observation and Health Worker Interviews Client Exit Interviews Facility Inventory Support SEDES and the health districts in the conduct of NIDs/campaigns through logistical and organizational support Train health workers in recognition and treatment of Vitamin A deficiency Coordinate with SEDES for regular supply of Vitamin A supplements to health facilities Project Objectives Measurement Method Major Activities Process Indicators Source of Data CDD: 1. 70% children 0-23 months with diarrhea in the past 2 weeks received more breast milk. 2. 70% of children 0-23 months with diarrhea in the past two weeks received more or the same amount of food. 3.70% of children 0-23 months with diarrhea in the past two weeks received more than the usual amount of fluids. 80%of mothers of Children 0-23 months can identify at least three danger signs of dehydration. 5. 80% of children 0-23 months with diarrhea during the past two weeks received ORT. 6. 90% of children under 6 months are exclusively breastfed. KPC Final Survey KPC Final Survey KPC Final Survey KPC Final Survey KPC Final Survey KPC Final Survey Adapt and disseminate culturally appropriate BCC/IEC materials including non-literate graphic materials regarding danger signs of dehydration, and home case management PROSIN/BASICS Train 48 health personnel in IMCI protocols for diarrheal disease. Train 140 CHPs in community surveillance of diarrheal disease outbreak and referral of severe cases (HH/C-IMCI). Ensure supply of ORS sachets to all health facilities and CHPs in coordination with SEDES. Train health workers, CHPs, and TBAs in promotion of exclusive breastfeeding. Adapt and disseminate culturally appropriate BCC/IEC materials for the promotion of exclusive breastfeeding. Percent of health workers and CHPs trained in comprehensive Diarrhea Case Management and message dissemination. Percent of BCC/IEC culturally appropriate material developed and disseminated. Percent of health facilities stocked with adequate supply of ORS. Percent of CHPs stocked with adequate supply of ORS. Percent of health workers, CHPs, and TBAs trained in breastfeeding counseling skills. Cassette debates reports. Project records of coordinating meetings with SEDES and community groups. Baseline and final FBA. Supervision records. BCC/IEC material developed and disseminated. Health Facility Records District HMIS Project HMIS Training Records Pre and Post Tests Family Health Card Monthly CHP Meetings Records Focus Group Discussions Key Informant Interviews Direct Observation and Health Worker Interviews Client Exit Interviews Facility Inventory Project Objectives Measurement Method Major Activities Process Indicators Source of Data PCM: 1. 70% of mothers of children with pneumonia seek help from appropriate providers. 2. 70% of mothers and caretakers able to recognize two danger signs of pneumonia as a reason to seek care at health facility. 3. 100% of CHPs trained to identify pneumonia danger signs and promote fast referral to health facilities. 4. 70% of mothers of children with signs of pneumonia seek care upon recognition of symptoms KPC Final Survey, Facility Based Assessment, SEDES Records KPC Final Survey Project Records District Records KPC Final Survey Support training for 100% of health personnel in IMCI protocols for pneumonia case management and communications skills. Adapt, develop and disseminate BCC/IEC messages on pneumonia and treatment seeking for caretakers and the community. Train CHPs on HH-C/IMCI concepts for pneumonia and acute respiratory infections. Train CHPs in community surveillance of pneumonia cases (HH-C/IMCI). Educate and sensitize TBAs, THs, and Community groups on pneumonia case management at community level. Percent of health personnel trained in IMCI. Percent of health facilities better equipped to provide quality SCM. Number of messages developed and disseminated. Number of BCC/IEC sessions conducted. Percent of CHPs trained . Number of children with pneumonia referred to an health facility. Number of TBAs, THs, and community groups that were educated Number of mothers/caretakers educated by different health providers Health Facility Records District HMIS Project HMIS Training Records Supervision records. Pre and Post Tests Family Card Monthly CHPs Meetings Records Focus Group Discussions Key Informant Interviews Direct Observation and Health Worker Interviews Client Exit Interviews Project Objectives Measurement Method Major Activities Process Indicators Source of Data MNC: 50% of pregnant women have the recommended four prenatal visits. 50% of pregnant women take the prescribed course of Iron/Folic Acid 50% of pregnant women deliver with trained family KPC Final Survey KPC Final Survey KPC Final Survey Provide refresher training for 100% health personnel on management of maternal and newborn health, including PC, delivery and postpartum, high-risk pregnancy and EOC. Develop, adapt and disseminate BCC/IEC materials for MNC. Train and equip HP at primary care level for management of obstetrical Percent of health personnel trained in MNC. Percent of health facilities using protocols for management of maternal health and obstetric emergencies. Number of BCC/IEC materials developed, adapted and distributed. Project training records. Project Family Card and MOH Maternal health Registration Card. Signed agreements with SEDES, Municipality and hospitals. Results of analysis of KPC disseminated to communities. birth attendants at home 90% of new mothers begin breastfeeding within eight hours of delivery 70% of women receive Vitamin A within eight weeks of giving birth 80% of women can identify at least three warning signs of emergencies for each stage (pregnancy, labor, birth and postpartum) 80% of women can identify the danger signs in newborns during the first 24 hours and during the first 7 days. 80% of new mothers have attended counseling sessions on birth spacing (LAM and modern family planning methods) 60% of newborns are immunized with BCG within the first month, by card KPC Final Survey KPC Final Survey KPC Final Survey KPC Final Survey KPC Final Survey KPC Final survey level for management of obstetrical emergencies. Train CHPs and TBAs in message dissemination and behavior change activities for MNC. Train 30 TBAs on safe delivery, recognition of danger signs, identification of intrapartum and postpartum emergencies, essential EOC, IPCC and prompt referrals Train CHPs and TBAs in message dissemination and behavior change activities regarding early breastfeeding and the use of family planning including LAM. Introduce or adapt the notion of "birth preparedness" among trained TBAs and family birth attendants. Number of women and families informed and reached. Percent of CHPs trained in promotion of early breastfeeding. Number of mothers’ groups and rural cooperative contacted Percent of existing TBAs trained in safe delivery, essential emergency care and prompt referral IEC material produced. Proposed plan for obstetric emergency transportation from the community to HS. Project data, MOH, HMIS registers. Project Objectives Measurement Method Major Activities Process Indicators Source of Data CAPACITY BUILDING AND SUSTAINABILITY Note: Capacity Building Indicators for MCDI and its partners including SEDES, will be finalized later this year after baseline Organizational Capacity Assessment is carried out by the organizations SEDES/District MOH At the mid-term of the proposed project, the District/Municipal MOH will have developed and implemented an IMCI program of services; will have further developed and implemented a plan to expand and improve EPI and maternal and newborn care services. At the end of the project, the MOH will have the ability to determine accurate costs of maternal-neonatal care according to new national standards consistent with WHO's Mother-Baby Package; current reimbursement ratios are lower than actual treatment costs putting the operational viability of project facilities at risk. IMCI planned, developed and promulgated by District and Municipal health directorates and approved by SEDES (SEDES Records). A cost analysis of maternal￾neonatal care services completed (SEDES Records). Technical assistance from MCDI. Additional technical assistance from PROSIN/BASICS and the MNH Project. District and municipal staff trained in planning and financial management. Technical assistance from MCDI Health Economist (World Bank and MNH Project). Improved capacity of MOH in planning and implementing IMCI, EPI, and maternal-neonatal care. Improved capacity of MOH to determine actual costs of services to ensure adequate reimbursement levels Mid-Term Evaluation. Cost Analysis report. Nur University, Colegio de Post Grado 3.0. Nur University staff exposed to methods of Monitoring and Evaluation, GIS and cost analysis and cost recovery systems 3.1 Nur University equipped with personal computers and provided technical assistance on distance learning programs for maternal and child health. Project correspondence and records Reports on joint site visits Equipment delivered Participation of Nur University technical advisor in periodic Child Survival Project meetings Practicum for selected M&E activities and utilization of the GIS Practicum for cost analysis and cost recovery systems Joint dissemination of lessons learned in Child Survival Students' theses on Child Survival M & E, GIS and Cost analysis for Child Survival and Maternal and Neonatal Health Project HMIS Interviews of students at the end of practicum Student thesis. 3. Program Location Project Location The project site is located in the southwestern Bolivian department of Potosí, in the health districts of Cotagaita and Puna, which encompass the provinces of Nor Chichas and Puna. The project covers two Health Districts and four Municipalities with a total population of 102,209, or 14.3% of the population of the department. The geographic area is 14,115 sq. km (12% of the total area of the department) with a population density of 7.23 inhabitants per sq. km. The original proposal called for working in two health districts; however, based on discussions with the director of SEDES (Servicio Departamental de Salud) Potosí, it was recognized that, due to geographic constraints, it was inadvisable to begin work in both districts simultaneously. Therefore, the focus of the project for the first two years will be the health district of Cotagaita. The decision to extend the project into the Puna Health District will be made after the Mid-term Evaluation and after an analysis is made of the extent to which MOH and other partner resources are marshaled. Target Population The area covers a total beneficiary population of 56,995, which consists of 16,104 infants and children under five years of age, 24,535 women of reproductive age, and 16,356 infants expected to be born during the life of the project (4,089 expected births per year). The following table shows some population totals and urban/rural percentages. Population Totals by District Health District Province Municipality Total Pop % Urban % Rural < 5 years WRA Births x 4 years Cotagaita 27,484 6.2 93.4 4.469 6.620 1103x 4=4412 Cotagai ta Nor Chichas Vitichi 15,483 0 100 2.462 3.716 619x4=2476 Puna 48,021 0 100 7.494 11.53 0 1922x 3=7688 Puna` Linares Caiza D 11,121 0 100 1.679 2.669 445x4=1780 Totals 102,10 9 16.10 4 24.53 5 4089x4=163 56 Source: Informe de Gestión 1997, SEDES Potosí Existing Health Infrastructure and Collaboration Administrative Organization Politically and territorially, Bolivia is divided into departments and provinces. The basic unit of administration in the provinces is the municipality, headed by an elected mayor who is responsible for all health infrastructures, as well as the financing of health care services, and some health training in the area. Health Infrastructure The health system is essentially a network of services administered locally and jointly by the SEDES, the municipal government, and the community. The network is organized into three levels: the tertiary level or SEDES, corresponding to a department; the secondary level, or health district, which generally encompasses a province; and the primary level, or health area, which corresponds to one municipality. The project will work with SEDES Potosí, headed by Dr. Angel Negron; two health districts, Cotagaita and Puna, which encompass the provinces of Nor Chichas and Linares; and four rural municipalities (Cotagaita, Vitichi, Puna and Caiza D) that are among the poorest in the department. According to the results of the District Assessment conducted by MCDI/EB (Esperanza Bolivia), the Health District of Cotagaita serves a total population of approximately 40,000 and comprises a network of 28 puestos de salud (HPs, or health posts), five health centers, and two hospitals (one in each municipality). Puna Health District comprises a total of 2 hospitals, four health centers, and 26 HPs. Current staffing within Cotagaita facilities includes 10 doctors, 5 licensed nurses, 32 auxiliary nurses, two dentists, one social worker, one administrator, and five support staff. In rural areas, auxiliary nurses are in charge of health posts, whereas junior medical doctors are typically heading the health centers. Given the existence of five individual health centers in the Cotagaita region, there is a total of five doctors who are on obligatory one-year provincial service and in charge of each health center. The district health office is located in Cotagaita, with a district director as its head, and the health area head, the Chief Municipal doctor, works out of the hospital in each municipality. Each health post serves an average of six communities; thus, of the 222 communities in the project area, 156 are served currently. Results from the district assessment, based on interviews conducted from the health offices at Potosí and Cotagaita, showed that in the Cotagaita health district, 15 of the facilities were in poor condition, nine were in fair condition, three had been remodeled, and eight were newly constructed and equipped, mostly with funds from the World Bank/Social Investment Fund (FIS). Thirty-four facilities rely on solar powered radios for communication, and three of the 36 facilities have no modern method of communication. The Facility Based Assessment revealed that out of the 11 health facilities surveyed, only five have radios in operational condition, raising the issues of management and maintenance of the system. In terms of transportation, ten facilities have no access to transport; the others rely on MOH ambulances and motorcycles, and/or private trucks and bicycles. In fact, the Facility Based Assessment revealed that in at least one of the health posts located in Callpauno, the local population arranged the means to have their own public transportation. Additionally, 94% of facilities surveyed do not have basic services such as electricity and water. Only the municipal hospitals have these basic utilities. Other fundamental problems in the health system are the lack of specialized health personnel, lack of some second line medicines and lack of regular training schedules for health personnel. Facility Functions and Staff Health Posts (HP) are attended by nurse auxiliaries with basic health training to provide preventive, promotive and limited curative health care in MCH. Auxiliary nurses can also handle deliveries without complications and can make referrals to the health centers. In the rural areas, there are transit beds for deliveries or other emergency. HP staff performs immunizations, and receive training for the maintenance of the cold chain. TBAs are considered an essential component of the traditional medicine practitioners, but their integration into the public network has not been smooth or widely accepted.1 A Health Center (HC) is staffed by a medical doctor (generally doing a required year of rural service) and nurses (licensed or auxiliary), and integrates preventive, promotive, and curative care, attends to general medicine cases, and refers cases that need specialized attention to the municipal, or to the district hospitals. HCs and HPs have transit beds for short admissions, normal deliveries and emergency hospitalization. A Municipal hospital ideally serves as reference for all health areas and is staffed by at least one doctor. In the case of Cotagaita, the Municipal Hospital is staffed by 3 doctors (GPs) and is awaiting a general surgeon. In addition, the Cotagaita Hospital is used as a referral facility for Vitichi Municipal Hospital. The District Hospital provides care in four basic specialties: pediatrics, OB/GYN, internal medicine, and limited or basic surgery and is equipped to handle non-surgical obstetrical emergencies, but lacks equipment to handle emergency surgical cases (these are referred to Potosí or Tupiza). In the case of Cotagaita, there is no specific district hospital hence this role is assumed by the Cotagaita Municipal Hospital. The closest district hospital is in Tupiza where tertiary obstetric and surgical care is provided (2 hours by motor transport). Community health promoters (CHP) MCDI plans to train a total of 140 community health promoters. The role of the health promoters is to participate in IEC and Immunization campaigns. It is expected that they will be used specifically to track patients as indicated by the municipal or health center doctor. Parteras or Traditional Birth Attendants The project area also contains a number of parteras or Traditional Birth Attendants who are active in the communities; these may be male or female. Typically, parteras play less of a role in deliveries in Bolivia than is the case in most other developing countries (MotherCare, 1991). This is corroborated for the project area by the baseline KPC survey that found that even though 80% of all mothers in the sample had delivered at home, parteras or TBAs had assisted in only 7% of deliveries. As many as 35% of deliveries were assisted by husbands, and a further 23% by other relatives or friends. However, parteras are usually summoned in cases where complications are noticed. An initial survey by field staff based on interviews with health facility personnel found that there are currently 30 parteras active in the project area. The project will collaborate with the MNH project to conduct IPCC training for the 30 parteras, as well as clinical training in safe deliveries, recognition of danger signs for both the mother and the newborn, essential emergency obstetric care, and prompt referrals. Curriculum for the training has been developed by SCF through the Warmi project, including a Parteras’ Manual listing information on pregnancy, birth, postpartum period, and care of the newborn. In addition, the project will arrange to provide trained parteras with safe delivery kits containing locally procurable materials that TBAs may use to instruct family/home birth attendants on safe delivery practices. Following the training, it is expected that these TBAs, along with CHPs, will engage in health education and prompt care seeking 1 Health in the Americas Volume II PAHO 1998 awareness building activities in the communities involving mothers’ groups, rural cooperatives, traditional healers, and husbands and other family members. CAIs The MOH has established a system of Communities for the Analysis of Information or CAIs at different levels of the health system within the department. These function at community, district and regional levels and involve community members and health workers meeting together at stipulated intervals to review and analyze a number of health indicators and formulate local action plans for implementation. From the MOH’s perspective, these are the principal media for community participation in health service delivery. The project proposes to reactivate CAIs in communities where they are not currently active, and to support their activities so as to broaden community participation. The principal links for this support will be provided through trained health workers and CHPs. National Health Policy Health facilities operate under the norms of the National Strategic Health Plan (NSHP) for 1997-2002, and the strategies set for Family and Community Medicine. One of the most promising opportunities for the project’s success is engendered in the national insurance plan, the Seguro Basico de Salud (SBS), adopted in 1998. The system provides effective and essential health services through the public health system, the Social Security network of services, NGOs, churches and others. The Seguro replaced the former Health Insurance for Mothers and Children, and has wider coverage for mothers, children, school children, adolescents and the elderly. The objective is to lower child, infant and maternal mortality, and to reduce the risk, duration and severity of the principal causes of morbidity and mortality in the population. It emphasizes services for children under 5 years of age, women of reproductive age to include pregnancy, reproductive health, and detection and treatment of cholera, malaria and tuberculosis. All health establishments provide services free of charge to the individual patient; the Municipalities reimburse the costs of services to the health system. The SBS has contributed nationwide to an increase in demand and utilization of public health facilities. IMCI is an integral part of the SBS and in fact, Bolivia was one of the WHO IMCI focus countries. IMCI is currently being introduced to the Potosí region with training centers established in Potosí and Tupiza; the project will support IMCI training. Although the results of the District Assessment showed that facility staff are trained and equipped to implement SBS, staff site visits evidenced unfamiliarity of auxiliary nurses with program requirements. According to the Cotagaita District Director, there is a great need for training and technical assistance for medical and nursing staff on the implementation of SBS services. In addition, the district teams have identified a need to build their capacities in program and human resources management, strategic planning, conflict resolution, participatory evaluation, teamwork, and project administration. As for the health area teams, they have stated that they need to be trained in the use of data collection instruments of the health information system, in referrals and counter-referrals, and in public health. A Cost Study of the Mother-Baby Package conducted by JSI’s MotherCare project in Bolivia revealed that reimbursement rates were lower than the cost of treatments, and that there is a great need for accurate cost information when implementing new health reforms. One of the capacity building interventions that the project will support will consist of a cost analysis of essential child survival related interventions, providing district and municipal staff with accurate information upon which to make resource allocations. Health and Child Survival Related Programs MCDI has contacted and anticipates close collaboration, where possible, with a myriad of organizations and national programs active in the health sector in Bolivia and Potosí department. The project plans to draw on lesson learned and standardized curricula and training materials in IMCI, BCC/IEC, MNC (EOC/IPCC), and RH developed by PROSIN/BASICS-II, CARE, Safe Motherhood and the Maternal and Neonatal Health Project, as well as implementation of the community auto-diagnosis strategies developed by Save the Children Foundation under the Warmi project. The project will also work through coordinating bodies such as PROSIN and PROCOSI, and national programs such as EPI, and draw on resources available through the MOH and SEDES. 4. Program Design Program Approach MCDI’s overall approach incorporates three essential components: community based activities, health worker training, and health system strengthening. Thus, MCDI believes that the synergy created by simultaneously addressing both demand (community) and supply (health facilities and health system) factors will contribute most effectively to the primary goal of the program which is to reduce morbidity and mortality among children under 5 years, and women of reproductive age. Program activities will involve: The improvement of clinical skills and counseling (IPCC) by health providers at all levels of the health system in the two districts, through training in IMCI and MNC skills, and close supportive supervision and monitoring The implementation of a community outreach program that encompasses promotion of healthy behavior and practices and educational mobilization involving the training and close supervision and support of 140 CHPs located in the communities, and interfacing with the auxiliary nurses at the health posts and 30 parteras; and BCC/IEC at the community and household levels will lead to improved knowledge and behavioral change that is reflected in practices that promote the health of mothers and children The strengthening of health planning and financial management and budgeting skills of municipal and district health staff. Based on intensive discussions with its partners, MCDI decided to implement program activities in a phased manner. Thus, initially, community outreach and BCC/IEC activities will focus on Cotagaita district for the first two-year period. Expansion to cover Puna district will occur based on the results from the midterm evaluation to be conducted at the end of Year 2. However, all training activities involving health workers at various levels will be conducted at the SEDES training centers and will include health staff from both Cotagaita and Puna districts. The decision to conduct program activities and interventions in a phased manner responds directly to the concerns voiced by SEDES and local partner that addressing both districts simultaneously would spread program resources too thin, particularly in light of formidable geographical constraints extant in this area. The program will assist the SEDES, the districts and the municipalities to access future financing under two World Bank projects, the Health Sector Reform Project and the Participatory Rural Investment Project, and builds on health infrastructure investments and improvements in the districts that have already occurred with financing from The World Bank. As recounted below, MCDI will also benefit from the lessons learned by a number of national and local level health sector programs that have operated as partners with the MOH, and will draw on their technical expertise e.g., MotherCare, PROSIN/BASICS-II, CARE/Bolivia, the new MNH project, Save the Children, and UNICEF. Male heads of households, women of reproductive age, children and newborns will enter and participate in the program through the outreach activities conducted by male and female community health promoters (CHP) that are supervised by auxiliary nurses based in health posts, who, in turn, are supervised by the district health team. Obstetrical emergencies and emergency care for newborns and under fives will be referred to Tupiza Hospital, located two hours away from Cotagaita. The process by which eligible women, children and newborns will benefit from the projects interventions will be enhanced through the expanded outreach capabilities of the health posts to be complemented by CHPS and parteras. The CHPs will assist communities to organize birth preparedness activities to include transportation. In addition, parteras will also be trained in clinical skills. The project will develop reporting instruments, as none currently exist, and will determine the feasibility, with MOH approval, to introduce the family card. The family card was originally developed under the Poverty Reduction Project and adapted by several NGOs, including EB and the Canadian Red Cross. The family card activities will be under the responsibility of the auxiliary nurses and will identify families at risk while simultaneously prioritizing cases in the community. 5. Partnerships It should be mentioned that since the submittal of application the partnership formed with Esperanza Bolivia was dissolved, primarily at the behest of SEDES Potosí. USAID/La Paz The Mission provided guidance to MCDI during the application development process, reviewed the proposal favorably, and has been supportive throughout the process. Ministry of Health/Department Level (SEDES) MCDI has established and maintained close contact with the Departmental Director of Health in Potosí, and worked closely with the SEDES office to prepare both its original application and the current Detailed Implementation Plan. Several meetings were held by the DIP development team with the head of SEDES in Potosí. Drafts of the program interventions and overall strategy for the program were also shared with him and his team. A further example of our close collaboration is provided by the minutes of a conference call conducted in May, which are enclosed in the Annex for perusal. Also attached therein is the formal agreement signed between the program and SEDES. MCDI intends to involve the SEDES very closely in all program activities, and for this purpose, will establish a Program Coordinating Committee to oversee the implementation of all program activities. The Program Coordinating Committee will consist of Dr. Angel Negron, and MCDI’s Program Manager, Mr. James Selph, and will remain open to the participation of other partners, such as Nur University, as the need arises. In addition, this committee also integrates the District Health Director, Dr Fredy Manurez. The Coordinating Committee will draw upon the technical resources of all organizations involved, as required; it will initially meet every month, and later every quarter. In addition, SEDES will assign a Technical Coordinator who will guide day-to￾day activities of the program and act as a liaison between SEDES and the implementing partners when required. District Level MCDI has developed close relationships with the District health directors at Cotagaita and Puna. In Cotagaita, the District Health Office has donated office space within district hospital premises to the program. MCDI intends to pursue the collaboration closely by involving district staff in all aspects of program implementation. The most immediate collaboration activity engaged has been the implementation of the Facility Based Assessment, completed in July of this year; FGD selected district staff were involved in the participatory training, data collection and analyses to improve these skills at the district level. Municipal Level Due to the importance of the local (municipal) administrative authorities, MCDI will participate and be an active stake-holder in the existing health committee that forms a part of the Municipal Committee. With this in mind and since the preparation of the original proposal, MCDI has worked in close collaboration with the municipal mayors who coordinate activities in their areas, are responsible for health infrastructure, and finance much of the health worker training activities conducted in the program area. MCDI’s collaboration at this level will extend to strengthening the capabilities of the municipalities in health planning, financial management, and budgeting. BASICS II Project has verbally committed itself to sharing with MCDI its work on IMCI, CDD, and BCC/IEC strategies. PROSIN/BASICS has established training centers all over the country, and trained a number of facilitators and trainers in IMCI training. The project has also trained auxiliary nurses who form the first point of contact for rural populations. MCDI will draw heavily on the resources of the project to support its own intervention activities in IMCI and immunizations. In addition, PROSIN, an integrated health project within the MOH that is working under the tertiary vehicle of BASICS, has also shown interest and commitment to work with the project in the IMCI training activities. Maternal and Neonatal Health Project This new national program funded by USAID and being implemented by a consortium led by JHPIEGO and consisting of JHU/CCP, PATH, and CEDPA has identified Potosí as a priority area for its interventions. MCDI has contacted the project director, Dr. Ariñez and received assurances of support through training materials for health workers and technical assistance in BCC/IEC strategies and materials. MCDI will work closely with SEDES Director and his staff, and MNH staff to develop a plan for the program area. The World Bank’s $36 million component of the Health Sector Reform Project for Bolivia strongly supports implementation of IMCI and the Mother-Baby Package, including training of trainers and health workers at the regional level. These activities will, in turn, contribute to MCDI’s strategic emphasis on IMCI and maternal and neonatal health, as well as immunizations. CARE is implementing a Reproductive Health project in the region, and MCDI has been in contact with their representative in Potosí to secure collaboration. Save the Children, which implemented a Child Survival Project among a predominantly Aymara population, developed an autodiagnosis strategy for community based assessments and action under the Warmi program. MCDI will explore the application of this approach to its Quechua population resident in the program area. Canadian Red Cross is currently implementing a health project in the Betanzos District in Bolivia and has adapted the HMIS to assist the MOH in the implementation of a family card. Lily Montano, visiting coordinator of the Canadian Red Cross, has agreed to coordinate Canadian Red Cross interventions with the project and share information with CPCSP staff, particularly in the use of the family card and the adapted HMIS. 6. Health Information Systems Overall M&E Plan The Monitoring and Evaluation Plan for this project encompasses both monitoring of project activities through process indicators, and compiling of outcome indicators for midterm and final evaluation purposes. Detailed process and outcome indicators by intervention are presented in Table D above, linked to project objectives. The basis for monitoring project progress will be the periodic collection of data on project activities and services provided to beneficiaries, coupled with the data already collected through the District Assessment and the Facility Based Assessment. Census-based tracking of individuals and households which is an alternative approach to setting up an M& E plan is not envisaged at this time due to the isolation of most communities in the project area, and the high marginal costs of accessing each additional individual and household. There are two caveats to this observation. First, we are aware that Save the Children has been implementing a Community Health Information System or SECI, utilizing health promoters for data collection at the community and household level in pilot communities in the Oruro department. While preliminary discussions were held at the time of application preparation regarding a possible collaboration, a complete understanding of how successful SC has been with the SECI, and more importantly, how cost effective the process is, is yet to be obtained. Second, we are aware that the CORE M&E Working Group in collaboration with the CSTS project is engaged in the task of reviewing community based health information systems, and developing guidelines and instruments for PVO use. Once these tools are made available to the CS community, we will review them for applicability to our project environment. Meanwhile, for our purposes, MCDI will proceed with developing and implementing a limited project HMIS fully integrated with the MOH HIS termed SNIS, details of which are provided later in this section. The outcome indicators have been compiled from the baseline assessments carried out so far, including the KPC survey and the District Assessment. Additional outcome and process indicators relating to the health facility network, including SEDES and the health facilities, will be compiled during the finalization of the project HMIS slated for later this year. The results from the qualitative inquiries already carried out for each of the intervention areas will also contribute toward the framing of these process indicators. As noted, a few illustrative capacity building indicators have been included in Table D; by mutual agreement, MCDI and its partners have decided that final indicators will be formulated after the application of a baseline organizational capacity assessment to be carried out later this year. This will provide the time deemed necessary for the partners to establish relationships of trust, at which point such a baseline assessment would prove to be most useful. No final decision has been taken about the selection of an instrument or methodology for the purpose. Copies of DOSA, MOST, and a variety of other such methodologies and instruments have been shared with the partners and are under review. Compatibility with MOH SNIS It should be noted that MCDI does not envision setting up a project HMIS, which is an alternate system to the current MOH SNIS in operation at the departmental, district, and health facility levels. Thus, service statistics for the various Project interventions which are required for compiling the project’s outcome and process indicators will be collected from the District Health Office and/or health facilities on a quarterly basis. Where lacunae are discovered in the health facility data forms submitted to the District Health Office, the Project Manager and Field Technical Director will arrange to visit the individual facility to validate the data. This will serve to strengthen the District HIS by providing an informal means of monitoring and supervision of the District’s HIS efforts, as well as inculcate a sense of ownership over data collection efforts by current health facility staff. Anecdotal evidence combined with the preliminary findings from the District Assessment suggest that a number of such staff do not understand the need for collecting and maintaining service statistics, as a result of which facility registers and forms are poorly maintained. One critical reason for this is the fact that no feedback is provided to health workers. CAIs The MOH has also established a system of Committees for the Analysis of Information (CAI) at different levels of the health system within the department. These function at three levels: Regional CAI, which meets every 4 months at the SEDES level (Potosí) District CAI, which meets every quarter at the District level, and Community CAIs, which meet every 6 months at the Health Post level. A number of indicators have been formulated relating to child and maternal health interventions, and these are required to be reviewed, and plans of action, as deemed necessary, drawn up for implementation. MCDI will review the functioning of the CAIs, particularly at community and district levels, during the first year of project implementation and formulate plans for strengthening the process. From the perspective of the SEDES, CAIs are the key institutions whereby members of the community can participate in active planning for their and their community’s health, and as such, MCDI sees the CAIs as playing a pivotal role in the monitoring of project activities. In addition to the data listed above, databases of the number and types of health workers trained in the various CS interventions including IMCI, the number and content of BCC/IEC materials developed and issued, and the number of health contacts carried out by CHPs will also be maintained. These data will then be plotted on project area maps to determine the extent and spread of project activities, and assist in the planning of activities for the next quarter. Data Analysis and Use The project HMIS will consist of a computerized system to be maintained at the field office by MCDI staff, with periodic assistance from Nur University. The system will consist of a series of Excel files to be designed by the stakeholders jointly that will summarize a number of service statistics and other relevant data by health facility and by quarter and year. Output from the database will be analyzed for trends in disease prevalence and incidence, and will also serve as a monitoring tool for the project’s training and BCC/IEC activities. Data Dissemination A key aspect of MCDI’s approach to the analysis and use of the data is to disseminate it to a number of stakeholders in the project area. First and foremost, summary data and indicators will be shared with the CAIs at all three levels within the project area. These will be the primary channels through which these data will be shared with beneficiary communities residing in the project area. MCDI sees this activity as alerting communities to health problems that they may be required to face, and promoting cooperation in succeeding phases of data collection, as well as implementation of planned interventions. It is also a community empowering strategy that provides them with more opportunities to take responsibility for their own health. Second, summary reports will be shared with the SEDES at all levels, and bi-annual summaries will be shared with other stakeholders such as local institutions and NGOs, and other international organizations active in the area such as PROSIN￾BASICS, MNH project, CARE, SCF, etc. GIS The MCDI Home Office has acquired significant expertise in the design and installation of simple GIS systems for health programs. For the purposes of this project, the field office will be equipped with hand held scanners to establish the geographical locations of health facilities, and their catchment areas. These spatial data will then be entered into an Arc View GIS dataset with each facility location linked to the summary HMIS data colleted on a quarterly basis by field staff. Map outputs from the database will be shared with our partners at all levels of the MOH/SEDES and with other stakeholders for analysis of disease incidence patterns, health services utilization, etc. Monitoring Health Worker Performance The Training Plan in the section above provides details of the steps MCDI plans to take to strengthen health worker performance, including pre and post testing of all training carried out by the project. Additionally, as part of project intervention and monitoring activities, MCDI will work with the SEDES to strengthen managerial practices, particularly supportive supervision of auxiliary nurses who are the backbone and frontline of the health care delivery. While the MOH does possess a supervision and/or monitoring plan for all levels, and has designed appropriate instruments and schedules for its implementation, anecdotal information suggests that the supervision is of poor quality, and in need of significant strengthening. On account of this, the project will work closely with MOH staff to carry out an assessment, and identify system constraints. This approach will also strengthen the future sustainability of the supervisory system. MCDI’s approach, in collaboration with the MOH supervisory staff, also will include direct observation of provider practices, and provider-client interactions, both at facility and community levels, from time to time and client exit interviews. Analysis of existing problems, on-the-job training and problem-solving approaches will be utilized to ensure improvement of current practices. In addition, a summary of the supervision findings of the previous periods will be presented to CAI members to be used as a “lessons learned” approach. Midterm and Final Evaluations MCDI plans to carry out the Midterm and Final Evaluations on schedule. The Midterm is tentatively programmed for the last quarter of 2001 (November), when the project will be midway through implementation activities. We expect to utilize the LQAS methodology to collect small samples to identify health facilities and catchment area communities that are lagging behind the objectives specified for each intervention. This methodology will incorporate limited questions from the baseline KPC, as well as the FBA recently carried out. Results from the assessment will permit the project in its two remaining years to target its resources at communities and health facilities requiring such attention. It is likely that CSTS/CORE will conduct LQAS regional training over the course of the next two years and, if so, field staff will be detailed to attend such workshops. MCDI has also recently applied the LQAS methodology to its Health Education in Primary Schools (HEPS) project in Benin, as part of final evaluation activities, based on its Home Office expertise. Thus, the Home Office will be closely involved with the field office in the design and implementation of the LQAS approach. Results from the Midterm evaluation will also be shared with all stakeholders, including the MOH/SEDES and the District and Municipalities. The final evaluation will be conducted at the end of project activities (July 2003), and will involve a final KPC survey to determine if the project can claim to have achieved its objectives as presented in Table D above. ATTACHMENT B: TEAM MEMBERS AND THEIR TITLES The evaluation team consisted of all MCDI field staff members plus the external evaluator. Ing. James Selph, Project Manager Dr. Ismael Gonzales, Technical Director Lic. Noemi Ayllón, nurse, community mobilizer and trainer Dr. William Leaño, physician, community mobilizer and trainer Lic. Mabel Rivera, nurse, community mobilizer and trainer Lic. Mercedes Benites, murse training and IEC supervisor Nabil Agramont, administrator and coordinator of cost-study Donald T. Whitson, MD, MPH, external evaluator In addition, input was received from the MCDI headquarters CS Coordinator, João Blasques de Oliveira, MD, MPH This report was prepared by: Donald T. Whitson, MD, MPH Tel/fax: 66-2-261-6789 e-mail: dtwhitson@hotmail.com ATTACHMENT C: ASSESSMENT METHODOLOGY The evaluation was carried out by an evaluation team, which consisted of the external evaluator and the entire MCDI/Bolivia field team. Well in advance of the evaluation, all partner organizations were invited to participate throughout the evaluation, both in writing and verbally, including Cotagaita District staff, SEDES, and Municipal government staff. Unfortunately, none did so, although a meal was offered for the final presentation meeting on the last day. The Director of SEDES initially advised MCDI of his intention to participate in the final presentation, but he cancelled at the last minute. The evaluation schedule and principal activities were as follows: Jan 21 Mon Document review. Travel Tarija-Cotagaita Jan 22 Tue Team meeting in morning. Introductions, review of schedule, explanation of objectives for evaluation, presentation of evaluation guidelines and questions. Review of purpose and objectives of the BHR/PVC/CS grants program and its restrictions. Afternoon: Review of CS project objectives, strategies, and activities planned. Brief review of progress for each activity planned. Jan 23, Wed am: Document review. Field visit to Vitichi. Visit to municipal hospital. Scheduled visit to Mayor and Civic Committee rescheduled due to travel to La Paz. Meeting with interim mayor. pm: Field visit to Yawisla health center followed by group discussion with Yawisla women’s group. Jan 24, Thur am: Document review. Interview with Mayor of Cotagaita, group discussion with Cotagaita Civic Committee pm: Group discussion with District IV, Cotagaita 25 Jan Friday am: document review. Field visits to Tumuzla (Cotagaita), Calcha (Vitichi) including visits to health posts, interviews with nurse auxiliaries and health promoters, and group discussions with women’s groups pm: participatory evaluation work with team: technical interventions 26 Jan Saturday am: Field visit to Thapi—inspection of health post, interview with nurse auxiliary, focus group with women’s group and community authorities pm: Field visit to Totora—inspection of health post, interview of nurse auxiliary, focus group with women’s group and community authorities 27 Jan Sunday am: Field visit to Kalaparqui—interview with nurse auxiliary from Laytapi, focus group with women’s group and authorities Field visit to Laytapi: community meeting including women’s group, inspection of health post pm: Field visit to Tocla: inauguration of women’s club exposition, interview with nurse auxiliary, inspection of health post, discussion with entire community and women’s group, presentation of results of training by women’s group. 28 Jan Monday All day: participatory evaluation work with team: technical interventions. Telephone interview with headquarters office. 29 Jan Tuesday am: Discussion with Mayor and Civic Committee of Vitichi pm: participatory evaluation work with team: Community mobilization, IEC, strengthening MCDI/Bolivia, capacity-building of local partners. 30 Jan Wednesday All day: participatory evaluation work with evaluation team: sustainability, strengthening health facilities, health workers, training, management (including HMIS) 31 Jan Thursday am: final presentation of evaluation results, conclusions, recommendations, final preparation of notes from evaluation 1 Feb Friday Interview with staff from Esperanza/Bolivia A planned visit to Potosí to interview the SEDES director was cancelled due to the threat of a nationwide strike that would have made travel impossible for an undermined period of time. Documents reviewed included the original proposal, DIP, monthly and quarterly reports, training curricula for IMCI and HH/C-IMCI, BCC/IEC plans and materials developed to date, SNIS II data, data from the internal HMIS, results of application of the IMCI supervision form, and results of the special studies carried out to date. The field evaluation was carried out in a participatory manner, with the evaluator moderating the discussion based on the evaluation guidelines, and the team preparing summaries of results, conclusions, and recommendations on flip-chart paper. For each section, team consensus was sought for conclusions and recommendations. Headquarters input was obtained through telephone interviews, e-mails, and written answers to relevant sections from the evaluation guidelines. This input was incorporated into the final report. Finally, both MCDI field staff and headquarters staff reviewed the report and offered corrections and clarifications before final submission. ATTACHMENT D: LIST OF PERSONS CONTACTED MCDI/Headquarters João Blasques de Oliveira, MD, MPH, Child Survival Coordinator MCDI/Bolivia Casilla 105 Tarija, Bolivia Tel: (Tarija) 591-664-8901 Tel (Cotagaita): 591-694-3472 e-mail : jselph@olivo.tja.entelnet.bo Ing. James Selph, Project Manager Dr. Ismael Gonzales, Technical Director Lic. Noemi Ayllón, nurse, community mobilizer and trainer Dr. William Leaño, physician, community mobilizer and trainer Lic. Mabel Rivera, nurse, community mobilizer and trainer Lic. Mercedes Benites, murse training and IEC supervisor Nabil Agramont, administrator and coordinator of cost-study Lisa Paton, peace corp. volunteer Esperanza/Bolivia Calle de Madrid 468 Entre Santa Cruz y Junín Tarija, Bolivia e-mail: speranza@olivo.tja.entelnet.bo Palmira Villarroel, Executive Director Ruth Crespo Tel: 591-664-8563, 591-664-7890 Nur University John Kepner, International programs and projects coordinator Municipality of Cotagaita Concepción Ayllán, Mayor Dr. Noel Vásquez, President municipal council Ing. Nelson Leaño, Vice-president municipal council Ing. Róger Serrudo, council member Municipality of Vitichi Dr. Jone Pari, area medical officer Dolores Chambi, head nurse of municipality Hospital staff of Vitichi Municipal Hospital Mayor LeonardoMiranda D. Félix Vique, president municipal council Pastor Berrios, secretary municipal council Health District IV—Cotagaita Dr. Fernando Azabele, Director Lic. Jael Chungara, Cotagaita District Office Head Nurse Marco Alemanes, statistician Gonzalo Suralta, Coordinator for basic health insurance Lic. Patricia Mendoza, Municipality/Family Program and IEC nurse for municipality Communities -Yawisla (Vitichi) Germán Piuca, nurse auxiliary, health post Women’s club of Yawisla (about 20 members) and authorities -Calcha (Vitichi) Eduardo González, nurse auxiliary, hospital Hospital staff (including nurses in internship) Women’s club of Calcha (about 45 members) and authorities -Thapi (Cotagaita) Tomás Flores, nurse auxiliary, health post Women’s club of Thapi (about 32 members) and authorities -Totora (Cotagaita) Gerardo Ribera, nurse auxiliary, health post Women’s club of Totora and authorities (over 100 people) -Kalaparqui (Cotagaita—satellite community of Laytapi) Women’s club of Kalaparqui (about 12 members) and authorities -Laytapi (Cotagaita) Gregorio Aramayo Cazón, nurse auxiliary Women’s club of Laytapi (about 12 members) and authorities -Tolca (Cotagaita) Milton Trujillo, nurse auxiliary Community meeting—over 150 people, including women’s club ATTACHMENT E: TRAINING MATRIX Approximate chronological order Course Date, length, and site Participants Institution administering / funding Material and curriculum Methods Evidence of impact / comments Management of SNIS 4 th quarter 2000, 4 days, Cotagaita 48 participants of Cotagaita district + MCDI staff SEDES, Government of Bolivia Photocopies of SNIS forms, registration books Theory and practice New SNIS II system functioning reasonably well IMCI facilitator and supervisor training October 2000 6 days in Potosí 4 Tupiza district 3 Cotagaita district 1 MCDI 2 Villazón district (CARE sponsored) SEDES Potosí administered MCDI and CARE paid PROSIN/Basics Theory and practice Tupiza training center functioning and training.IMCI supervisors trained Management of rotating drug fund 1 st quarter 2000 X 2 days. Tarija 3 MCDI 2 Cotagaita district PSF (project that financed and implemented drug fund) Drug fund manual and cardex Theory and group work Not useful. New system very complicated. Funds for implementation of plan never disbursed. Use of focus groups to collect qualitative information 2 nd quarter 2000 X 2days; Tarija 3 MCDI 2 Cotagaita district Esperanza/Boli via + MCDI Guide for focus groups Theory and practice MCDI did focus groups after training to complement KPC. Note that no District personnel have used focus groups. Planning and use of indicators 2 nd quarter 2000 X 2 days Cotagaita All Cotagaita district staff + MCDI SEDES PSF Ministry of Health materials and forms Theory and practice District and municipalities are able to carry out planning and CAIs on their own Quality consults 4 th quarter 2000 X 2 days, Cotagaita Cotagaita district MCDI CARE Quality attention manual (CARE) Theory and practice Unknown—not evaluated RPS (Health promoters) 3 rd quarter 2000 X 1 day 18 community members MCDI (administered) Own materials-- improvised Theory and practice Evaluated as not effective. Led to decision to change Course Date, length, and site Participants Institution administering / funding Material and curriculum Methods Evidence of impact / comments Toropalca UNICEF (financed) focus away from health promoters and toward “madres vigilantes” RPS and TBAs 3 rd quarter 2000 X 3 days, Cotagaita 20 community members MOH UNICEF (financed) Own materials— improvised Theory and practice Evaluaated as not effective. Led to decision to change focus away from health promoters and toward “madres vigilantes” EPI II 2 nd quarter 2000 X 4 days, Cotagaita Cotagaita district MCDI UNICEF (financed) SEDES (administered) Immunization cards, MOH EPI II manual Theory and practice New (pentavalent) vaccines introduced successfully. EPI management improved Supervision and leadership 4 th quarter 2000 X 2 days, Toropalca 8 MOH 1 MCDI CARE Manual on supervsion and leadership Theory Not evaluated Participatory training methodology February 2001 X 4d, Cotagaita Cotagaita district MCDI Núr University (adminstered) MCDI (financed) Materials from Núr Cochabamba Theory and practice Beginning of community organization activities Supervision and monitoring of IMCI March 2001 X 2 days, Tupiza Tupiza training center facilitators 4 Cotagaita district 2 MCDI PROSIN and SEDES Material from PROSIN, supervision instrument from MOH Theory and practice Beginning of supervision for IMCI by MCDI team (District personnel not supervising) Humanized delivery March 2001 X 2 days, Cotagaita Cotagaita district (all) MCDI (all) CARE (Chilean nurse) Manual of humanized delivery, neonatal resuscitation, positioning for Theory, much practice, video Increased number of deliveries being performed in facilities. Mothers recognize improved treatment. Course Date, length, and site Participants Institution administering / funding Material and curriculum Methods Evidence of impact / comments breastfeeding, danger signs during delivery, newborn. NOTE: did not include prenatal or post-partum issues, nor obstetrical emergencies. Repeat humanized delivery April 2001 X 2 days, Tumusla and Tocla 2 auxiliary nurses MCDI Same Same Same IMCI 6 day course April 2001 Tupiza May 2001 Potosí July 2001 Tupiza November 2001 Potosí 12 auxiliaries 10 auxiliaries 12 auxiliaries 5 MDs, 7 RNs Training centers Tupiza and Potosí; SEDES, PROSIN MCDI (financed) PROSIN materials. Theory, practice IMCI implemented as routine by all participants. HH/C-IMCI July 2001 X 6 days, Cochabamba 4 MCDI 1 MOH 1 Tupiza Núr University MCDI (financed) Materials from PROSIN/Núr pilot project Theory and practice MCDI began implementing in communities Seminar in HH/C-IMCI for NGOs August 2001 X 2 days, Cochabamba 6 NGOs, Núr, PROSIN, USAID, MCDI PROSIN MCDI Workshop Workshop Sharing experiences methods. Method adjusted. BCC/IEC Workshop 2 nd quarter 2001 X 2days, Cotagaita 4 MCDI 31 Cotagaita district CARE How to produce and validate IEC materials Theory and practice MCDI producing and validating IEC materials using methodology from workshop Municipalty / family project and growth August 2001 X 2 days, Cotagaita 48 Cotagaita district 3 MCDI SEDES Breastfeeding manual, growth monitoring Theory and practice Improved registration of growth monitoring Course Date, length, and site Participants Institution administering / funding Material and curriculum Methods Evidence of impact / comments monitoring materials, active case-finding National Health Insurance program July 2001 X 2 days, Cotagaita All Cotagaita district staff 3 MCDI SEDES New regulations for National Health Insurance program Theory and practice New regulations successfully implemented Methodology for cost-study in primary health care for the National Health System February 2001 X 2 days. Santa Cruz 6 Núr University graduate school 3 Municipality of Santa Cruz 1 Municipality of Cotagaita 2 MCDI MCDI-USA Own materials. Covered objectives and data-collection for cost study. Did not include data analysis. Theory Validation of data￾collection instrument. Adjustments to software. Data-collection began in Cotagaita district. ATTACHMENT F: COST STUDY PROPOSAL Medical Care Development International 1742 R Street NW, Washington, DC 20009 USA Telephone: (202) 462-1920; Fax: (202) 265-4078 Internet Electronic Mail: mcdi@mcd.org URL: http://www.mcd.org Central Potosi Child Survival Project Cost and Financing Analysis of Child Survival Services Supplied under the Seguro Basico de Salud in Cotagaita and Puna Districts Study Design 16 November 2000 MCDI 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 1 Table of Contents Introduction .................................................................................................................. 2 Project Description........................................................................................................ 5 General Objectives..................................................................................................... 5 Specific Objectives .................................................................................................... 5 Study Design............................................................................................................. 6 Timeline and Level of Effort .....................................................................................10 Technical Assistance Team...........................................................................................12 Organizational Goals and Structure...........................................................................12 MCDI’s Proposed Technical Assistance Team..........................................................14 Qualifications and Experience of the Consultants..................................................14 Cost Study Coordinator and Senior Economist – Dr. Christopher Schwabe.......14 Cost Study Clinical and Public Health Specialist – Dr. Joao Blasques Oliveira...24 Annex 1: MedCost©......................................................................................................26 Benefits of MedCost©-IMCI .....................................................................................28 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 2 Introduction In December 1998 the Government of Bolivia adopted a national health insurance scheme, the Seguro Basico de Salud (SBS), to provide essential health services free of charge to women, newborns and children, adolescents and the elderly through the existing network of government health facilities, the Social Security network of services, NGOs, churches and others. The objective of the SBS is to lower child, infant and maternal mortality, and to reduce the risk, duration and severity of the principal causes of morbidity and mortality in the population. It emphasizes services for children under 5 years of age, women of reproductive age to include pregnancy, reproductive health, and detection and treatment of cholera, malaria and tuberculosis. The SBS replaces and expands upon the former Seguro Nacional de Maternidad y Ninez (SNMN) which was introduced in 1996 to provide coverage for pregnant women and children under five. The services covered under the SBS include those incorporated in the Integrated Management of Childhood Illness (IMCI) protocol1 , those identified under the Mother￾Baby Package (MBP)2 , as well as the treatment of cholera, meningitis, tuberculosis, the transport of emergency obstetrical cases, outreach visits to communities without direct access to a health facility, and the cost of alcohol and fuel for immunizations. Under the SBS, all incorporated health facilities (including all government facilities) provide covered services free of charge to patients, and are reimbursed by the municipal health authority for the direct non-labor variable costs of drugs and medical supplies, selective laboratory examinations, obstetric ultrasound examinations, neonatal incubation, alcohol and fuel for immunization, transporting emergency obstetrical cases, and conducting outreach visits to communities without a health facility. A unique set of reimbursement rates applies under the SBS to all health facilities regardless of level (i.e., reimbursement rates for health posts are identical to those for hospitals for the same service)3 . These reimbursement rates are intended to finance only the direct non￾labor variable costs of covered services. Labor costs are financed under the Ministry of Health budget at the Prefectura level, and indirect costs including maintenance and repair 1 This includes coughs or colds, pneumonia, bacterial infections, diarrhea, fever, malaria, measles, ear infections, malnutrition, anemia, and immunizations. 2 This includes abortion complications, anemia, antenatal care, eclampsia, family planning, hemorrhage, neonatal complications, normal deliveries, obstructed labor, postpartum care, sepsis, and STDs. 3 Since higher level facilities tend to treat more complicated cases that require more and/or higher cost drugs and supplies, reimbursement rates are probably higher on average at the higher level facilities. This is most certainly the case when comparing health posts with tertiary hospitals, but may not be the case when comparing tertiary and secondary level hospitals. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 3 of equipment, transport and buildings are financed the Municipios or Alcadia. A comparison of the reimbursement rates under the SBS with those that prevailed under the SNMN (see Table 1) indicates that rates have been increased for a number of services (e.g., Normal Deliveries (+167%), Induced Labor (+160%), Hemorrhage during 3 rd trimester (+109%), Neonatal Sepsis and Pneumonia in neonates (both +106%)), but have been lowered for others (e.g., Preeclampsia (-55%), Neonatal Asfixia Gravia (-25%), and Maternal Sepsis (-13%)). In addition, the new SBS reimbursement rate structure includes a number of conditions that were not covered under the SNMN (e.g., Neonatal Meningitis, Childhood Bacterial Infections, Ear Infections, TB, Cholera, Malaria, and STDs). Table 1 also contrasts the new SBS reimbursement rates to the average direct non-labor variable cost estimates derived by Abt Associates in their 1998 evaluation of the SNMN adjusted for inflation4 . As can be seen, it is evident that the SBS rates were not predicated on the SNMN cost estimates, and thus in many cases the estimated direct non-labor variable costs considerably exceed the prevailing reimbursement rates. This is particularly the case for prenatal examinations, childhood pneumonia, childhood IRA without pneumonia, and childhood diarrhea and dysentery handled on an ambulatory basis. Moreover, in most cases it evident that the gap between cost and reimbursement rates increases the higher the facility level given that reimbursement rates are the same regardless of level of care. It is not clear why the new SBS rate structure is not based on the cost estimates derived from the SNMN evaluation, but it is possible that this is due to the recommendation emanating from the study and the study dissemination workshops5 that further cost analyses be conducted to more accurately derive the total costs of producing child survival services taking into account institutional, geographic and epidemiological differences as well as short run fixed costs (see more below). Based on discussions with the principal author of the Abt study, it is also likely that government is seeking cost estimates based on a technically optimal mix of inputs6 . It is also possible that the SBS rates have not been adjusted given the need to base this type of decision on a more complete assessment of the overall operating costs of government health facilities, the efficiency with which services are produced, and an assessment of the performance of other financing sources (the government budget, and private out-of-pocket payments by patients). Finally, it is also evident that the cost basis for the additional services added into the SBS relative to the SNMN has not yet been evaluated. 4 Dmytraczenko et al. 1998. Evaluacion del Seguro Nacional de Maternidad y Ninez en Bolivia. Informe Tecnico 22. Bethesda, MD: Partnership for Health Reform Project, Abt Associates Inc. 5 See Capra Seoane et al . 1998. Bolivia: Difusion de los Resultados de la Evaluacion del Seguro Nacional de Maternidad y Ninez. Bethesda MD: Abt Associates Inc. 6 The Abt study evaluated the direct non-labor costs of providing maternal and child health services based on the prevailing treatment protocols obtained from interviews with clinical staff at each facility. Though they noted that there was not a great deal of variation between facilities, they did not apply a standard treatment algorithm that had been sanctioned by the MOH. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 4 Table 1: Reimbursement Rates Under SNMN and SBS as Compared to Non-Labor Direct Cost Estimates Condition SNMN(2) SBS(3) % Change SNMN->SBS Health Posts and Health Centers 2nd Level Hospitals 3rd Level Hospitals Health Posts and Health Centers 2nd Level Hospitals 3rd Level Hospitals Maternidad Ambulatorias Prenatal (costo par consulta) / Consulta prenatal 2.00 3.00 50% 56.79 61.15 109.20 1793% 1938% 3540% Consultas repetidas 24.02 18.56 48.05 Consulta prenatal en embarazo conriesgo obstetrico 2.00 Atencion de post-parto (puerperio) / Consulta post-parto 2.00 3.00 50% Pre-eclampsia (leve moderada) 55.00 25.00 -55% 31.67 38.22 45.87 27% 53% 83% Attencion parto limpio (atencion por agentes comunitarios) 15.00 Hospitalarias Conduccion del parto / Induccion de trabajo de parto 15.00 39.00 160% 74.26 85.18 90.64 90% 118% 132% Amenaza de parto prematuro 250.00 287.00 15% Parto normal y atencion al recien nacido 45.00 120.00 167% 58.97 68.80 63.34 -51% -43% -47% Parto y episiotomia 79.72 99.37 95.01 Cesarea y atencion al recien nacido 250.00 330.00 32% 383.30 399.68 16% 21% Prevencion hemorragia 1ra mitad de embarazo 60.00 Tratamiento hemorragia 1ra mitad de embarazo 270.00 Homorragia de tercer trimestre (placenta previa y desprendimiento prematura) / Meorragia 2da mitad de embarazo 115.00 240.00 109% Hemorragio puerparal (atonia uterina y retencion placentaria) 90.00 138.00 53% Eclampsia y pre-eclampsia severa 100.00 105.00 5% 226.05 259.90 115% 148% Infeccion puerperal / sepsis 360.00 315.00 -13% Sepsis obstetrica 465.00 303.58 411.69 -35% -11% Recien Nacido Ambulatorias Infecciones baterianas locales 5.00 Infecciones baterianas graves (pre-referencia) 4.00 Hospitalarias Ictericia patologica de recien nacido 15.00 Recien nacido con sufrimiento fetal (asfixia grave) 80.00 60.00 -25% Sepsis neonatal 175.00 360.00 106% 477.21 712.00 33% 98% Neumonia en recien nacido a menores de dos meses 175.00 360.00 106% 325.42 589.69 -10% 64% Meningitis <2 meses 360.00 Ninos de 0 a 5 Anos Ambulatorias Infecciones baterianas graves (pre-referencia) 3.00 Neumonia de manejo ambulatorio / Neumonia no grave 2.00 3.00 50% 22.93 30.58 32.76 664% 919% 992% IRA sin neumonia 2.00 9.83 16.38 19.66 391% 719% 883% Infeccion aguda del oido 5.00 Diarrea de manejo ambulatorio / Diarrea sin deshidratacion 2.00 2.50 25% 18.56 17.47 16.38 643% 599% 555% Diarrea con deshidratacion 3.50 18.56 17.47 16.38 430% 399% 368% Disenteria 15.00 22.93 33.85 37.13 53% 126% 148% Consulta crecimiento y desarollo 3.00 Anemia 5.00 Suplementacion con aciete vitamnado 3.50 Parasitosis intestinal 1.00 Hospitalarias Neumonia grave hospitalaria 160.00 236.00 48% 66.61 891.09 1454.58 -72% 278% 516% Infecciones respiratorias agudas sin neumonia 2.00 Diarrea de manejo hospitalario / Diarrea con deshidratacion grave 40.00 50.00 25% 131.04 116.85 162% 134% Meningitis 2 meses a < 5 anos 500.00 Otros Ambulatorias Consulta tuberculosis 3.00 Colera (diagnostico presuntivo) 7.00 Malaria 3.00 Consulta planification familiar 4.00 Sifilis 8.00 Gonorrea 10.00 Candidiasis 10.00 Clamidiasis 4.00 Tricomoniasis 3.00 Diarrea (atencion por agentes comunitarios) 2.50 Neumonia no grave (atencion por agentes comunitarias) 2.50 Otros Hospitalarias Malaria grave 55.00 Colera con deshidratacion grave 85.00 Laboratorio y Rayos X Grupo sanguimeo y rH 2.00 2.50 25% Hemograma (mas Hb y Rec Plaq) 3.00 6.50 117% Examen de Orina o Uroanalisis 3.00 3.50 17% VDRL 2.00 Average Direct Variable Costs(1)[1] % Difference between SBS Prices and Cost Estimates 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 5 Project Description General Objectives The purpose of the child survival cost and financing analysis is two-fold: (1) to evaluate the impact of the Seguro Basico de Salud (SBS) on the sustainability of child survival services provision in two districts of Central Potosi, and (2) to initiate a process of establishing an accounting system that will enable health authorities to be able to track costs and eventually to negotiate or set rates that take costs into account. Specific Objectives The study will assess the full operating costs of all health service provision at the facility level under existing technologies (i.e., prior to the Project’s interventions), and will contrast this with the costs of service provision under optimal technologies (i.e., in accordance with national standards) in order to evaluate current efficiency and identify specific aspects of production that could benefit from performance-enhancing investments. The cost analysis will be undertaken within the context of a complete cost assessment of the facility as a whole in order to derive estimates of the full operating costs of service provision (i.e., including indirect costs of overhead and support services) and in order to contrast these full operating costs with total revenues derived from the SBS, the Government budget, and private payments. Positing the analysis within a complete assessment of the fiscal position of the health facilities will permit us to evaluate the impact of alternative production and financing scenarios for child survival interventions (including possible changes in production technologies, adjustments to the reimbursement rates, modifications to budgetary provisions, or changes in fees-for-service or other changes) on service sustainability, taking into account the trade-offs or implications for the supply of other services. The second purpose of the study is to begin to develop capacity both at the Prefectura level and DILOS level to evaluate financial performance and produce financial plans based on an accounting system that allows them to track costs and finances. As part of the process of introducing systems for tracking costs, MCDI will introduce and use an adapted version of its facility and service costing software, MedCost© - a child survival specific version that will facilitate future costs assessments in Bolivia and elsewhere (see Annex 1 for a description of MedCost©). The objective is to create a tool that complements WHO’s Mother-Baby Package and IMCI Recurrent Cost spreadsheets (which are prescriptive in nature) to evaluate the actual cost of child survival service provision within the context of overall facility costs under real life conditions. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 6 Finally, MCDI seeks to utilize the study process as a practicum for students at Nur University, MCDI’s NGO partner, and as the basis for producing theses under the guidance and supervision of MCDI’s Health and Public Finance Economist, Dr. Schwabe. Study Design The study will be based on the sample of health posts, health centers and hospitals in the municipios of Cotagaita and Vitichi selected by MCDI for its Facility-Based Assessment undertaken in preparation for its Detailed Implementation Plan for the current Project. A listing of the proposed facilities is presented in Table 27 . Table 2: Health Care Facilities to be Included in the Cost Analysis N Facility Name Number of Beds Health Posts 1 Yawisla 3 2 Pecajsi 2 3 Calila 0 4 Vichacla 3 5 Mocko pata 2 6 Ckara ckara 3 7 Thapi 4 8 Collpa uno 3 9 Chui chui 3 10 Laytapj 2 11 Cazón 3 Health Centers 12 Toro palca 2 13 Palca de Higueras 3 14 Hospital San Vicente Paul (Vitichi) 4 15 Calcha 6 Hospital 16 Hospital Cotagaita (San Juan de Dios) 12 7 Two alternative extension scenarios to the study design could be envisaged depending upon the availability of supplementary funding: (1) extending the analysis to all facilities in the municipios of Cotagaita and Vitichi with the view to fully establishing a cost-tracking system and to providing a complete financial assessment of these municipios; and/or (2) extending the analysis to other municipios in Potosi Province or elsewhere in Bolivia. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 7 The study will derive the full operating cost of service delivery based on current technologies employed at all health facilities, as well as the cost of services based on an optimal technology consistent with Government standards and treatment protocols. In the former case, the study will utilize MCDI’s MedCost© software to conduct a complete inventory of inputs used at each facility to produce child survival and other health services. Inputs will be inventoried by service department and cost center to reflect the prevailing organizational and production characteristics of each facility, and in order to focus on the final services reimbursed on the SBS, which include child survival services. Three types of cost centers will be defined as relevant depending on the size and scope of service delivery at the various types of facilities: (i) overhead, (ii) intermediate support service, and (iii) final. Overhead cost centers will include service departments or functions that exclusively produce support services for other departments within the facility (e.g., Administrative services, laundry, kitchens etc). Intermediate support service cost centers will include service departments that provide support not only to other departments within the facility, but also supply services directly to patients (e.g., pharmacy department, laboratory, etc.). Final service cost centers will include service departments that only supply services directly to patients (e.g., outpatient: maternal care: preeclampsia, outpatient: maternal care: post-natal examination, inpatient: under 5: diarrhea with severe dehydration, etc.). The analysts (MCDI, Nur University and local health officials) will use MedCost© to distribute or share the overhead costs to the appropriate intermediate and final cost centers, and to distribute the intermediate department costs to the final cost centers. In this way, the unit cost estimates not only incorporate the direct costs of service production (i.e., the cost of those inputs used exclusively to produce the final goods and services), but they also incorporate the indirect costs of overhead or intermediate support services that are also required to efficiently and effectively produce the final goods and services. To achieve this, MedCost© employs an accounting-based cost estimation algorithm that relies on accepted accounting procedures for classifying costs as either direct or indirect, and for attributing the indirect costs to intermediate and final service departments using a set of “step-down” allocation rules. The step-down accounting cost methodology generates an estimate of the cost of production at a given observed level of output subject to prevailing technologies8 . It is essential for the cost study to not only evaluate short run variable costs (as was done 8 MedCost© is indispensable because step-down costing requires a series of inter-related decisions to be made about the distribution of overhead and intermediate costs to intermediate and final cost centers, and because allocation decisions need to be made for each input used in the production of services, the methodology is not easily or readily implemented on a manual basis. The software serves not only as a database entry environment for recording all inputs used in the production process and their associated costs, but also automatically carries out the step-down costing algorithm and generates summary unit costs. Because the software is multi-lingual and because it operates as an add-in to Microsoft Excel®, it is anticipated that the provincial and central level health authorities in Bolivia will be able to use it to maintain up-to-date inventories and costings of its health services. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 8 in the very useful Partnership for Health Reform evaluation of the Seguro Nacional de Maternidad e Ninez), but also to evaluate the short run fixed costs9, in order to derive unit total costs by final cost center. A full and complete costing is essential for evaluations of sustainability, for assessments of efficiency, and ultimately for rate setting for non￾publicly subsidized health care providers that seek to adhere to the SBS. The determination of the units used to derive unit costs will be based on a cost center classification scheme that conforms to the SBS reimbursement structure, the child survival service mix, and the prevailing organizational and management characteristics of the facilities. Table 3 provides an indicative list of the units costs that will be derived in the study at the hospital level – a list which will need to be refined in discussion with health officials in Bolivia. As indicated above, in addition to evaluating the full and complete cost of service production under prevailing technologies, the study will also evaluate the costs of production under an optimal technology that reflects government standards and treatment protocols10. In this way, the study findings will not only be used to evaluate the adequacy of current financing arrangements relative to prevailing production technologies (i.e., adequacy of public expenditures, SBS reimbursements and other revenues for financing all operating costs including child survival-related services), but will also be used to evaluate the relative efficiency of different facilities as currently configured both relative to other facilities given current technologies and relative to the optimal technologies prescribed by Government standards. This methodology is essential from the standpoint of devising strategies for improving the cost-effectiveness of service delivery. The study will also attempt to extend the work undertaken by PHR to obtain more accurate information on actual drug costs at the local level, taking into account pricing differentials that reflect in-country logistical variations, and the need to rely on private sources of drugs and medical supplies. The study will also not only account for actual expenditures on indirect overhead and support services, but will also attempt to the extent to which these costs are under or un-funded. 9 Short run variable costs are costs that vary with the volume of output during the short run period that is usually defined as a period of time during which the quantity of some inputs to the production process (e.g. labor) are fixed (i.e., their quantity cannot be changed). For the purposes of this study, the short run is defined as one year (assuming, for example, that the re-deployment of personnel takes place once a year). An example of a variable cost is the cost of drugs and medical supplies which increases as the volume of prescriptions or consultations increases. Fixed costs are costs that do not vary with the volume of output. An example of a fixed cost is the cost of equipment since, at least in the short run, the quantity of equipment cannot be changed, and so does not vary with the level of output. 10 Where standards have yet to be defined, MCDI will rely on discussions with the MOH to develop an approximate set of standards for analytic purposes. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 9 Table 3: Unit Costs to be Evaluated at the Hospital Level · Cost per outpatient consultation by: ÿ Maternal care by o Prenatal Examination (with and without obstetrical risk) o Postnatal Examination o Preeclampsia ÿ Neonatal and Under 5 care by o Bacterial infections (local and serious) o ARI without pneumonia o Pneumonia (non-severe) o Diarrhea (with and without moderate dehydration) o Dysentery o Acute Ear Infections o Anemia o Intestinal Parasitosis ÿ Other Ambulatory o Tuberculosis o Cholera o Malaria o Family Planning o Sexually Transmitted Diseases o Other · Cost per admission by: ÿ Maternal care by o Induced labor o Threat of premature delivery o Normal delivery o Normal delivery with episiotomy o Cesarean o Prevention of hemorrhage – 1st half of pregnancy o Treatment of hemorrhage – 1st half of pregnancy o Hemorrhage – 2nd half of pregnancy o Puerperal Hemorrhage o Eclampsia and severe Preeclampsia o Puerperal infection / sepsis ÿ Neonatal and Under 5 care by o Icteric pathology among neonates o Fetal asphyxia o Neonatal sepsis o Severe pneumonia o ARI without pneumonia o Diarrhea with severe dehydration o Meningitis ÿ Other Inpatient o Severe malaria o Cholera with severe dehydration o Other · Cost per laboratory examination · Cost per surgical case · Cost per pharmacy prescription · Cost per radiology or imaging examination · Cost per emergency transport 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 10 Timeline and Level of Effort The timeline and level of effort outlined in this section are based on funding available under the Central Potosi Child Survival Project only. MCDI would be very interested in extending this study to other facilities in Bolivia should additional funding become available. In this case, MCDI is fully prepared to submit a revised proposal and budget estimate to cover this extension. The Gantt chart on the next page presents the proposed workplan and timeline for the Study. Table 4 presents the proposed level of effort required to implement the study and disseminate the findings. Table 4: Level of Effort for Cost and Financing Study Consultant Function Person Days Unit Cost Total Cost Dr. Schwabe Preparation of survey instruments and adaptation of MedCost for child survival services 3 $365 $1,095 Dr. Blasques " 1 $270 $270 Dr. Schwabe Trip 1: Introductory meetings with MOH, SBS, USAID, PHR in La Paz 2 $365 $730 Dr. Schwabe Trip 1 continued: Introduction and planning meeting with MCDI project staff and Nur University; Identification of Study Team 1 $365 $365 Dr. Schwabe Trip 1 Continued: Introduction and planning meeting with MOH in Potosi and Cotogaita and travel 3 $365 $1,095 Dr. Schwabe Trip 1 Continued: Refinement and finalization of study methodology, sample and survey schedule, survey instruments 3 $365 $1,095 Dr. Schwabe Trip 1 Continued: Training of Study Coordinators (surveying and use of MedCost); Pre-test and finalization of survey instruments 5 $365 $1,825 Dr. Schwabe Trip 1 Continued: Debriefing in La Paz with MOH, SBS, USAID, PHR 1 $365 $365 Nur University Students Survey of health facilities, data entry in MedCost, and production of initial cost and fiancing tables 45 $0 Dr. Schwabe Data review and cleaning during survey period 3 $365 $1,095 Dr. Blasques " 3 $270 $810 Dr. Schwabe Production of draft report 5 $365 $1,825 Dr. Blasques " 1 $270 $270 Dr. Schwabe Presentation and discussion of findings in Bolivia 5 $365 $1,825 Sub-total, Dr. Schwabe 35 $365 $12,775 Sub-total, Dr. Blasques 5 $270 $1,350 Sub-total, Nur University 45 $0 Total $14,125 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 11 (Insert Gantt chart here} 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 12 Technical Assistance Team Organizational Goals and Structure Medical Care Development International (MCDI) is a division of Medical Care Development, Inc. (MCD), a non-profit organization incorporated in 1966 in the state of Maine. The original mission of the organization was to provide health services to underserved rural communities in the U.S. To this end, MCD developed and managed a network of community health centers, established a statewide EMS system, and developed residential care programs for special population groups. In 1977, the International Division was established with support from the federally funded Regional Medical Program to adapt MCD’s successful approach to health program development in the U.S. to meet the needs of disadvantaged and vulnerable population groups in developing countries. The Mission Statement of MCD’s International Division was updated during a 1997 Strategic Planning process, and can be found in the sidebar to the right. MCDI has worked in nearly 40 countries implementing health sector projects supported by various donors including USAID, the International Office of the U.S. Public Health Service, The World Bank, the African Development Bank and, most recently, the European Union. In 1997, Medical Care Development International (MCDI) was officially designated as a US NGO in Consultative Status with the Economic and Social Council of the United Nations. The mission of MCDI is to enhance the well being of peoples and communities in developing nations through superior technical assistance in health and socio-economic development. We will seek to empower families with the knowledge and behavior needed to improve infant and child survival and maternal health and care. We will develop and disseminate tools, mechanisms and strategies that improve access and management of sustainable levels of health care services; we will develop broadly defined health-enhancing support systems. We will work with a full spectrum of organizations, from grassroots community groups to multi-lateral donor institutions, to enable continuing and progressive improvements in the quantity and quality of care available and affordable to the neediest populations on earth, and seek to enhance their financial ability to benefit from these improvements. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 13 MCDI's core capabilities include child survival and community health development; health system management; health system planning; health economics and health care financing; financial planning and management; water and sanitation projects; and institutional strengthening. Nonetheless, the diversity of programs conducted by MCD in both the domestic and international arenas continues to expand. The major program areas in which the organization has established a proven track record include the following: · Health service costing · National Cost Recovery Programs Design and Implementation · Financial Management Systems Design and Implementation · Hospital Financial Management Systems Design · Health Systems Planning, Management and Evaluation · Computerized Health Management Information Systems Design. · Health Facilities Design and Construction Management · Medical Equipment Maintenance and Repair Systems Design · Pharmaceutical Supply and Distribution Systems Design · Manpower Needs Assessment and Projection Modeling · Health Manpower Training (including senior ministry personnel, primary care practitioners, nurses, educators, technicians, and village health workers). · Health Education Program and Curriculum Evaluation · Design and Implementation of Information, Education and Communication Programs · Community Organization and Participation · Child Survival Program Design and Implementation · Micro-credit and Economic Empowerment Programs · Emergency Medical Service Systems Design and Implementation · Emergency/Disaster Management Systems Design · Water Supply and Sanitation Systems Design and Implementation · Communicable and Chronic Disease Control MCDI activities are funded by grants and contracts with The World Bank, United States Agency for International Development (USAID), African Development Bank (ADB), Inter-American Development Bank (IADB), Pan American Health Organization (PAHO), United States Department of Health and Human Services, National Institutes of Health (NIH), the Centers for Disease Control (CDC), and private commercial organizations. MCDI relies on full-time staff whose areas of expertise include clinical medicine, public health, economics, planning and evaluation, education, health administration, financial management, communications, nutrition, architecture, engineering, nursing, epidemiology, and survey research. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 14 The strength of the senior staff is more than academic. Many have entered their specialization at the program implementation level, later moving in to management and policy-making positions. They have lived and worked in many developing countries, and their cross-cultural professional experiences have been important factors in the success that MCDI has enjoyed. Staff expertise is supplemented by a cadre of senior consultants available through MCD's affiliations with universities, private consulting firms, and technical centers throughout the world. Over the years, MCD staff have acquired sound and proven skills in contract management and technical assistance services management. Financial management and reporting systems have been incorporated into operational manuals for MCD home office and field staff to ensure the uniform, reliable, and consistent report of project expenditures. MCDI’s Proposed Technical Assistance Team Based on our institutional philosophy with respect to the selection of personnel, as well as our understanding of the working environment in Mozambique, MCDI proposes that Dr. Christopher Schwabe serve as the Cost Study Coordinator and Senior Economist, that Ms. Jacky Mahon serve as the Cost Study Service Management Specialist (Health Economist), and that Dr. Joao Blasques Oliveira serve as the Cost Study Clinical and Public Health Specialist. Qualifications and Experience of the Consultants Cost Study Coordinator and Senior Economist – Dr. Christopher Schwabe. Education 1993 Ph.D., Economics, Syracuse University. Fields of specialization: Public finance, health economics, labor economics. Dissertation: “Assessing the Equity Implications of User Fee Financing of Health Care.” 1993 MA., Economics, Syracuse University. 1980 BA., with Honors, Economics, Haverford College. Honors thesis: "The Political Economy of Agricultural Development in the Sudan". 1978-79 Princeton University, Special Languages Program. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 15 Professional Experience 11/91-Pres. Health Economist/Public Finance and Health Care Financing Specialist, Medical Care Development International. Health Sector Finance Software Development: Responsible for ongoing development of Medsolve© -- an interactive, and multi-lingual Windows-based, health sector financing program which consists of the following integrated sub-modules: (1) MedCost©, a facility and service step-down costing software; (2) Investment Plan and Budgeting Module which can be used either to generate program and line-item budgets based on detailed health sector plans, or to conduct cost-effectiveness analyses of alternative program delivery strategies; (3) Personnel Module which (a) inventories and costs current health sector manpower supply by program classification, by occupation, by health facility or institution affiliation, and by geographic area, (b) projects future supply, (c) generates demand-based estimates of health sector manpower requirements by occupation; and (d) generates costs a gap analysis between requirements and future supply; (4) Drug Procurement, Inventory Management and Pricing Module which manages the procurement, storage, distribution, pricing and sale of drugs; and (5) Health Service Pricing Module which calculates the recurrent costs of health service delivery at individual health facilities, and generates break-even prices for health services under alternative pricing and subsidization policies. Designed and programmed a Health Facility Requirements Planning Model that projects the future requirement for hospitals and health centers taking into account the impact of the following demand and supply side factors on inpatient and outpatient services utilization: (1) Demographic changes including internal migration, emigration, expected changes in fertility, and changes in mortality associated with the AIDS epidemic; (2) Major epidemiological shifts such as the AIDS/TB/STD epidemic that has already substantively increased the demand for inpatient and outpatient care; (3) Policy changes affecting the pricing of care and the quality of care provided, which in turn affect the demand for services; (4) Supply side improvements in clinical case management and referral practice between health centers and hospitals; (5) Supply side improvements in preventive and public health service delivery that reduce the demand for curative care; and, (6) Supply side developments in mental health service capacity that increases the demand for inpatient and outpatient care. Designed and programmed an Emergency Service Ambulance Demand Model that projects the future requirement for ambulances based on a probabilistic model that minimizes the likelihood of queuing for an ambulance once an emergency dispatch call has been placed. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 16 Consultative services in health economics, health financing, health planning and management: Angola: Health Economist/System’s Analyst. Work includes: Reviewing and assessing Ministry of Health financial planning and management systems at the central government and provincial levels. Designing and assisting in the implementation of a new financial information system based in part on the use of the MedSolve© Planning and Budgeting Module, the MedSolve© Personnel Module, and the MedSolve© Drug Module, and a financial Management database application developed in Microsoft® Access that interfaces with the Government’s computerized Budget and Expenditure Control System. Chad: Public Finance and Health Economist, USAID-Financed Child Survival Project. Work included: (1) a cost-effectiveness analysis of alternative drug procurement and distribution systems for decentralized (district) health services; (2) a detailed analysis of pharmaceutical pricing and the elaboration of a break-even pricing approach that simultaneously achieves distributive equity objectives; (3) the design of a decentralized financial management system for cost recovery; (4) an evaluation and report on past cost recovery experience in Chad; and, (5) an assessment of the future potential for cost recovery in the province of Moyen Chari. The Gambia: Chief-of-Party and Public finance and Health Economist on the ADB￾funded Health Sector Requirements Study. Work included: (1) major responsibility for the preparation of MCDI’s technical proposal to the African Development Bank and the government of The Gambia; (2) organization and technical oversight responsibility for the Health Sector Requirements Study as Chief-of-Party; (3) institutional design analysis of the Gambian health sector performance in the areas of health finance, planning and management, manpower planning and management, pharmaceutical supply, equipment and building maintenance. Guinea: Chief-of-Party and Public finance and Health Economist, World Bank funded Health Center Pricing Assessment. Served as Chief-of-Party and Health Finance Economist for a 5-person consultant team contracted to assess the existing health center pricing system and to recommend a new, more efficient, and equitable national cost-based pricing system and fee structure that will ensure the sustainability of service provision based on substantial community financing support. Lesotho: Chief-of-Party and Health Facility Planning Specialist, ADB-funded Health Study. Work includes: (1) organization and technical oversight responsibility for the Health Study; (2) preparation of a national medium term health facility plan for hospitals and health centers using the Health Facility Requirements Planning Model (see above); and (3) situation analysis and project design specification for improving administrative management performance over the medium term. Lesotho: Public finance and Health Economist, Irish AID-funded CHAL-GOL Partnership Study. Work has included (1) a detailed financial analysis of the Christian Health Association of Lesotho hospitals, an evaluation of the current subsidies provided by the Government of Lesotho, and a critical assessment of the contractual agreement 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 17 between the two parties, and articulation of a new financing strategy, (2) articulation of a new categorical, matching and equalizing grant and drug trading account public funding formula for the CHAL hospitals; (3) facilitation of a set of meetings between the GOL and CHAL to negotiate a new financing strategy and plan the steps required to implement a new partnership agreement; and (4) development of an Interim Service Provision and Financing Agreement between the GOL and CHAL Institutions. Madagascar: Finance and Health Economist, USAID-funded Child Survival Project. Work has included designing and overseeing the start-up of community-based village pharmacy scheme that is financed through a combination of revenues granted from the Beza Mahafaly Special Reserve (derived from gate receipts), individual pre-payments for drugs and health care services, and revenues from the direct sale of essential drugs. The scheme is piloting a community-managed collateral fund that ensures participants access to care on credit. The scheme is linked to the Bamako Initiative that is being introduced in public health centers throughout Madagascar. Mozambique: Public finance and Health Economist, USAID-funded Child Survival Project. Work has included an analysis of the cost of hospital and health center service provision in Niassa Province and an assessment of the performance the current user fee system. The cost analysis was derived using MedCost© (see above). Plans are now underway to evaluate the cost and financing of IMCI service provision. Nigeria: Public Finance and Health Economist, ADB-funded Institutional Strengthening Project. Work included: (1) preparing and teaching a 3-week intensive course on program planning and budgeting (including how to conduct feasibility studies/evaluations using cost-effectiveness analysis); (2) preparing and teaching a 1 week intensive course on health sector manpower planning (including how to estimate future manpower supply and manpower requirements); (3) a comprehensive public finance assessment of Bauchi State with particular reference to health sector financing, including a fiscal analysis of the capacity of local governments to finance decentralized PHC services; (4) a study of manpower supply and demand in Bauchi State; and (5) a preliminary analysis of Bauchi State Drug Revolving Fund performance. South Africa: Health Economist, USAID-funded Child Survival Project. Authored a cost-effectiveness analysis of decentralized dipstick-dot immunoassay approach to the detection of antibodies to HIV 1 and 2 versus the current centralized 3-test ELISA approach. Swaziland: Chief-of-Party and Public Finance and Health Economist on the ADB￾funded Health Sector Study. Work included (1) significant responsibility for the preparation of MCDI’s technical proposal to the African development Bank and the Government of Swaziland; (2) organization and technical oversight responsibility for the Health Sector Study; (3) analyses of the Swaziland health sector performance in the areas of health finance, and financial planning; and (4) project design specification for improving health sector performance over the medium term. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 18 Tanzania: Public finance and Health Economist, World Bank-funded IDA HNP External Evaluation. Work involved evaluating the Health and Nutrition Project components related to community financing (user fees and prepayment schemes), national health insurance, and drug sales under a restructured national pharmacy system. Provided technical oversight for the evaluation and was responsible for training local counterparts in the evaluative framework employed. Togo: Health Economist, USAID-funded Child Survival Project. Work included: (1) a comprehensive evaluation and report on Togo's health care financing arrangements; (2) an empirical (econometric) assessment of the demand for medical care with revenue projections under alternative cost recovery pricing policies; and, (3) a detailed proposal for restructuring current financing arrangements (including drug and health service pricing) which served as the basis for recently adopted nation-wide health sector pricing policies. Tunisia. Health Economist, World Bank-funded Emergency Health Services Assessment. Work included an economic analysis of emergency health service provision in Tunisia, a projection of future demand for EMS and associated ambulance requirements using an Emergency Service Ambulance Demand Model (see above), an estimation of the future recurrent cost of EMS development in Tunisia. Zanzibar: Chief-of-Party and Health Planning and Financial Management Specialist, ADB-funded Health Sector Study. Work has included: (1) organization and technical oversight responsibility for the Health Study; (2) preparation of a national medium term health facility plan for hospitals and health centers using the Health Facility Requirements Planning Model (see above); and (3) situation analysis and project design specification for improving planning and financial management performance over the medium term. 11/92-1/93 Health Economist and Health Care Financing Specialist, John Snow Inc.: Authored institutional economic analysis of financial sustainability of the Expanded Program of Immunization presented at international conference on immunization sustainability. 1/92-10/92 Health and Public Finance Economist, UNICEF-Pakistan: Planned and provided technical assistance to UNICEF-Pakistan for a detailed assessment of the cost and financing implications associated with achieving the World Summit Goals for children in Pakistan. 12/92 Public Finance and Health Economist, Associates in Rural Development: Co-authored Inter American Development Bank-financed institutional design analysis of factors inhibiting hospital and other social service sector maintenance in Latin America. Assumed principal responsibility for the health and public finance sections. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 19 5/91-10/91 Public Finance and Health Economist, Associates in Rural Development: Côte d'Ivoire and Ghana: Member of USAID-funded missions to Ghana and Côte d'Ivoire to assess the impact of decentralization and structural adjustment policies on the ability of local public and private institutions to finance and supply health and education services. Study employed the new Institutional Design Analysis methodology. Co-author of two country case studies and of the synthesis report on improving governance in Sub-Saharan Africa. 7/89-11/89 Health Economist and Project Director, UNICEF-Sudan: Prepared UNICEF health and education plans of assistance for phase II of Operation Lifeline Sudan. Officer-in-Charge of UNICEF-Juba, and field-based UN representative in government-held areas of South Sudan. Conducted and analyzed a survey of pastoralist households' willingness-to-pay for cattle vaccination to finance jointly administered cattle and child immunizations. 9/86-6/89 Economic Research Assistant, Syracuse University, Syracuse, New York: While a graduate student, co-authored cost-containment analysis for union health and welfare fund (AFL-CIO). Assisted in comparative study of local government education finance experiences in developing countries. 8/83-7/86 Assistant Project Officer for Health, UNICEF-Juba, Sudan: Responsible for programming UNICEF's health assistance for South Sudan. Planned and provided technical assistance to UNICEF-supported urban PHC project run by local government for Juba's 120,000 population. The project was featured in UNICEF's 1985 State of the World's Children report and 1986 Urban Examples. Conducted detailed health expenditure survey of 350 households in Juba. 1/81-7/83 UN Volunteer (Peace Corps), UNICEF-Juba, Sudan. Reports and Publications: 1. 2000. Government of the Kingdom of Lesotho, Lesotho Health Study, Phase I Report. (Editor and author of Administrative Management component, and health care demand projections). Medical Care Development International, Washington D.C. (July) 2. Schwabe, C.L. 2000. Report of Proceedings: Government of Lesotho and Christian Health Association of Lesotho Partnership Meeting. Medical Care Development International, Washington D.C. (June). 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 20 3. Schwabe, C.L. 2000. Interim Service Provision and Financing Agreement Between the GOL and CHAL Institutions. Medical Care Development International, Washington D.C. (June). 4. 2000. Revolutionary Government of Zanzibar, Health Development Requirements Study, Phase I Report. (Editor and author of Policy and Planning components). Medical Care Development International, Washington D.C. (May) 5. Schwabe, C.L., Eddie McGrath and Dan Kaseje. 2000. Government of Lesotho and Christian Health Association of Lesotho Partnership Study. Medical Care Development International, Washington D.C. (March). 6. Schwabe, C.L. and Jacqueline Mahon. 2000. Health Service Costs and Cost Recovery in Cuamba District, Mozambique. USAID Child Survival Project, Mozambique. Medical Care Development International, Washington D.C. (March). 7. Schwabe, C.L. 2000. Cost-Effectiveness of a Decentralized Dipstick-Dot Immunoassay Approach to the Detection of Antibodies to HIV1 and 2 Versus the Current Centralized 3-Test ELISA Approach. USAID Ndedwe Child Survival Project. Medical Care Development International, Washington D.C. (March). 8. Schwabe, C.L, Daniel LeFrancois. Etude Economique sur le Reseau des soins d’Urgence en Tunisie (Economic Analysis of Emergency Medical Services in Tunisia). Medical Care Development International, Washington D.C. (April). 9. 1998. The Government of the Kingdom of Swaziland Health Sector Study Phase II Report – Medium Term Health Sector Strategy and Action Plan. Medical Care Development International. (Editor and author of Financial Management and Health Services Financing components). 10. Schwabe, C. L. 1997. Community Financing of Health Services: Situation Analysis and Proposals for Pilot Initiative. Betioky Child Survival Project, Madagascar. Medical Care Development International, Washington D.C. (July). 11. Schwabe, C.L. 1997. A Preliminary Design Assessment of a Financial Information System for the Ministry of Health. Angola Health Sector Project. Medical Care Development International, Washington D.C. (April). 12. 1997 The Government of the Kingdom of Swaziland Health Sector Study Phase I Report. Medical Care Development International. (Authored demand, health finance, financial planning evaluation, project design sections, and technical “Burden of Disease” assessment of the HIV/AIDS epidemic). 13. 1996. The Republic of The Gambia Health Sector Requirements Studies Phase I, Phase II and Phase III Reports. Medical Care Development International, Washington D.C. (Authored health finance sections, edited the reports, and prepared and consolidated project budgets). 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 21 14. Schwabe, C.L. 1995. Public Sector Drug Pricing: Enhancing Distributive Equity while Assuring Financial Sustainability of Drug Supply Systems. Medical Care Development International, Technical Working Paper Series, Number 1. Washington D.C. (May). 15. Schwabe, C.L. 1994. “Financing of Health Services in the Sudan,” in Omer Ertur and William House (Eds) Population and Human Resources Development in the Sudan. Ames: Iowa State University Press. pp. 83 - 110. 16. Schwabe, C.L. 1994. Public Sector Pharmaceutical Pricing -- Assuring Financial Sustainability and Distributive Equity Under Alternative Drug Supply Modes in Moyen Chari, Chad. Report prepared for USAID and the Government of Chad under contract to Medical Care Development International (September). 17. Schwabe, C.L. 1994. The Bauchi State Drug Revolving Fund: Preliminary Review of the Evidence and Proposal for Future Strengthening. Report prepared for the African Development Bank under contract to Medical Care Development International (April). 18. Schwabe, C.L. 1994. Health Sector Manpower Planning -- Course Notes. Prepared for the African Development Bank under contract to Medical Care Development International (April). 19. Schwabe, C.L. 1993. A Health Sector Manpower Demand and Supply Study for Bauchi State, Nigeria. Report prepared for the African Development Bank under contract to Medical Care Development International (December). 20. Schwabe, C.L. 1993. Analyse de Coût-Efficacité des Modes Alternatifs d'Aquisition, de Stockage et de Distribution des Médicaments -- Projet Survie d'Enfant au Tchad. (Cost Effectiveness Analysis of Alternative Modes of Drug Procurement, Storage and Distribution -- Chad Child Survival Project). Report prepared for USAID and the Government of Chad under contract to Medical Care Development International (November). 21. Schwabe, C.L. 1993. Assessing the Distributive Equity Implications of Health Sector Pricing Policies in Juba, Sudan. Doctoral dissertation completed November, 1992; defended July, 1993. 22. Schwabe, C.L. 1993. Bauchi State Health Care Financing Study, Volume 1: An Analysis of State and Local Government Public Finance with Special Reference to the Health Sector. Report prepared for the African Development Bank under contract to Medical Care Development International (June). 23. Schwabe, C.L. 1993. Health Sector Cost Recovery in Chad with Special Reference to the Province of Moyen Chari: A Review and Evaluation of Past Experience, and an Analysis of Future Potential. Report prepared for USAID under contract to Medical Care Development International (May). 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 22 24. Garnier, Maurice, Louis Seigel, Christopher Schwabe, Susan Wynne. 1993. Maintenance Issues in Infrastructure and Social Services in Latin America and the Caribbean. Report prepared for the Inter-American Development Bank under contract to Associates in Rural Development (March). 25. Schwabe, C.L. and Schwartz, Brad. 1992. Immunization Sustainability: An Institutional Economic Perspective with Special Reference to the Problems of Financial Sustainability in the Philippines and in Kenya. Paper prepared under contract to the REACH project, for presentation at the International Children's Center meeting from December 8 - 10 in Paris, on EPI sustainability in Africa. 26. Schwabe, C.L. 1992. Assessing the Cost and Finance Implications of the World Summit Goals for the 1990s in Pakistan: Final Report. Consultancy report prepared under contract to UNICEF-Pakistan. (December). 27. Schwabe, C.L. 1992. Élaboration d'un Système de Tarification pour les Formations Sanitaires au Togo. (A Pricing Proposal for Health Centers in Togo). Report prepared for the Government of Togo and USAID under contract to Medical Care Development International. (October). 28. Schwabe, C.L. 1992. Assessing the Cost and Finance Implications of the World Summit Goals for the 1990s in Pakistan: An Evaluation of the Data Collection Process. Consultancy report prepared under contract to UNICEF-Pakistan. (July). 29. Fiadjoe, F., Green, D., Schwabe, C.L., and West, T. 1992. Decentralization: Improving Governance in Sub-Saharan Africa. Ghana Case Study. Report prepared for the USAID/AFR/DP under contract to Associates in Rural Development. (March). 30. Schwabe, C.L. 1992. Assessing the Cost and Finance Implications of the World Summit Goals for the 1990s in Pakistan: A Proposed Methodology and Workplan. Consultancy report prepared under contract to UNICEF-Pakistan. (February). 31. Garnier, M., Noel, A., Schwabe, C.L., and Thompson, J. 1992. The Experience in Ivory Coast with Decentralized Approaches to Local Delivery of Primary Education and Primary Health Services. Report prepared for the USAID/AFR/DP under contract to Associates in Rural Development. (January). 32. Schwabe, C.L. 1992. Analyse des Potentialités de Recouvrement des Coûts des Soins de Santé Curatifs au Togo, et Proposition pour une Politique de Tarification Uniforme des Soins Curatifs pour les Établissements Sanitaires Ruraux. (An Analysis of the Potential for Curative Care Cost Recovery in Togo, and a Proposal for a Uniform Curative Care Pricing Policy for Rural Health Facilities.) Report prepared for the Government of Togo and USAID under contract to Medical Care Development International. 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 23 33. Garnier, M., Green, D., Ostrom, E., Schwabe, C.L., Thompson, J. and West, T. 1991. Decentralization: Improving Governance in Sub-Saharan Africa. Synthesis Report. Report prepared for the USAID/AFR/DP under contract to Associates in Rural Development. (December). 34. Schwabe, C.W. and C.L. Schwabe. 1991. “Veterinary Cooperation for Delivery of Primary Health Care to Pastoralists.” Journal of Comprehensive Health (Community Health Association of Southern Africa), Vol. 1, No. 2. pp. 116 - 120. 35. Schwabe, C.L., Schwabe, C.W., Basta, S.S. 1990. Vaccination of Children and Cattle in Pastoral Africa: Practical Intersectoral Cooperation. Working Paper. 36. Johnson, W.J., Schwabe, C.L. 1989. Cost Containment and Health Promotion Strategies for Union Health and Welfare Plans. Prepared under grant from Local 825, International Union of Operating Engineers and the Workplace Health Fund, AFL/CIO. 37. Schwabe, C.L. 1989. Lifeline II Plan of Operations for Primary Education and Health Care in South Sudan. Consultancy report for internal UNICEF use. (November). 38. Schwabe, C.L. 1989. The Accomplish Joint Immunization Project -- An Evaluation of Immunization Coverage and Willingness to Pay for Vaccination. Consultancy report prepared for UNICEF-Sudan. (November). 39. Schwabe, C.L. 1986. “GOBI-FFF: The Basis for Urban Primary Health Care Development in Juba, Sudan.: Urban Examples, February. 40. Schwabe, C.L. 1985. "Community Health -- The Oral Rehydration/ Vaccination Project" in Urban Development and Local Resources. The Inter-Regional Training Project, Development Administration Group, Institute of Local Government Studies, University of Birmingham, UK. pp. 88-98. 41. Schwabe, C.L. 1984. The Expanded Program on Immunization Health Education Demonstration Project in Juba, Sudan: A Review of the Experience. Paper presented at UNICEF Middle East and North Africa meeting to plan UNICEF's regional strategy for promotion of child survival strategies. 42. Schwabe, C.L. 1980. The Political Economy of Agricultural Development in the Sudan. Honors Dissertation. Haverford College, Haverford, Pennsylvania. Languages (max 5/5) Language Speaking Reading Writing English 5/5 5/5 5/5 French 5/5 5/5 4/5 Spanish 3/5 4/5 3/5 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 24 Cost Study Clinical and Public Health Specialist – Dr. Joao Blasques Oliveira Education · 1991: Specialization in Public Health Course, National School of Public Health of Lisbon · 1980: Medical and Surgical License from the University of Agostinho Neto, Luanda, Angola Professional Experience 1999-present Senior Project Officer, Medical Care Development International, Washington, USA Provides technical backstopping support to child survival, water/sanitation, and architectural and engineering projects, including providing primary backstopping support for MCDI’s CS project in Mozambique. This includes but is not limited to designing project proposal in C.S. , preparing terms of reference for consultants, preparing budgets, reviewing reports and coordinating activities between different partners in the projects. 1992-7 Director, Technical Coordinator of Health Sector Project, Ministry of Health/World Bank, Angola. Responsible for overall administration of 20 million project with the following objectives: improve the management and planning capabilities of the MOH at central and selected provincial levels, perform rehabilitation of selected health facilities including the construction of a Nurse School, provide financial and technical support to the AIDS and Family Planning programs. As director he had the responsibility to guarantee the coordination of activities between consultants and the MoH Planning Unit, prepare, supervise the preparation and launch of international bids for consultancy, drug acquisition, construction services, vehicles, school and hospital equipment and furniture. Also he coordinates the teams that analyze technical and financial proposals. Supervises and coordinate the supervision of the nativities in the two provinces where the project was implemented and worked directly with the planning unit on the preparation of activities related with capacity building and organization of MoH at central level. 1987-90 Head, Department of Primary Assistance of the National Cabinet of Public Health, Ministry of Health, Angola Responsibilities include: · Assisted planning of primary health care programs, including EPI, student/worker health, maternal care, and TBA and health promoter training programs 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 25 · Participated in the international evaluation of EPI in Mozambique as an observer invited by WHO · Participated in the preparation of evaluation of vaccination, promoters, and TBA programs · Prepared prioritized Program of the Municipalities and launched the essential medicines program in the National Public Health Directorate · Maintained clinical practice in the medical post of the National Electrical Company and Hospital Américo Boavida, Luanda 1984- 1987 Provincial Health Delegate, Ministry of Health, Angola Responsibilities included: · Planned and managed health activities for approximately one million persons · Carried out central direction of maternal care and adapted it to provincial conditions · Created, coordinated, programmed actions and progression of provincial health team · Directed Provincial Hospital for approximately six months · Provided clinical care and pre-competition evaluations to athletes at Sports medicine center of Huambo Languages (max. 5/5) Language Speaking Reading Writing Portuguese 5/5 5/5 5/5 English 4/5 4/5 3/5 French 4/5 4/5 3/5 Spanish 3/5 3/5 2/5 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 26 Annex 1: MedCost© MedCost© is a health facility and service costing software that is part of the MedSolve© series of health sector financial management tools developed by MCDI. As can be seen in the ensuing figures, it provides an interactive spreadsheet environment for calculating total capital, recurrent, fixed and variable cost as well as unit costs by service produced at either the hospital level or at a health center or clinic level. It incorporates a “step-down” costing algorithm that automatically allocates indirect overhead and support service operating costs to a user-defined set of final cost centers or service. Users enter information on the inputs used (or needed) to produce services at the facility level, the cost of these inputs, their current state of repair, and their useful lives. MedCost© uses this information along with actual or projected utilization data and a number of user-defined parameters (e.g. the rate of inflation, currency exchange rates etc) to automatically generate the costs by health facility. Figure 1: MedCost© Cost Center Definition Sheet Overhead Departments: Support Service Departments: Administration Ambulance & Vehicles Outpatient Laundry Food Service Inpatient: Pediatrics Maintenance Laboratory Inpatient: Maternity Stores Surgery Inpatient: Medical & Surgical Orthopedics & Physiotherapy Pharmacy Radiology & Imaging Final Health Service Departments: Cost Centers Enter cost centers or service departments for the health facility below. Inpatient Outpatient Inpatient Outpatient Inpatient Outpatient Inpatient Outpatient Inpatient Outpatient Inpatient Outpatient Inpatient Outpatient Users define a set of cost centers or service departments that define the production technology employed at each health facility (in this case a district hospital). IMCI-specific final services will be entered here in order to generate total and unit costs by clinical condition (e.g. diarrhea, respiratory problems etc.) 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 27 Figure 2a: Support Service Production Entry Sheet Service Department or Cost Center Support Service Production Allocation by Final Service Department (%) Annual Service Volume or Output Patient Days Average Length of Stay Total Service Allocation % Outpatient % Inpatient: Pediatrics % Inpatient: Maternity % Inpatient: Medical & Surgical Support Services Ambulance & Vehicles 1 NA NA 0.0% Food Service 1 NA NA 0.0% Laboratory 8 41,097 NA NA 100.0% 55% 19% 10% 16% Surgery 4 796 NA NA 100.0% 19% 81% Orthopedics & Physiotherapy 7 260 NA NA 100.0% 100% Pharmacy 10 NA NA 100.0% 68% 8% 17% 7% Radiology & Imaging 8 240 NA NA 100.0% 25% 25% 25% 25% Service production indicator NA NA Examinations Cases Consultations Prescriptions Examinations Users enter service production statistics for all support service departments for which data are available, and the share of these services produced for each final service department. Figure 2b: Final Service Production Entry Sheet Final Health Services Annual Service Volume or Output Patient Days Average Length of Stay Outpatient 13 87,873 NA NA NA NA NA Inpatient: Pediatrics 2 808 6,106 7.6 26 64.3% 31.1 Inpatient: Maternity 2 1,856 5,568 3.0 18 84.7% 103.1 Inpatient: Medical & Surgical 2 634 6,743 10.6 30 61.6% 21.1 Number of Beds Bed Occupancy Turnover Rate Visits Admissions Admissions Admissions Users enter final service production statistics, and MedCost© calculates a number of productivity statistics (e.g. turnover rates etc.). Figure 3: Input Specification Sheets Item Type: Tax Applicability: Code Item Description Stethoscope with plastic tubing dual head 1 $6.40 5 3 0 0 Foetal Stethoscope 1 $1.33 5 1 0 0 Adult Scales - 130kg 1 $498.00 10 3 0 0 Weighing scales for new babies - table model 1 $298.00 10 1 0 0 Tray instrument + lid (305*255*50mm) s/s 1 $35.44 10 3 0 0 1= Equipment 2= Instruments 3= Furniture 4= Supplies 5= Drugs Unit Purchase Price US$ (FOB) Useful life (Years) Good Condition Poor Condition Non Functional Cost Center Quantity Available T1 T2 T3 T4 Outpatient Outpatient Outpatient Outpatient Outpatient Separate input specification sheets are used for overhead, intermediate support and final service departments. Users assign the inputs they enter to their relevant service department(s), enter purchase prices, and the useful lives and current state of repair of all assets. Differential maintenance costs are calculated based on the current state of repair of the input. MedCost© can thus be used not only to generate actual cost based on current 5 / 8 / 02 : - ATTACHMENT F CS STUD Y PROPOSAL.DOC Medical Care Development International Page 28 production technologies, but also can be used to generate cost estimates under alternative (technically preferred) production technologies. Based on these data, MedCost© produces three summary cost tables: (1) a total capital and recurrent cost table, (2) a fixed and variable cost table, and (3) a unit cost table. Benefits of MedCost© A child-survival / SBS-specific version of MedCost© will provide a flexible and user￾friendly tool for program analysts to evaluate the costs and efficiency of service delivery under actual field conditions. Because MedCost© is designed both to allow users to enter the inputs actually used to produce health services, as well as to select from are pre￾specified set of technically desirable inputs (based on WHO’s IMCI and Mother-Baby prescriptive cost models), it will provide the flexibility required not only to cost child survival services under prevailing production technologies, but will also allow program analysts to evaluate the marginal (incremental) cost implications of adding selected inputs prescribed in the IMCI and Mother-Baby cost models. Since MedCost© is facility-based, it will permit analysts to evaluate the range of costs under alternative programmatic, geographic, or structural conditions, and thus provide the information necessary to develop sustainable financing strategies based either on insurance reimbursement, government subsidization or private payments through fees. Because MedCost© captures the total cost of all service production at the facility level, it will also provide analysts (and program evaluators) with important information concerning the trade-offs that implementing child survival services at the facility level requires vis-à￾vis other services – information that is likely to be critical from the standpoint of understanding the success or failure of child surival program strategies in a referral context. By not just focusing on child survival activities, MedCost© is a tool that facility or district managers will find useful not only for costing and developing financing strategies for child survival activities but also for costing and developing financing strategies for all health services produced at the facility level. This added benefit increases the likelihood that the tool will be used by national staff for management purposes. However, since MedCost© will highlight child survival interventions, it could also help to keep the focus of national health staff on child survival strategies. By saving progressive versions of the spreadsheets over time, MedCost© will permit analysts to evaluate the impact of program interventions on the total and unit costs of child survival service delivery – a critical component of program monitoring and evaluation. ATTACHMENT G: HMIS Medical Care Development International 1742 R Street NW, Washington, DC 20009 * USA Telephone: (202) 462-1920; Fax: (202) 265-4078 Internet Electronic Mail: MCDI@MCD.ORG World Wide Web URL: WWW.MCD.ORG ______________________________________________________________________ _______ DRAFT HEALTH MANAGEMENT INFORMATION SYSTEM GUIDELINES FOR UTILIZATION CENTRAL POTOSI CHILD SURVIVAL PROJECT BOLIVIA M C D I DRAFT GUIDELINES FOR THE UTILIZATION OF THE CENTRAL POTOSI CHILD SURVIVAL PROJECT HMIS BASICS defines monitoring and evaluation as “collecting and analyzing information that is accurate and reliable and can be put to practical use Monitoring involves plotting progress in meeting implementation goals or measuring outputs and process, while Evaluation takes a broader perspective, determining if the course is the best one for assessing overall outcome or impact. In the PVO Child Survival Program, monitoring and evaluation provide program managers, local partners and USAID with: a clear understanding of how the PVO program is functioning; evidence of results that have been achieved, and the importance of these achievements to the design and implementation of future programs” RATIONALE FOR A HMIS The Health Management Information System (HMIS) being used by the MCDI CS project in Bolivia is an integral component of the project’s Monitoring and Evaluation process; it is a basic system for collecting and analyzing key pieces of data and information related to child health care programs. It is based on the reporting requirements of the “ Central Potosi Child Survival Project” as defined in the DIP and is compatible with the HIS (SNIS II) of Bolivian MOH, with which MCDI is collaborating. This HMIS system allows the project team and key stakeholders to better organize their data and transform it into more useful and practical information for decision-making towards the attainment of project objectives. The source for health information obtained in the region comes from the MOH district office in Cotagaita. The specific data sets reported to MCDI headquarters are determined by the MCDI field project team and the MCDI CSST in Washington. MCDI has developed its HMIS taking into account the aforementioned SNIS II system and utilizing whenever possible the data collected by the existing system thus avoiding duplication of data collection. The principal aim of these guidelines is to address the information needs related to the monitoring and evaluation process at the project level. However the HMIS includes information related to home office activities. A. The Objectives of HMIS The HMIS will capture information and data required to monitor and evaluate the progress of the project towards the attainment of project objectives, as they relate to community mobilization, behavior change, capacity building and improved access to quality CS health services at the facility community and household levels. The HMIS will also permit the tracking of project expenditures. MCDI expects that the HMIS will be used not only to track project related objectives but may be useful to the MOH in its efforts to improve management practices and the quality of its services. The HMIS should provide timely data to measure project progress towards the objectives defined in the DIP. Within the Child Survival framework, monitoring and evaluation focuses on collecting simple quantitative and qualitative information that can be used to develop and implement effective programs, monitor their performance and ultimately measure their impact. This will be achieved through three types of activities: · Program Monitoring: will measure outputs and processes utilizing supervision, the HMIS, and self-assessments; · Evaluation Research: will develop and test new monitoring and evaluation instruments utilizing cost studies, and community based child survival surveillance; · Performance/Impact Evaluation: will assess overall outcome or public health impact, through population-based surveys such as the KPC, qualitative methods such as FGDs, and health facility assessments. The project HMIS will accommodate the activities described above, making comprehensive use of the various data sets of the system. B. Program Monitoring The HMIS includes the routine information collected monthly in the Prioritized Health Facilities, by the auxiliary nurses on forms prepared by the project. These data sets will be primarily quantitative and will provide information that is used as part of a monitoring process to ascertain trends in outputs and processes, and the epidemiology of child diseases in project communities. (The Excel workbook with proposed worksheets and data sets are included as Annex 1). The data sheets collect output data (number of vaccinations, deliveries, etc.), process data (number of meetings in the communities, days without vaccines, etc.), as well as epidemiological data (number of cases of diarrhea, pneumonia, etc). The routinely collected data also provides information related to the process of institutional strengthening and capacity building, which are important components of the project. The HMIS collects management and financial data on a set of Excel spreadsheets. This information permits the project team to closely monitor project expenditures on a line item basis. The routine data collection referred to above is expected to be complemented by regular formative supervision visits, using guidelines, such as the IMCI supervision form that the project is assisting the district MOH to utilize. These support activities provide information by the direct observation of MCDI field staff and district/municipal health teams related to the following: · The assessment of children under 2 years old as per IMCI protocols; · The assessment of pregnant women as per SBS protocols; · The assessment of the vaccination process conducted in the health facility; · Availability of essential drugs for IMCI; · Availability of drugs and equipment for Maternal And Newborn Care; · Provision of counseling and information to mothers and pregnant women. C. Performance/ Impact Evaluation The project will also utilize complementary tools to the HMIS such as focus group discussions, key informant interviews, and population based methods such as KPC (knowledge, practice and coverage) data provided through KPC surveys, as well as by data produced using the facility based assessment. Project baseline data, upon which quantifiable population based objectives have been formulated, will provide the basis for EOP evaluation collected through the KPC. D. Evaluation Research The Cotagaita Project HMIS will utilize results from specific surveys such as the cost study survey and EPI management survey, to support the sustainability and capacity building objectives. For example recommendations from the studies will support the MOH in improving their financial and program management practices that will impact positively on the quality of services, e.g. avoiding vaccines and essential drugs stock-outs. HMIS Components: 1. DATA COLLECTION The HMIS will input routinely collected data using a group of Excel worksheets with tables, which include indicators and activities to be reported on a monthly, quarterly and annual basis. Each specific form includes data sets that can be easily taken from the existing SNIS II and additional data required by the project. e.g. the number of outreach meetings conducted by clinic personnel. Routinely collected data will be presented in the following summary tables: · control of diarrheal disease, · control of pneumonia, · immunizations, · maternal and newborn care, · institutional strengthening at the community level including the activities and organization of mothers groups, · Institutional strengthening at health facility level that includes training activities, · monitoring and evaluation including supervision activities and participation in TAIs(Talleres de Analises de Informacion/Data Analysis Workshop) . Each table includes specific intervention activities and/or services selected in accordance with project objectives, including information on outputs and processes; epidemiological information is provided in some, not all of the tables. Summary tables provide basic information required to monitor progress towards the attainment of objectives as prescribed in the DIP (Table D). The summary table templates are presented in Annex 1. The same format is used for monthly, quarterly, and annual reports. (A simplified data summary table will be attached to the quarterly performance reports (see Annex 2) submitted to the home office CSST. The tables present data for all health facilities in both Vitichi and Cotagaita municipalities. For project management and reporting purposes, only the data from the prioritized facilities and related communities will be reported. Coverage indicators and community based activities from priority health facilities will provide the project team and the MOH with the information needed to undertake performance evaluations. Even though the priority of the information to be collected and summarized for project management and reporting focuses on Priority health facilities, the project staff is encouraged to collect the same data for the remaining non-priority Facilities so the project staff can use it to provide feed-back during the TAIs (Data Analyses Workshops). The first source of data for the non-prioritized facilities should be the existing SNIS II forms received at district level. Due to the fact that the SNIS only collects aggregate data for children under five years old, the project staff will need to collect, with the help of the auxiliary nurses in the health facilities2 , the data for the under two population; indicators related to disease control, and any other indicators that the SNIS II does not provide such as the number of training sessions and community activities such as meetings of mothers groups, etc. Flexibility is built in the data collection sheets so that they can accommodate future additions, e.g. a column can be added to collect data on SBS revenues collected by facility. The tables are expected to improve during the life of the project but any 2 For this objective the project coordinator can if necessary develop special arrangements to compensate the extra burden on data collection for the auxiliary nurses. proposed changes in reporting requirements should be discussed, by the field team and the home office in order to establish that these changes are justified. The Financial Management Information System is an integral part of the HMIS. The Project Accountant and Project Coordinator collect information at the field level, on a monthly basis. This information consists of project expenditures supporting receipts and documentation, and the monthly request for funds. The Excel spreadsheet for budget management, links project line items to appropriate codes compatible with MCD’s accounting system. Field information is reviewed by the CSST and then forwarded for input into MCD’s overall financial management reporting system in Maine. NOTE: The Community Based Health information System: In collaboration with the MOH, the project anticipates the design and implementation for a community based HIS that will be introduced in the context of the project’s HH/C-IMCI activities.. This will help to more accurately reflect the data from priority communities, and at the same time will contribute to the empowerment of these same communities. The specific models and forms are to be adapted from existing models being used in the region. They will be designed as graphically as possible in order to facilitate the responsibility of the community volunteers to fill them in. Before being used in the field, the forms must be approved by the MOH and by the CSST. The Flow of Information and Responsibilities for Data Collection The project data collection forms are provided to every priority health post enabling them to report on a monthly basis. During the first week of each month, the project staff will collect the forms, filled-in by the auxiliary nurses for the previous month. The MCDI nurse will guarantee the quality of the data, correcting errors while in the facilities. (This will strengthen capacity of the health staff to manage data.) The data forms are taken to the project field office and entered in the appropriate HMIS spreadsheets. Copies of the final tabulated monthly summary data sheets will be given to the Municipal and District health authorities. Copies will also be provided to each Priority Health Facility. Consolidating these monthly data sheets into the quarterly summary data sheets to be sent to headquarters along with the quarterly report is the responsibility of the Project Technical Director who also has the responsibility of the final version and data quality control. In accomplishing this, the Project Technical Director will coordinate the work with all project field staff and the Project Coordinator. The monthly summary data sheets will mainly be used at the field level for management and feedback proposes. Unless advised otherwise the CSST does not need to receive monthly data summaries In the quarterly report output and process indicators will be presented as appropriate. The Project Technical Director has the responsibility to prepare the final summary data sheets and report. These will be approved by the Project Coordinator and sent to MCDI where the Child Survival Coordinator and the CSST are responsible for their review and revision. A copy of the quarterly data sheets and report (in Spanish) should be sent to our MOH partners. This same process should be used for the annual summary data sheets and reports. In the specific case of the Financial Management data, the flow of information is mainly to the Home Office were it is reviewed prior to being sent to MCD headquarters in Maine. Cumulative expenditure reports are compiled by the Financial Management Department and distributed to all MCDI Child Survival Project Managers. Requests for funds are analyzed and approved at MCDI in Washington and forwarded to the Financial Management Office in Maine. 2. DATA ANALYSIS Program Monitoring Based on the summary data sheets completed every month by MCDI health personnel, in collaboration with the auxiliary nurses of each Priority Health Facility visited, a summary analyses of the main features of the interventions by facility will be prepared by the Project Technical Director in collaboration with project staff and will be fed into the monthly report. The same process will be used to prepare the quarterly and annual reports. The data analyses will occur at ad-hoc project staff meetings where the data findings will be discussed. These meetings will include the Project Coordinator, Project Technical Director and Project Nurses. In this meeting the team will discuss the implications of the data and make the appropriate managerial and technical recommendations. These analyses will be included in the monthly, quarterly, and annual reports and will also be the basic instrument that project staff will use to provide feed-back to municipal and district health authorities, to auxiliary nurses in the Priority health facilities and to the community groups and volunteers. At home office in Washington, the data and reports will be analyzed by the CS Coordinator with support from the CSST, for consistency and accuracy. The analysis will be used to respond to the field team and recommendations will be made related to the project implementation. The Project Technical Director has the ultimate responsibility for preparing and editing the summarized data and the quarterly reports, with the Project Coordinator signing off on the report. The appropriate indicator tables must be filled in. As previously indicated quarterly and annual reports must include the data summary table (see Annex 2) Evaluation Research / Performance Evaluation In the case of data and information obtained by a process of evaluation research, for example, Cost study of the SBS, the study on the Management of the EPI and Essential Drugs, the draft results and recommendations should be revised and approved by the CS Coordinator supported by the CSST in Washington. The same procedure applies to the KPC and other performance evaluation activities. 3. DATA RESPONSE / FEEDBACK Program Monitoring Providing feedback to project partners is a crucial element of the monitoring process. The feedback will be ensured through all levels of the system e.g., District and Municipal MOH, health facility staff, community groups and volunteers. MCDI believes that the process of providing feedback represents a unique opportunity to promote partner participation and capacity building. A major vehicle for securing feedback will be the TAIs implemented at municipal, district and SEDES levels. Based on the analyses of the data, a summary bulleted list of the main achievements and constraints should be prepared for discussion during the TAIs with the auxiliary nurses. The project Technical Director assisted by other members of the field team will have the responsibility of presenting this feedback. These summary lists will be used to guide the formative supervision process, during which the auxiliary nurses will be briefed on the data and participate in formulating possible solutions to constraints and problems encountered (and be praised for positive results achieved). At a more strategic level and in preparation for the Regional TAIS the Project Coordinator and the Project Technical Director will provide feedback to the District Health Director and municipal directors at least on a quarterly basis, discussing the trends identified in the data and analyzing the steps to overcome problems detected in areas such as: access to quality services, institutional strengthening, community mobilization and HH/C-IMCI. Evaluation Research / Performance Evaluation The reports and recommendations from Evaluation Research/ Performance Evaluation activities, such as the SBS Cost study and the EPI and Drug Management study, should be presented and shared/discussed in workshops with the District and Municipal health teams and local authorities. The workshops should include community representatives, with whom strategies for implementing the recommendations will be prepared, and appropriate activities to achieve these strategies should be included in the district and municipal work plans. Local NGOs in the area e.g., CARE and Calchanachispa must also be invited especially those participating in implementing any recommended measures. These workshops will make a substantial and practical contribution to strengthening local partners. The Departmental Health authorities (SEDES) will receive copies of the studies surveys and reports, and be invited to participate in the workshops. How to use the data collected to inform community members The information feedback process for the communities is particularly important for the project team and health authorities. This process will keep communities better informed and as a result they will be active partners in health interventions at community and household levels. In the project area there are routinely convened meetings mobilized by community leaders. MCDI is encouraging participation in these meetings of auxiliary nurses, MCDI staff and community volunteers such as vigilant mothers, to discuss and share the findings of routinely collected information. Calling residents of rural indigenous communities to public meetings usually result in large turnouts. Given the facility which all MCDI field staff have with speaking the Quechua language fluently, it is safe to say that there will be no problem whatever with providing and obtaining significant feedback from the people of those communities relative to the data collected. The project staff, health staff and community volunteers should utilize this feedback information to adjust the interventions as needed. ATTACHMENT H: DISKETTE WITH ELECTRONIC COPY OF THE REPORT