FINAL EVALUATION-QUALITATIVE COMPONENT IMPROVING THE HEALTH OF THE MOST VULNERABLE POPULATION OF GUATEMALA- MIGRANT MOTHERS AND THEIR CHILDREN IN THE BOCA COSTA OF GUATEMALA Prepared by Jack Reynolds, Ph.D. Project HOPE Consultant Prepared for: Project HOPE The People to People Foundation Millwood Virginia Cooperative Agreement No. FA0-A-00-97-00030-00 October 2001 i TABLE OF CONTENTS SUMMARY............................................................................................................................. 1 THE QUALITATIVE SURVEY.................................................................................................... 1 OVERALL ASSESSMENT.......................................................................................................... 1 CROSS-CUTTING AREAS ......................................................................................................... 2 PROGRAM MANAGEMENT ...................................................................................................... 2 OTHER ISSUES IDENTIFIED BY THE TEAM ................................................................................ 4 A. INTRODUCTION........................................................................................................... 7 1. OBJECTIVES ................................................................................................................... 7 2. DESIGN .......................................................................................................................... 7 3. DATA COLLECTION AND ANALYSIS................................................................................. 9 `B. OVERALL ASSESSMENT OF THE PROJECT.................................................... 10 1. PLANTATION ASSISTANCE ............................................................................................ 10 2. ACHIEVEMENTS............................................................................................................ 11 3. PROBLEMS ................................................................................................................... 13 4. CONTINUATION............................................................................................................ 14 C. CROSS-CUTTING ISSUES ......................................................................................... 15 1. COMMUNITY MOBILIZATION ........................................................................................ 15 2. COMMUNICATION AND BEHAVIOR CHANGE.................................................................. 16 3. CAPACITY DEVELOPMENT............................................................................................ 19 4. SUSTAINABILITY .......................................................................................................... 22 D. PROGRAM MANAGEMENT..................................................................................... 25 1. PLANNING.................................................................................................................... 25 2. STAFF TRAINING .......................................................................................................... 26 3. COORDINATION AND FOLLOW-UP SUPERVISION ............................................................ 27 4. HUMAN RESOURCES .................................................................................................... 28 5. FINANCIAL MANAGEMENT ........................................................................................... 29 6. LOGISTICS.................................................................................................................... 30 7. INFORMATION MANAGEMENT ...................................................................................... 31 8. TECHNICAL AND ADMINISTRATIVE SUPPORT................................................................. 32 E. OTHER ISSUES IDENTIFIED BY THE TEAM........................................................ 33 1. CONSTRAINTS AND FACILITATING FACTORS.................................................................. 33 2. ACCESS TO HEALTH SERVICES...................................................................................... 34 3. PROJECT EFFECTS ON HEALTH BEHAVIOR..................................................................... 35 4. TARGET GROUPS.......................................................................................................... 37 5. INTERVENTIONS ........................................................................................................... 39 6. DESIGN AND IMPLEMENTATION ISSUES ......................................................................... 39 F. ATTACHMENTS ........................................................................................................... 1 1. QUALITATIVE EVALUATION PARTICIPANTS ..................................................................... 1 2. FIELD WORK SCHEDULE................................................................................................. 2 3. DOCUMENTS .................................................................................................................. 4 ii TABLES TABLE 1: DATA COLLECTION AND ANALYSIS MATRIX................................................................ 8 TABLE 2: RESPONDENTS AND QUESTIONS ANALYZED............................................................... 10 TABLE 3: ASSISTANCE PROVIDED BY THE PLANTATION............................................................. 10 TABLE 4: RESPONDENTS AND QUESTIONS ANALYZED............................................................... 16 TABLE 5: RESPONDENTS AND QUESTIONS ANALYZED............................................................... 19 TABLE 6: SUPERVISION GUIDE FOR TRAINING ACTIVITIES ........................................................ 21 TABLE 7: SUPPORT PROVIDED BY THE PLANTATION TO THE HEALTH UNIT................................. 22 TABLE 8: OTHER SUPPORT THAT THE PLANTATIONS ARE WILLING TO PROVIDE TO THE HEALTH UNIT................................................................................................................................ 23 TABLE 9: PROJECT ACTIVITIES THAT SHOULD CONTINUE.......................................................... 24 TABLE 10: VISITS TO PLANTATION HEALTH UNITS BY MIGRANT CHILDREN < 5 YEARS OF AGE, OCTOBER - DECEMBER 2000............................................................................................ 34 TABLE 11: PROJECT PERFORMANCE INDICATORS ..................................................................... 36 TABLE 12: PROJECT BENCHMARKS, 1997-2001........................................................................ 41 ABBREVIATIONS AID Agency for International Development AIDS Acquired immune deficiency syndrome ANACAFE National Association of Coffee Growers APROFAM Guatemalan Welfare Association ARI Acute respiratory infections BBC Behavioral change communication CS Child survival HC Health center HIV Human immuno-deficiency virus IEC Information, education and communication IGSS Guatemalan Social Security Institute JHPIEGO Johns Hopkins Program in Education in Gynecology and Obstetrics KPC Knowledge, practice and coverage (survey) MEC Standard case management MECA Comprehensive standard case management MSP Minestario de Salud Pública (Ministry of Public Health) NGO Non-governmental organization ORS Oral rehydration solution PAHO PanAmerican Health Organization SIAS Sistema Integral de Atención en Salud (Integrated Health Service System) SIS Sistema de Información en Salud (Health Information System) TT Tetanus toxoid 1 SUMMARY The Qualitative Survey The final evaluation has three components: the Results Survey, a Qualitative Survey and a Process/Activity Assessment. This report summarizes the findings, conclusions and recommendations from the Qualitative Survey. This survey consists of a series of interviews with health councils, plantation owners/administrators, project counterparts, trainers, health care providers, promoters and beneficiaries (migrant and resident mothers). Additional issues identified by the Evaluation Team members and the HOPE Consultant are also included. It is important to remember that this is a qualitative assessment. The respondents are not necessarily representative of the groups interviewed and the results should not be generalized to the total population of those groups. The purpose of the survey is to complement data yet to come from the other evaluation components and to provide insights into the perceptions of the various groups interviewed. The additional issues identified by the Team reflect the views of various team members and the consultant, based on their observations, experience, discussions within the group and additional data provided by HOPE. The survey is divided into three parts: an Overall Assessment, Cross-cutting Areas and Program Management. These are supplemented by Other Issues Identified by the Team. Overall Assessment Conclusions: The plantations are providing valuable assistance, but it is neither comprehensive nor consistent. The most common types of assistance are the promoters, the facilities and promoter attendance at training sessions. Almost all of the respondents saw significant improvements due to the project. Most important are: greater access to health services and improved health status among migrants and residents; and lower costs and improved worker productivity for the plantations. The most significant and important problem is the lack and shortage of medicines. There are some problems with promoters, especially turnover, unavailability when needed and inability to communicate with migrants. Inconvenient hours of operation, lack of coordination, inadequate supervision and poor continuity of care for migrants are also problems. Practically all of the respondents believe that the program should continue, but with modifications to resolve the problems identified. Recommendations: HOPE should develop a list of resources needed/required to implement the program in a plantation, together with the sources of these resources (plantation, MSP, etc.). Investment in capacity development should continue, especially at the plantation and health center levels. The linkage between the two needs to be strengthened and institutionalized so that it becomes a single service delivery system. A strategy is needed to ensure that adequate medicines will always be available in all health units. HOPE needs to identify and test alternative supply systems and not rely too heavily on the MSP to provide medicines. HOPE should also design and test a system to provide continuous and comprehensive health services to migrants. The program should continue, but with new strategies to resolve the major problems mentioned above. One of the most important operational objectives should be to ensure continuity and reliability in all of the key project components. 2 Cross-cutting Areas Conclusions. Not much can be drawn from this survey regarding community mobilization, except that there may not be very much of it. The promoters do not seem to understand what it means and the principal “participatory” activities mentioned (training of promoters and attendance at vaccination campaigns) are not participatory at all. Not much can said about the effectiveness of the behavioral change strategies, either. The respondents generally believe that changes have occurred in health-seeking behavior, utilization of services and morbidity and mortality. However, this seems unlikely given the small number of beneficiaries who say they attended a health education session. Better information is needed about this activity before any conclusions can be drawn about its effect on behavior. According to both the promoters and providers there have been significant improvements in the health units provided and the use of those services by the target population. There is not enough data in this study to assess field worker performance adequately, but providers note better screening and referral from the plantations to the health centers, which has improved performance at both levels. Provider and promoter training appear to be excellent. As a result, service quality has improved and that has led to improved health for migrants and residents. Some of the weaknesses include the shortage of time and the lack of follow-on field supervision. Sustainability is uncertain. There is a great deal of interest in continuing the project activities, especially from the plantations, many of whom appear willing to continue – or expand – their support. However, certain project components appear vulnerable: promoters, transportation, medicines and coordination are project dependent. It is not at all certain that they will continue once HOPE support ends. Recommendations. If community participation is part of the project strategy then HOPE should undertake a study of what is actually going on in order to determine what should be done in the future. HOPE should conduct an assessment of the BCC/IEC activities to determine: 1) whether the messages are being heard and understood by the target groups; 2) whether the target groups have internalized those messages; and 3) whether that has had any effect on their health￾seeking behavior. This assessment should include representative migrant populations as well as residents. It would help to have more detailed information about the improvements that have been made in the health units throughout the 150 plantations and the effect that has had on utilization and health. If HOPE plans to promote a “model” system for replication, then this information would be critical and it should be compiled and analyzed. If field worker performance is a priority for the project then HOPE should undertake a study to assess it at the community and health center levels. This should include clear and standardized performance indicators as well as unit costs and productivity data. Training should continue and the following should become a standard part of the project: refresher training; continuing education; follow-up field supervision; additional training in preventive health care; and basic training for new providers, promoters and midwives. HOPE should undertake a sustainability analysis to determine what would be required to ensure the continuation of key project activities after the project ends. Specific attention should be given in all of these activities to migrants and their needs. Program Management Conclusions. The counterparts and some providers have been involved in planning and problem solving, but not promoters or communities. Almost everyone responded that staff training has had an effect, not only within project work, but outside as well. In general, this training seems to 3 be effective and valued. Most respondents believe that coordination and follow-up supervision are good and are already institutionalized. The fact that most of the promoters had been visited in the past 1-3 months is a good indicator of this. In general, or in theory, there are adequate human resources to maintain the program. However, there are gaps, especially at the health center and promoter levels, but also at the counterpart and trainer levels. Turnover will require recruiting and training replacements. Migrants, in particular, increase the seasonal need for additional human resources, especially those who can speak their languages. Financial management. The counterparts are optimistic about being able to continue to cover the costs of the program in the future. However, one respondent mentioned a key issue with respect to government financing, that it is unreliable. The promoters are also heavily reliant on MSP and HOPE to cover the cost of medicines. The plantations and families are not significant sources of funding at the moment. Financial sustainability seems anything but assured. Logistics: The general view is that HOPE and its associates have done a good job in making essential medicines, educational materials and transportation available. Nevertheless, it is clear that at least some sites are not getting the assistance they need, do not enough and/or it is not consistent and timely. Information management: data is collected, but it is integrated into the health center activity data. It is not compiled or displayed separately. It does not seem that data is being used to assess achievement of project health objectives. The technical assistance provided by HOPE has been important and valuable to the providers and counterparts, most of who said that all of their needs were met. The additional needs identified by a few were in medicines, transportation, nutrition and training. c. Recommendations Recommendations. Continue involving counterparts and providers in planning and problem solving; develop strategies for involving promoters and beneficiaries (i.e., residents and migrants) as well. Staff training should continue, in response to individual needs. Inter-institutional coordination should be strengthened to improve the training activities. Other counterparts, not just HOPE, should contribute to personnel training. Coordination and supervision are important and need to continue and be strengthened so that they operate at all levels. Human resources: if the program continues there will be a need to identify, recruit and train personnel at all levels: counterparts, trainers, providers and promoters. Expansion of the program and personnel turnover will require that. Continuing education will also be required for all staff. To serve migrants better, each plantation should have at least one promoter or assistant promoter who can speak the migrant’s language. With respect to finances, a prudent recommendation is to look for alternative sources of funding, especially for medicines. It would also be helpful to calculate the unit costs of each component so that alternative budget scenarios can be generated. Local government units may be a source of funding under the decentralization initiative. Logistics: The achievements made to date need to be maintained and the gaps in logistical support filled. The associates need to become more involved in logistics planning and strategies need to be developed to ensure true sustainability, such as sales outlets and rural drug stores. It would seem appropriate to take a closer look at how project data is compiled and used by the MSP. The data needs to be disaggregated so that the effects of the project can de measured. Otherwise it will not be possible to determine if the project is having any impact on disease prevention and health￾seeking behavior. The project needs to maintain its technical and administrative assistance to current providers and counterparts. This will also be a need in the project extension. HOPE will need to identify and respond to the needs of the new counterparts and providers so that they can 4 carry out their responsibilities. Other Issues Identified by the Team Access to Health Services Conclusion: From a qualitative perspective, the major achievement of the project seems to be that it has increased access to basic health services for thousands of plantation workers, both resident and migrant. This has resulted in increased utilization of priority maternal and infant care services. And that has resulted in improved health for both mothers and their children. Quantitatively, it is more difficult to determine how many plantation workers have been served. Recommendation: HOPE should update its utilization data to show: 1) the number of visits to plantation health units each month by mothers, children under five and others, for migrants and for residents; 2) the number of individuals served by the plantation health units in 2001, broken down the same way; 3) the number of plantation residents and migrants served by Health Center staff during their visits to the plantations; and 4) the number of referrals made to the Health Centers by plantation promoters and midwives. Project Effects on Health Behavior Conclusion: Preliminary data are now available on some of the key health indicators, such as fully-immunized children, but only for residents. In general, the changes seem modest and fall far short of project objectives. No data are available for migrants, but they are likely to be even lower. Overall performance is well below expectations and should raise questions about the viability of the project design as well as its implementation. Recommendation: HOPE should step back and reexamine both the project design and its implementation before expanding to new sites or expanding the service package. The “model” does not appear to be ready for replication as yet and should not be expanded prematurely. Target Groups Conclusion: The project has had a very difficult time reaching its primary target group, migrant women and their children. It has been difficult even getting data about the health status and needs of this group. The current project strategy is not conducive to serving migrants and, given the recent KPC results, it is unlikely that the project will be able to demonstrate any significant improvement in the health of this group. That raises a fundamental question: should the project keep this as its priority objective? Recommendation: HOPE should conduct a careful analysis of feasible options and decide whether to develop a migrant-oriented service or scale back its objectives. Interventions Conclusion: The original project interventions were all health related: immunizations, nutrition diarrhea disease control, pneumonia and maternal care. The project performance indicators still conform to this set of interventions. In 1999 these were changed to “facilitation” interventions, 5 such as staff training, coordination, training of master trainers and facility development and the “model” that was to be expanded and replicated became a facilitation model. Since AID guidelines require the assessment of the results of each “intervention,” this shift in the definition of the “interventions” is not only confusing, it also gives the impression that HOPE wants the project to be assessed on the basis of its success in facilitation instead of its success in improving health and health service utilization. Recommendation: The effectiveness of HOPE’s model should be judged in terms of its effectiveness in achieving the health objectives of the project. If the facilitation activities do not improve health and health service utilization, the model cannot claim to be a success and should not be replicated. Design and Implementation Issues The Evaluation Team raised five other issues. Health Units. The first is the condition of the existing health units at the plantations. The recommendations are to: conduct an inventory of the facilities, including equipment and furniture; develop a list of standard requirements; and examine their history (current status, turnover rate, etc.). Promoters. The second is the status of the promoters. During the field visits the Evaluation Team learned that some plantation do not have promoters and others have promoters who have not been trained. The recommendations are to: 1) update the database on promoters to determine how many are active, the average turnover rate and length of service, how many have been trained, how many are paid and how much they are paid; 2) examine the characteristics of promoters who resign to identify possible predictive criteria that can be used in the future to select promoters. Visits by health center staff to the plantations: The link between health services at the plantation and at the municipal Health Center is a very important element in the strategy because it ensures greater access to trained health providers. Theoretically, the link works both ways: Health Center doctors and nurses go to the plantation to provide services to residents and migrants; and promoters and traditional midwives refer clients to the Health Centers. When this works it is very effective. It has become even more valuable since the project has provided IMCI and maternal care training to Health Center providers. However, there are a number of problems that have been noted, for example, HC staff who have a “9-5” attitude and leave before the workers finish their work and HC staff who limit themselves to their assigned tasks and do not take advantage of the visit to provide other education or services. HOPE needs to develop a strategy for ensuring that the link between the health centers and the health units is ensured and strengthened. A standard set of activities and procedures should be developed to maximize the productivity of each team visit. HOPE should also work with the MSP to set up a flextime system for health staff that work in coffee plantations where clients are more likely to come for services in the afternoon and evening. Transportation. The lack of reliable transportation is a critical concern. It affects interaction between the plantation health units and the municipal health centers. It affects the availability of essential drugs. One of the principal factors limiting training and supervision of volunteers by the 6 health center providers is the lack of funds within the MSP for this kind of activity, especially transportation and related costs. The Team recommends that the plantations should provide transportation for the health center staff to get to the health units to provide services as well as to supervise and train the volunteer promoters and midwives. Transportation funds should also be sought from municipal governments and others. Behavioral Change Communication. There are some doubts that the BCC/IEC campaign is having any effect on the target groups. This includes the radio spots, the project vehicle loudspeakers, the messages themselves and the educational sessions. HOPE should take a close look at the BCC/IEC strategy to see if it is having any effect. BCC/IEC strategies should be adapted to fit the expectations and information needs of the target populations. Adolescents and youth should be priority targets, especially among the migrant populations. Family planning should be included in the BCC/IEC strategy. 7 A. INTRODUCTION This chapter summarizes the methodology and schedule for the qualitative component of the final evaluation. 1. Objectives The AID guidelines for a final evaluation call for the assessment of project impact (changes in health status), project results (changes in health behavior) and the effectiveness of the technical approaches used (key interventions) to achieve those results. The original goal of the project was to reduce infant, child and maternal morbidity and mortality in the Boca Costa region of Sololá, Quetzaltenango and San Marcos, with a focus on migrant populations. The purpose was “to bring about changes in disease prevention and care-seeking behaviors in the target population and to improve service delivery of the MSP and NGO health units. Project HOPE’s role will be that of trainer and facilitator, not as direct provider of services or community education.”1 This project has five interventions: nutrition of children < 1 year; immunization of children < 1 year; pneumonia among children < 5 years; diarrhea of children < 5 years; and maternal care of women aged 15-44.2 The specific objectives for each intervention are summarized in the original proposal (p. 21) and in the Detailed Implementation Plan (DIP).3 2. Design The final evaluation has been divided into three parts: Results Survey: A Knowledge, Practices and Coverage survey (KPC) designed to provide data on the impact of the project on health and the effects achieved by each intervention on disease prevention and health-seeking behavior. The results will be compared with a similar baseline survey to assess changes over time. Qualitative Survey: A series of convenience surveys of different groups (health councils, plantation owners/administrators, counterparts, trainers, providers, promoters and beneficiaries). The results of these surveys are described in this report. Process/Activity Assessment: An assessment of the processes employed in each intervention and support activity (training, logistics, etc.). The purpose of this assessment is to assess the inputs, processes and outputs of each of the key project activities. This assessment may be combined with a proposed capacity assessment study scheduled to be undertaken later this year. 1 Improving the Health of Guatemala’s Most Vulnerable Population – Migrant Women and Their Children I the Boca Costa of Guatemala. A Proposal to the U.S. Agency for International Development, December 13, 1996, p. 16. 2 Ibid. p. 20. 3 Improving the Health of Guatemala’s Most Vulnerable Population – Migrant Women and Their Children I the Boca Costa of Guatemala. Detailed Implementation Plan. Grant No. FAO-A-00-97-00030-00. April 1998, pp. 8- 11. Table 1: Data Collection and Analysis Matrix Health Council Counterparts Owners & Managers Trainers Radio Providers Promoters Beneficiaries Health Unit Guide Training Guide Visits Meetings Observations No. Respondents: 4 8 24 6 2 11 19 20 11 11 x x x Synthesis A Sin A A A Plantation Assistance 2 Achievements A2 3,4,5 A2 A2 A2 Problems A3 6 A3 A3 A3 Continuation A4 A4 A4 Cross-cutting Areas 3 Community mobilization 3a 3a Acer Behavioral change communication 3b x 3b 3b Com Capacity development 3c 3c 3c 3c Health Facilities 3ciii 3ciii Fort x Health worker performance 3civ Training 3ci 3cvi x Sustainability 3d Sus 3d 3cd 3d Program Management x C C C C Planning c1 c1 c1 Training of staff and associates c2, 3cii c2 Coordination and follow-up super. c3 c3 c3 c3 Human resources c4 c4 c4 Financial management c5 c5 Logistics c6 c6 c6 c6 Information management c7 c7 c7 Technical & administrative assistance c8 c8 Note: Letters in boxes refer to relevant sections of each questionnaire. 8 Matrix de Analisis Benificiorios Promotores Proveedores Contrapartes Capacitadores Consejo Duenos & Admin. Radio Guia Unidades Guia Supervis. Visitas Reuniones Observacione Sintesis, impresion general A A A A Sin Apoyo 2 Logros A2 A2 A2 A2 3,4,5 Problemas A3 A3 A3 A3 6 Continuacion A4 A4 A4 Resultados del abordaje tecnica Inmunizaciones B B3 Manejo de enferm comun B B3 Salud materna B B3 Lactancia materna y alimentos B B3 Resultados areas que cruzan 3 Comunicaciones cambio comp. Com 3b 3b 3b x Construccion capacidada 3c 3c 3c 3c Establecimientos de salud Fort 3ciii 3ciii x Disempano del trab. Social 3civ Capacitacion 3cvi 3ci x Acercamiento, partic, comun Acer 3a 3a Sustanibilidad 3d 3cd 3d 3d Sus Manejo del programas C C C C x Planeacion c1 c1 c1 Coordinacion y seguimiento c3 c3 c3 c3 Logistica c6 c6 c6 c6 Manejo de informacion c7 c7 c7 Apoyo tecnica y adminis. c8 c8 Capacitacion personal y socios c2 c2, 3cii Recursos humanos c4 c4 c4 Manejo de financiero c5 c5 9 3. Data Collection and Analysis The Qualitative Survey instruments were based on similar questionnaires used in the mid-term evaluation that follow the major categories listed in the AID guidelines. HOPE staff in Virginia and Guatemala drafted the questionnaires. About 10 members of the 21-person Evaluation Team reviewed them over a two-day period (October 1-2). The data collection schedule was also reviewed at the same time. Two data collection teams were formed and began fieldwork on October 3. The teams met with and interviewed Health Area officials, Health Council members, trainers and counterparts. The bulk of the interviews were with plantation owners/administrators, health providers from Health Centers and promoters. Table 1 for a summary of the number of respondents interviewed and the topics covered. Also see F 2. Field Work Schedule for a summary of the sites visited. Data collection continued through October 9. Compilation of the data was done simultaneously (October 8-9). The eight questionnaires and two observation guides included approximately 600 questions. Due to time constraints only the responses to high priority questions (about 1/3 of the total) were compiled. The compilations were grouped according to major themes: Overall Assessment, Cross-cutting Areas and Program Management. A 13-person analysis group was divided into three subgroups that met October 10-11 to analyze the data and prepare summaries of findings, conclusions and recommendations. The group met in plenary on October 11 to review these results and to make revisions. The final versions of the summaries were then given to the HOPE consultant who then drafted this report the following week. It is important to remember that this is a qualitative evaluation based on a convenience sample. It is not representative of the groups interviewed and the results should not be generalized to the total population (providers, promoters, plantation managers, etc.). Its purpose is to complement the data to come from the other evaluation components and, hopefully, provide insights into the perceptions of the various groups interviewed. The report consists of two principal parts. Chapters B-D summarize the responses to the priority interview questions and the conclusions and recommendations that the group drew from those responses. Chapter E summarizes other issues that were identified by the Evaluation Team during the field work and analysis stages. The conclusions and recommendations in this chapter reflect the views of various members of the Evaluation Team, including the HOPE consultant. Chapters B-D are organized the same way. First, a topic is identified (e.g., Achievements, Community Mobilization) and the number and type of respondents summarized (as in Table 2) because not all respondents were asked each question. Next, the first question is given for the first class of respondent (e.g., Owners and Administrators or Promoters). The questions are written in italics so they can be recognized easily, for example (What is your support to this program?). Then the responses are summarized (e.g., eight of the 11 providers believe that the program should be continued). This procedure is followed for each question asked of that class of respondent. Then the questions for the next class of respondent are examined. After all of the questions have been examined, the conclusions for that section are summarized followed by the recommendations. 10 `B. OVERALL ASSESSMENT OF THE PROJECT This chapter summarizes the views that various respondents have about project achievements, problems and whether the project should be continued and/or revised. The chapter begins with a summary of the type of assistance provided to the project by the plantations. The respondents are not necessarily representative of their groups, for reasons discussed in the methodology section. Nevertheless, they do provide valuable insights. Table 2 shows the type and number of respondents as well as the number of questions selected for analysis from the questionnaires. 1. Plantation Assistance a. Questions and Findings The 24 owners and administrators were the only ones asked to respond to this question. What is your support to this program? The assistance varied tremendously, from nothing to a full range of support: the salary of the promoter, equipment, medicines and transportation to enable the promoter to attend training sessions. The most mentioned type of support was the promoters themselves. That is, the plantations either allowed one of their workers to act as a promoter when there was a need or paid their salaries. It is not clear how many actually paid salaries and how many simply allowed their workers to serve as promoters (in addition to their regular plantation work, in lieu of that work, or when there was an emergency or a health campaign at the plantation). The next most common type of support mentioned was the facility itself. In most cases this was probably space, converted from a storeroom or other existing room to a “health unit.” In some cases the plantation may have constructed a stand-alone facility. Sometimes this included renovation, painting, provision of furniture, and so forth. Next comes training of the promoter, which was provided by the project, usually at health centers in the town. The plantations either gave permission for the promoter to attend or provided/paid for transportation to and from the training site. Few of the plantations (five) provided medicines. One claimed to have set up a complete drug store; another provided four medicines monthly from a $100 annual budget for that purpose; some provided medicines occasionally. Among the other types of services provided were: transportation of clients to the health center or Table 2: Respondents and Questions Analyzed Respondent No . Questions Owners & Administrators 24 6 Counterparts 7 3 Trainers 5 3 Providers 11 3 Promoters 19 2 Table 3: Assistance Provided by the Plantation Type Assistance No . Promoter/Salary 15 Facility/Infrastructure 13 Access to training for promoter 8 Medicines 5 Other 11 11 hospital, educational materials, occasional financial help, equipment, sanitation facilities (latrines and garbage disposal areas) and “coordination and implementation.” b. Conclusions The plantations are providing valuable assistance, but it is neither comprehensive nor consistent among plantations. The most common types of assistance are the promoters, the facilities and promoter attendance at training sessions. However, even these are not consistent. Some “allow” workers to act as promoters when necessary, other pay part or all of their salaries. There is no standard list of assistance that is needed (or required of the plantations). c. Recommendations HOPE needs to become more active in facilitating meetings of the owners and/or administrators with officials from IGSS and the MSP, including those from the municipalities, in order to coordinate the assistance needed for the project health budget. HOPE should develop a list of resources needed/required to implement the program in a plantation, together with the sources of these resources (plantation, MSP, etc.). A quick study should be undertaken to identify the current contributions made by the plantations, especially transportation, health unit refurbishment and the percentage of promoters paid to be promoters. 2. Achievements a. Questions and Findings All four respondent types were asked similar questions about the project’s achievements or strengths to date. The owners/administrators were asked about the benefits to the residents, the migrants and the plantation itself. Owners and Administrators: From your point of view, what are the achievements to date for the residents? What have been the achievements for the migrants? The owners and administrators mentioned many benefits, which can be broken down into two categories: improved access and improved health. Sixteen of the 24 identified various ways that access had been improved for residents; and even more (20) described ways that it had improved for migrants. Among these were: having access to a convenient health unit, a promoter and appropriate medicines; greater access to health education; greater confidence in the health providers; and less need to travel outside the plantation to get services. Eight respondents saw improvements in the health of residents; three saw improvements in migrant health. Among these were declines in such health problems as diarrhea, vomiting, cholera, respiratory infections and intestinal infections. What have been the achievements for the plantation? There are many benefits, most of which can be summarized in five categories: greater convenience (3); improved health of the workers (4); improved health services (6); lower costs (6) and a better, healthier work force (7). In fact, all of the factors mentioned can be seen as contributing to two factors: reduced costs and improved productivity. Counterparts: From your point of view, what have been the strengths of this project? Five of 12 the seven counterpart respondents mentioned training. Three mentioned coordination with the health services. Other achievements mentioned were: strengthening of the health units in the communities; improved access to health care through the promoters and the health centers; and increased coverage. The resources invested by HOPE were another “strength” mentioned. Trainers: From your point of view, what have been the strengths of this project? What have been the principal achievements in these areas? The trainers tended to see strengths from a training perspective. They mentioned improvements in the quality of training, trainers and training methodology. However, the overall effect of that was increased adherence to norms and standards of care. In other words, the principal achievement was improved health services. Providers: From your point of view, what have been the strengths of this project? What have been the principal achievements for the resident population? The primary achievement of the project from the provider perspective was increased access to basic health services within the plantations. The combination of physical facilities, trained promoters and midwives and medicines all contributed to this increase in access. The providers mentioned other strengths, such as improved logistics, supervision, follow-up care, transportation, technical assistance, etc., but these are all related to improved access to basic health care. Promoters: From your point of view, what have been the strengths of undertaking health activities in the plantations? What have been the results, achievements, and principal areas of progress for the resident population? For the migrants? For the plantation itself? The respondents didn’t answer all of the questions, but in general, they identified greater access to health services, medicines, trained providers and health units as the principal achievement. This includes improvements in the capabilities of promoters, midwives and health center staff, all of which have led to better quality of care. b. Conclusions Almost all of the respondents saw significant improvements due to the project. However, the respondents saw achievements from different perspectives, depending on the group they were from and the phrasing of the question. Owners and administrators saw benefits to residents and migrants in terms of much greater access to health services and improved health status. They saw benefits to their plantations in an interrelated set of factors that can be reduced to: lower costs and improved productivity. Counterparts mentioned better training and coordination among health centers, HOPE and the plantations. Trainers described how improved training led to better adherence to norms and standards of care. Providers spoke of better access due to the community health units, staffing, training, educational materials, all of which have led to expanded coverage in immunization and maternal and child care. Promoters also stressed improvements in access. A potential lesson learned from these responses is that different groups may have different objectives, but they can all be complementary and compatible. The bottom line from all of these perceived strengths is that the project, by improving access to quality health care, is helping to produce healthier, more productive workers. 13 c. Recommendations The primary recommendation is to continue pursuing these multiple objectives, as long as they are compatible and consistent with the overall goal of improving the health of mothers and their children. This implies continued investment in capacity development, especially at the plantation and health center levels. The linkage between the two needs to be strengthened and institutionalized so that it becomes a single service delivery system. 3. Problems a. Questions and Findings Owners and Administrators: What are some of the problems that have affected the functioning of the health unit? Eight of the respondents did not have any problems. Of the remaining 16 the major problem mentioned was the lack or shortage of medicines (12). In several cases the respondents said they had no medicines at all at the moment. Others complained of continual stock outs. Three respondents cited promoter problems: turnover, lack of initiative and a promoter who was only available for emergencies. The other problems cited were: the economy, lack of information (educational materials?) from the Health Center, and the unit’s hours of operation (not open after 5 pm). Why do you think that other plantation administrators have not approved the establishment of health units? Five of the respondents did not know. The reasons given by the others were: lack of information (6); Lack of confidence in the program (4); lack of conscience/sensitivity (4); differences in the owners’ philosophies (2); and lack of need (2). Other reasons mentioned were: the economy, costs, too much work, afraid the workers would be organized into a union, and afraid to raise hopes and not be able to deliver. Counterparts: From your point of view, what are the principal problems or difficulties? Two of the seven counterpart respondents said there were no problems. Three others identified the lack of medicines as the principal problem. Other problems mentioned were: the fall in coffee prices, the lack of access in some areas (no place to put a health unit, no time for training), and the unstable (migrant) population. Trainers: From your point of view, what are the principal problems or difficulties in these areas? Only four trainers responded. They identified the following problems: lack of time, lack of educational materials, multiple activities, unplanned activities, lack of monitoring and supervision from the Area level, limited access to medicines, low levels of education of clients lack of a supervision guide. Providers: From your point of view, what are the principal problems or difficulties? Four of the 11 providers did not identify any problems. Among the others, the lack of medicine was the most common problem cited. Others included: the lack of interpreters for Mam speakers; cultural problems; inconvenient hours of service; lack of transportation to the plantations; and the lack of coordination, including between the plantation and the migrant’s home of origin. Promoters: From your point of view, what are the principal problems or difficulties? How can the health unit continue to be active in the plantation? Again, the lack of medicines was the most 14 mentioned problem, followed by the inconvenient hours of operation. In some areas the promoter is working on the plantation and is not available, in others the promoter works elsewhere; in some areas the health unit is poorly located. Other problems mentioned were: the lack of information (a census) about the families and serious health problems that can’t be treated at the unit. b. Conclusions Clearly, the most significant and important problem is the lack and shortage of medicines. All respondent groups mentioned this. Hours of operation often do not coincide with the work schedules of the clientele. There are some problems with promoters, especially turnover, unavailability when needed and inability to communicate with migrants. Lack of coordination, supervision and continuity of care for migrants are also problems. c. Recommendations A strategy needs to be developed to ensure that adequate medicines will always be available in all health units. HOPE needs to identify and test alternative supply systems and not rely too heavily on the MSP to provide the needed medicines. HOPE should also design and test a system to provide continuous and comprehensive health services to migrants. The other problems identified also need to be examined in more detail to see if they are prevalent. If they are, then solutions need to be developed and tested for those as well. 4. Continuation a. Questions and Findings This topic was only asked of three groups: counterparts, trainers and providers. Counterparts: Should the Child Survival program for Boca Costa be continued? As it is or with what modifications? Six of the seven respondents said yes, it should be continued. One did not reply. The general suggestion was to continue as is but with strengthening of various components and with some modifications, such as integrated services, medicinal sales outlets, collaboration with bank programs and community participation. Trainers: Should this program be continued? As it is or with modifications? All five respondents said it should continue as is, or with better planning, more educational materials and the strengthening of supervision. Providers: Should this program be continued as it is or with what modifications? All 11 respondents said the program should continue, largely as is, but with some modifications to address problems identified above. The major suggestion is to find a way to ensure a constant supply of essential medicines at the health units. Another important recommendation is to include a service program for migrants in their place of origin. A further suggestion is to resolve the problem of transportation so that the health center staff can attend to migrants, especially mothers and children, at a more convenient time (from 5 pm on). 15 b. Conclusions Practically all of the respondents believe that the program should continue. The most important modification would be to design and install a system to ensure a constant supply of essential medicines at the health unit. Other modifications that were suggested include: greater community participation, strengthened supervision, greater continuity of care for migrants, transportation for Health Center staff to be able to provide services at more convenient times, increased educational materials and development of more medicinal sales outlets. c. Recommendations The program should continue, but with new strategies to resolve the major problems mentioned above, in particular a reliable supply of essential medicines, continuity of care for migrants, better supervision and transportation for health center staff to and from the plantations. One of the most important operational objectives should be to ensure continuity and reliability in all of the key project components. C. CROSS-CUTTING ISSUES This chapter summarizes the views that various respondents have about community mobilization, behavioral change communication, capacity development and sustainability. Capacity development includes facility strengthening, field worker performance and training. 1. Community Mobilization a. Questions and Findings Unfortunately, the interview questions do not correspond well to the AID evaluation guidelines. The community mobilization and participation strategies emphasize participatory education and health campaigns in the plantations, especially vaccination campaigns. The questions on this topic were addressed only to the 19 promoter respondents. Promoters: Do the migrant mothers participate in the health campaigns? Why? Sixteen of the 19 promoters answered yes. One did not respond and two said no (one because there were no migrants in that plantation). Several promoters said their participation was sometimes limited due to language problems, lack of interest, lack of immunization cards and because many women did not have small children. The data indicate that the primary (or only) “participation” was to bring their children to be vaccinated. What community mobilization activities has the project and its associates promoted? How effective have they been? This question may have been misunderstood, since most of the responses identify training of promoters as the principal activity. Other activities mentioned were the development of health units and the selection of promoters. To what degree has the community responded to these activities? Has the response been greater from migrants or residents? Only seven promoters answered this question. One mentioned that the timing of the campaign affected the response; one said residents responded more and another that migrants responded more; one mentioned that the response was greatest after an education 16 session. What recommendations do you have for the future? Only six promoters responded. The suggestions were: continue monthly training; provide more training of promoters and mothers; make training more participatory and maintain coordination. Two respondents had health-related suggestions. One suggested greater education regarding growth monitoring and another suggested that a study be undertaken of the nutritional status of children over six months. What impact on program implementation have such factors as security, policies, roads, access to the plantations and coffee prices had? Again, there were only six respondents. Four commented on the negative effects of the lower price for coffee. Another noted that the number of migrants had declined, probably as a result. b. Conclusions Not much can be drawn from this information regarding community mobilization, except that there may not be very much of it. The promoters do not seem to understand what it means and the principal “participatory” activities mentioned (training of promoters and attendance at vaccination campaigns) are not participatory at all. c. Recommendations If community participation is part of the project strategy then HOPE should undertake a study of what is actually going on in order to determine what should be done in the future. 2. Communication and Behavior Change a. Questions and Findings Counterparts: What communication/IEC activities have been provided by the project and its associates? Who was in charge: How effective was it? The seven respondents did not answer all of the questions. Three identified the types of activities: training of promoters by the HOPE technical team and reproduction of cassette health messages in migrant languages (2). Three others identified the groups in charge: HOPE personnel; HOPE and institutional personnel; and community participants. To what degree has the community responded to these activities? Only five respondents answered this question. The answers ranged from quite large to moderate responses from the community. What recommendations do you have about IEC that can be applied in the coming years? Two did not respond. Recommendations were: continue the training; evaluate the personal constantly; keep repeating the messages; “socialize” the messages to be more appropriate in other areas. The most specific recommendation was to use an announcer with a clearer and “graver” voice that will carry greater distances, and to adapt the music to each locale. Table 4: Respondents and Questions Analyzed Respondent No. Questions Counterparts 7 3 Providers 11 4 Promoters 19 3 Beneficiaries 20 4 17 Providers: What communication/IEC activities have been promoted by the project and its associates? The 11 respondents mentioned a variety of activities and materials that have been used: broadcasts of basic messages; IEC targeted to specific problems (diarrhea, ARI, immunization, nutrition); talks, videos, television, loudspeakers, flip charts, cassettes, interpersonal communication, and so forth. How effective have these approaches been? Most of the respondents said they have been good, they get attention, and they have been effective. Were the IEC materials adequate? Five of the six who responded to this question said they were adequate. Several said that people understand and accept the messages. One said that they are adequate but could be better (better messages, larger figures, more colors). One critic said the material is not adequate. The messages aren’t clear and are not direct. To what degree has the community responded to these activities; has there been a change in maternal behavior as was expected? Two respondents said it was difficult to tell, but the other nine said there have been positive changes: health has improved; maternal and infant morbidity and mortality have declined; responses to vaccinations and hygiene have improved; the lectures have had an impact; people understand more and comply more with healthy behavior norms. What lessons have been learned that can be applied in the coming years? “Lessons learned” should be expressed as “if – then” statements, such as “if communities are involved in running a program then they are more likely to accept it.” The respondents did not phrase their responses in this manner; rather they made recommendations, which are still valuable: · Train the volunteers · It’s not enough to involve the promoter, community leaders also must be involved · Continue the assistance · Reinforce training, undertake supervision, and motivate mothers to pay more attention to their children · Provide more education, understand the needs of the population, obtain the needed resources · Train the midwives, promoters, plantation managers, ministry staff · Write the basic health messages in the languages of the migrants, use more graphic material · Improve the coordination with MSP/HOPE/IGSS regarding medicines, vaccinations, transportation of personnel, equipment and medicines · Coordination should begin at the district level; planning should be with the nurse and doctor; in short, all personnel should be involved in the process · Provide more education, more permanent personnel from HOPE Promoters: To what degree has the community responded to these activities; has there been a change in maternal behavior as was expected? What changes have been noted (e.g., better hygiene, greater demand for services, reduced emergency transportation due to timely service; greater use of latrines, protection of water sources)? Of the 19 promoters, 14 responded positively, noting such changes as: better hygiene, better child care; fewer visits to physicians, more visits to the health centers for antenatal care; greater and faster demand for health services; 18 and less need for emergency transportation. One promoter said there are no deaths now. A few people were equivocal. Two said it was hard to see much change, another said that some things have improved and others have not, and one commented that the men didn’t like the talks. Do you believe that is easy or difficult to persuade mothers to change their behavior? Why? Only three promoters said it was easy – or not too difficult. Eleven said it was difficult, especially for migrants and illiterates, who often don’t understand the messages and don’t attend the educational talks as much as the residents and those who are literate. Several promoters said that some people understand the messages but don’t change. One said that young people are more likely to change than older people. What recommendations do you have for increasing the participation of mothers in educational activities in the coming years? Four promoters did not answer this question. Many of the others suggested more education: meetings, IEC materials, visits, etc. Several suggested motivation, for example, to get people to take responsibility for their own health. Beneficiaries: Have you participated in any talk-session on health education in the plantation? If you listened, what was discussed? Of the 20 respondents, three did not answer and 14 said no, they had not participated in a talk. The three that did mentioned hygiene, childcare, nutrition and health issues as discussion topics. Do you think that these messages speak to the most important health problems in you family? Why? Only the three respondents who answered positively to the previous question were supposed to respond to the remaining questions. However, there were six respondents. These three all responded yes. The reasons given were quite general: because one learns; because it is good; because it helps to educate people; so that the children don’t get sick, and so forth. Do you plan to do what was recommended? Again, only three persons were supposed to be asked this question, but there are six responses. All responded yes. How would you do what was recommended? Only one of the six respondents seems to have understood the question. She said that she would wash her hands, the utensils and keep clean. b. Conclusions Not much can be drawn from these data about the effectiveness of the behavioral change strategies. The respondents generally believe that changes have occurred in health-seeking behavior, utilization of services and morbidity and mortality. However, this seems unlikely given the small number of beneficiaries who say they attended a health education session. Better information is needed about this activity before any conclusions can be drawn about its effect on behavior. c. Recommendations HOPE should conduct an assessment of the IEC activities to determine: 1) whether the messages are being heard and understood by the target groups; 2) whether the target groups have internalized those messages; and 3) whether that has had any effect on their health-seeking behavior. This assessment should include representative migrant populations as well as residents. 19 3. Capacity Development a. Strengthening of Health Units Questions and Findings Providers: Has the project done anything to improve health services? All of the 11 providers answered yes to this question. The most common input was training, but other contributions mentioned were educational materials, loudspeakers, and transportation for midwives to attend training, and a refrigerator to keep vaccines cold. How effective was this for the health center and the community? Most of the providers said it has been effective. Examples: more people coming for health services; acceptance by the population has increased; the MSP is now providing medicines; the management and treatment of illnesses has improved; and the health units complement the health center services. What lessons have been learned that can be applied in the coming years? As before, these are mostly recommendations, such as: continue support; treat people humanely; continue the training and put it into practice; evaluate it; continue follow-up supervision and continuing/refresher training; and improve the training techniques. Promoters: Has there been any improvement in the health unit in this plantation? What? Four of the 19 promoters saw no improvement. One of these said that the assistance was poor. However, 12 said that there had been improvements: medicines, equipment, remodeling of the unit, even a medicinal outlet. Do the people know that this unit exists and the services it offers? Everyone answered yes. One noted that people were coming from other plantations. Do the mothers come back when they have an appointment? Except for three who did not respond there were only two promoters who said they did not come back. Sixteen said they did. Conclusions According to both the promoters and providers there have been significant improvements in the health services provided and the use of those services by the target population. Recommendations It would help to have more detailed information about the improvements that have been made throughout the 150 plantations and the effect that has had on utilization and health. If HOPE plans to promote a “model” system for replication, then this information would be critical and it should be compiled and analyzed. Table 5: Respondents and Questions Analyzed Respondent No. Questions Counterparts 7 3 Providers 11 4 Promoters 19 3 Beneficiaries 20 4 20 b. Field Worker Performance Questions and Findings Providers: How effective was the assistance of HOPE in improving your performance as a health provider? All 11 respondents gave positive responses. One provider said that as a result he/she works better and has more knowledge than he of she is using at the moment. Another commented that it was effective because the assistance included resources for training (of promoters) and materials. Do you believe that the cases referred by the plantation health units to the health centers have been selected correctly? Have you received more, the same or fewer referrals from the plantations than before the project began? All responded that the referrals have been appropriate. One provider commented that only complicated cases are referred now. Another mentioned that more pneumonia cases are being referred now. Overall, it seems that some centers now have more referrals than previously and some have fewer – both due to better screening at the plantation health units. Conclusions Although the questions are limited, it appears that the assistance from HOPE has led to better screening and referral from the plantations to the health centers and this has improved performance at both levels. Recommendations There is not enough data in this study to assess performance adequately. If this is a priority for the project then HOPE should undertake a study to assess the performance of the key health workers at the community and health center levels. This should include clear and standardized performance indicators to ensure comparability across sites. If possible, it would be useful to include cost data at the same time so that unit costs and productivity could be measured in monetary terms. That information could be very useful in marketing this program to commercial enterprises like the plantations. c. Training Questions and Findings Providers: How was the training that you received? Was it practical? What did you like best? What did you like least? All 11 respondents except one responded positively. Some commented that the cascade approach worked, that the training was effective, simple, clear and dynamic. One respondent said that now they could do an evaluation and train the promoters. One person criticized the messages as being incomplete, unclear and the time too short. What is the limiting element in this (training) chain? Two respondents did not answer the question and two had no complaints. The other seven identified a number of limitations: the lack of monitoring and accompanying supervision at the community level; the lack of providers to train; the shortage of time, which led to some confusion and uncertainty about the messages; the limited availability of medicines; and that not everyone is capable of transmitting such messages, 21 only in summarizing them. Were the educational materials used in the provider training adequate? All 11 providers said yes. Comments were: they are simple and applicable to the community; they are very important, they provide them and they are used; the themes are adequate and focus on key points; there should be more with clearer messages for the user. Promoters: How was the training that you received? Was it practical? What did you like best? What did you like least? Almost all of the promoters loved the training and commented on how both the theoretical and practical segments were very good. In general, they believed that the training gave them knowledge and skills that they could use in serving mothers and children. Is there any evidence that the training has resulted in greater knowledge and/or greater performance on your part? In what aspects? Two promoters did not answer, but the rest were unanimous in their belief that they had learned something valuable and were able to put it into practices. Several said that they now have a better knowledge of different types of illnesses, how to identify signs and symptoms, how to treat them. The result has been better performance and improvements in the health of their clients. Training Assessment Guide: As part of this study the Evaluation Team completed 14 observations of training using the supervision checklist as a guide. The training observed was mostly of midwives and promoters. The results are summarized in Table 6, which shows that the performance of the health personnel in educating volunteers is very good. However, the facilitation skills could be improved. It is also obvious that this guide can be used to verify the knowledge and skills acquired during training. Note: item 13 is not appropriate for group training. It is more appropriate in tutorials and individualized training. Table 6: Supervision Guide for Training Activities Supervision criteria Yes No NA 1 Education objectives are explicit 11 1 2 2 Educational objectives are know by the participants at the outset 11 2 1 3 Education is participatory 13 1 0 4 The educational materials are adequate 10 2 2 5 Focus is more on behavioral change than increased knowledge 9 2 3 6 The session stimulates critical thinking, not memorization 10 2 2 7 The session is entertaining, the participants maintain interest 12 1 1 8 The facilitator/trainer asks questions and listens to answers 12 0 2 9 The trainer verifies that the key concepts have been learned 10 1 3 10 The facilitator is a model of facilitating skills 7 3 4 11 The technical concepts and messages are acted out 13 1 0 12 The facilitator evaluates (e.g., pre-post test or questions) 7 3 4 13 The facilitator negotiates behavioral changes with the participants 5 3 6 Conclusions Provider and promoter training appear to be excellent. Both groups, especially the promoters, praised the theoretical and practical segments and believe that they have received valuable 22 information and learned important skills. As a result, service quality has improved and that has led to improved health for migrants and residents. Some of the weaknesses include the shortage of time and the lack of follow-on field supervision. The training observation guide indicates that the health staff is doing a very good job in training community volunteers (promoters and midwives). Recommendations Training should continue, in fact, steps should to be taken to ensure that continuing education becomes a standard part of the project. Follow-up field supervision also needs to be expanded. Providers need more in-service training, especially in the communities. This implies that the master trainers need to be able to accompany them to the field to conduct on-site, practical training, supervision and continuing education. Volunteer promoters and midwives need more training in preventive health care. Health Center staff is missing a great opportunity during the vaccination campaigns to supervise the volunteers and to observe their performance in educating migrants and residents about basic health care concepts and messages. This would also reinforce the public perception of the volunteers as key providers and bona fide members of the health team. Facilitators also need refresher training. This could be provided in several ways, including distance learning, provision of materials from other institutions. 4. Sustainability a. Questions and Findings Owners and Administrators: How is your plantation providing support to the head of the health unit? Table 7 shows the spontaneous and prompted responses from the 24 owners and administrators interviewed. Practically all of the plantations claim to facilitate promoter training, largely by giving permission and/or providing transportation to and from the Health Center. Most also say that they pay the promoter’s salary. However, previous data indicates that this is a loosely interpreted question. It includes promoters who are also plantation workers and are paid for that work, not for being promoters. Some of these promoters are only available for emergencies. Table 7: Support Provided by the Plantation to the Health Unit Type of support provided Spontaneous Prompted Total Facilitating attendance at training for promoters 19 4 23 Paying the promoter’s salary 14 5 19 Providing a vehicle for emergency transportation 11 6 17 Supporting the health campaigns 7 7 14 Improving the infrastructure of the health unit 8 5 13 Providing medicines and medical supplies 8 3 11 Paying a professional or technician to provide services or supervision 1 1 2 Other Quite a few plantations claim to provide transportation to health centers and hospitals for workers 23 who need emergency care. In most cases this would probably be provided to adult workers who were injured in the course of their work. This might also include emergency obstetrical care, although this would probably be a rare event. Other support includes the following: nutrition, promotion of health activities, supervision and health education. What other help could the plantations provide to improve the health activities for your resident and migrant populations? Table 8 shows the responses using the same categories as in Table 7. This question did not use the spontaneous/prompted distinction. It could be interpreted as showing the additional support that the plantations are willing to provide. However, it may indicate that they are not willing to provide much more than they are now. For example, 23 of the 24 plantations already facilitate promoter training, so what did 10 of those mean when they said yes to the second question? Table 8: Other Support that the Plantations are willing to Provide to the Health Unit Type of support provided Total Facilitating attendance at training for promoters 10 Paying the promoter’s salary 9 Providing a vehicle for emergency transportation 9 Providing medicines and medical supplies 8 Improving the infrastructure of the health unit 5 Paying a professional or technician to provide services or supervision 2 Supporting the health campaigns 1 Other 5 Other support includes “improving” salaries and benefits, setting up a medicinal sales unit, more training and time. There are a number of interesting responses from individual owners/administrators that may be worth quoting: ß No medicines ß Whatever is necessary at the time. It’s open. ß No more than now. ß Pay the promoter the same as a plantation worker. ß Medicines, up to a certain point. ß At the moment, only medicines ß Depends on the economy. Right now, nothing. Are you interested and disposed to help in the continuation of this program, under what conditions? Of the 19 who answered the question, 16 said they would continue, basically without conditions. However, two respondents wanted the medicine problem resolved and two thought that the MSP should take more responsibility for health services. Only one respondent mentioned economic conditions as an obstacle to support. Are the costs of operating this unit in the plantation’s budget? Eleven of 24 respondents answered the question. All but two said that the operating costs are already included or will be. They did not elaborate on exactly what would be covered, however. 24 Is the international price of coffee affecting your ability to continue supporting the health unit? In what way? Three respondents didn’t answer, five said yes and 12 said no, the price of coffee would not affect the continuation of the project. This was somewhat surprising, since the general perception was that most plantations would not be able to continue supporting health units in a declining market. Is it helpful to have a health unit in the plantation? Three respondents did not answer this question, one said no and the rest all said yes, it is a benefit, not just in terms of greater access to health services for the workers, but also greater income for them and the plantation. Again, a healthy worker is more productive and that benefits the worker as well as the plantation. Counterparts: Do you believe that the technical assistance for the health activities of mothers and children should be maintained when this project ends? In what form? All eight counterpart respondents support the continuation of technical assistance. Among the suggestions were: give more attention to the volunteers; quality training; seek funding from other sources; organize the mothers to adopt self-care practices. Which of the activities that the project supports should continue after the project ends? Which activities need more assistance in order to continue? Why? The responses are summarized in Table 9. It is interesting to note that the counterparts believe that the Area councils and Continuing Education Commissions, which they say should continue, would need no additional assistance. Training and supervision, however, would require assistance. The other categories that would need assistance include: immunizations; supervision of community health unit activities; the health units themselves; provision of medicines; community IEC and motivation activities. Table 9: Project Activities that Should Continue Activities Continue Need Help The Area Health Councils 6 The Continuing Education Commission 6 Supervision of personnel and community volunteers 6 2 Training of personnel and community volunteers 5 2 Group trainers 4 3 Other 4 4 Trainers: Do you believe that the concept of having a stable group of master trainers is sustainable? Why? All seven trainers responded positively to this question. In general, they believe that a stable group has already been formed, it is cohesive, the members are well trained, they enjoy what they do and no additional budget would be required to keep them functioning. Providers: Do you believe that it is conducive to the sustainability of the education process to have a stable group of master trainers? Why? Unfortunately, the providers were not asked about the sustainability of health services. Three respondents did not answer the question. One said that there is no master trainer group in the Health Center. The others responded positively but suggested that they should be more involved in supervision and monitoring, that there should be a district-level plan and that there should be more coordination with other institutions. 25 Promoters: What health activities in this plantation can be continued indefinitely even with the low prices for coffee? Three respondents did not answer the question. One did not know, three said no, none of the activities could continue. One said nothing could continue without resources. The remainder said that all of the services could be continued or mentioned individual services that could continue (e.g., mothers’ education, hygiene in the galeras). b. Conclusions There is a great deal of interest in continuing the project activities, especially from the plantations. Most claim that they are already providing significant support (especially in facilitating promoter training and paying their salaries) and most are willing to continue – or expand – that support. Eleven of the plantations already have a budget for their health units. Surprisingly, half of the plantations owners/administrators said that the price of coffee would not affect their support. Nevertheless, it is also clear that external assistance will still be needed, especially in training, supervision and the provision of medicines. The project model seems fragile. Such key areas as promoters, transportation, medicines and coordination are project dependent. It is not at all certain that they will continue once HOPE support ends. HOPE is the glue holding the model together. Its facilitating contributions are very important, at all levels, from the Health Councils to the technical staff who visit the plantations regularly. Once that glue is removed, the model could break apart. Who will take on those facilitating roles? The current strategy seems to assume that the MSP, IGSS and the plantations will work it out. But if a plantation is not able to provide transportation for the health center staff, or the district cannot get medicines to a plantation, or no one is responsible for supervising the midwives, then those critical components of the model could end and the whole system could break down. c. Recommendations The most important recommendation is that HOPE should undertake a sustainability analysis to determine what would be required to ensure the continuation of key project activities after the project ends. This analysis should not be limited to financial considerations, but should also include personnel, transportation, medicines and such key systems as logistics, information, coordination, supervision and training. HOPE should also organize meetings of plantation owners and administrators to define clearly what is needed to support the health services and what the plantations can provide. The selection criteria and role of the promoters should also be defined clearly. Specific attention should be given in all of these activities to migrants and their needs. D. PROGRAM MANAGEMENT This chapter covers eight topics: planning, staff training, coordination and follow-up supervision, human resources, financial management, logistics, information management and technical and administrative support. 1. Planning a. Questions and Findings Five questions were selected for analysis: 1 from providers, 2 from counterparts and 2 from 26 promoters: Counterparts: What personnel were involved in the planning of the child survival program? In the analysis of problems and development of solutions? All of the eight respondents mentioned two or more people from their staff who participated, including physicians, nurses, technicians and managers from local and district departments. Are you familiar with the work plan for the project? To what degree is the plan realistic? What could have been done to improve the implementation of this plan? One person did not answer; two others just answered yes or no. The remainder gave generally positive responses about the plan. Providers: Have you participated in any type of analysis of problems and development of solutions? Seven of the 11 respondents said they had participated. Promoters: What groups or persons have been involved in the planning of the program? Eight of the 19 promoters interviewed didn’t respond or said that no one that they knew had been involved. The remainder mentioned managers, plantation owners, HOPE, health centers and people at those levels. No promoter was involved. Did you participate in the analysis of problems and development of solutions? How was this information shared, how often and when? Ten of 19 didn’t respond or said they didn’t participate. The others mentioned times when they were involved, usually providing information to or discussing registration data with health center and similar staff. b. Conclusions The responses indicate that counterparts and some providers have been involved in planning and problem solving, but not promoters. c. Recommendations Continue involving counterparts and providers in planning and problem solving; develop strategies for involving promoters and beneficiaries (i.e., residents and migrants) as well. 2. Staff Training a. Questions and Findings Two questions were selected for analysis from the 11 counterpart respondents and two from the eight providers who were interviewed. Counterparts: Has there been any change in the knowledge, skills and competency of project personnel and their associates? Is there evidence that the personnel have applied these skills inside and outside of the program? All responded yes, mentioning MECA, IEC and health services in general. One mentioned that all of the volunteers and immigrants have not been trained yet. 27 Are there enough resources for personnel training? Seven of the eight said yes, one did not respond. Providers: From the training facilitated by this project, have you seen a change in your knowledge, skills and competence? All 11 responded positively, except for one who had not received training. Have you had the chance to apply the new skills learned from this project? Where? Seven of 11 responded yes and mentioned applying what they learned with community vigilantes, in the private sector, with the evangelical church, in the health center and at home and in the community. b. Conclusions Almost everyone responded that the training has had an effect, not only within his or her work, but outside as well. In general, this training seems to be effective and valued. c. Recommendations This type of training should continue, in response to individual needs. Inter-institutional coordination should be strengthened to improve the training activities. Other counterparts, not just HOPE, should contribute to personnel training. 3. Coordination and Follow-up Supervision a. Questions and findings Two questions each were selected from the Counterpart, Provider and Promoter questionnaires, and one from the Trainer questionnaire. Counterparts: How effective is the help from Project HOPE personnel to your personnel so that they can carry out their duties effectively? Are the number, roles, activities and frequency of their supervision visits adequate to meet the technical needs of the program? All eight respondents praised the assistance from HOPE, but mentioned ways to improve it, for example, more frequent visits, more time per visit and more trained supervisors. Do you believe that the coordination and follow-up (supervision) system is institutionalized and that it can be maintained? Only one respondent mentioned that the system still wasn’t quite institutionalized. Another said yes, but that decision makers need to be more involved. All of the others thought that the system was already institutionalized. Trainers: Do you believe that the coordination and follow-up (supervision) system is institutionalized and that it can be maintained? The six respondents were all very positive about the system and most said it was institutionalized. One said it wasn’t for lack of finances. Providers: Do you believe that the coordination and follow-up (supervision) system is institutionalized and that it can be maintained? What do you recommend to improve it? All but one of the 11 providers answered positively and gave examples of various types of coordination, such as between HOPE, the plantations and the health centers and coordination with IGSS and 28 MSP. One respondent replied that they haven’t done any follow-up yet, but need to do so. Do you believe that the performance of the plantation promoters that you supervise has improved? Why? All but two responded yes and credited the training and/or the supervision for improving performance. One responded that he/she did not supervise promoters; another said that the promoters lack adequate training in ARI, diarrhea and malnutrition. Promoters: Have you been visited by health personnel in the past three months? Who visited you? What did he or she do (review registration, provide recommendations)? Only two of the 19 respondents had not been visited. Over half mentioned HOPE as having made a visit. Others mentioned visits from IGSS, Agrosalud, the health centers, the Malaria program, etc. It seems that some of the health center/IGSS/HOPE visits were only for vaccination campaigns. Others were to review forms, registration books, materials and even medicine stocks. Have you visited the health center in the last month? Did you bring a report? Collect medicines? Did anything else happen during the visit? Sixteen answered yes. The other three did not respond. The most common reasons mentioned were to collect medicines, attend meetings and deliver reports. b. Conclusions Most respondents believe that coordination and follow-up supervision are good and are already institutionalized. The fact that most of the promoters had been visited in the past 1-3 months is a good indicator of this. c. Recommendations Coordination and supervision are important and need to continue and be strengthened so that they operate at all levels. 4. Human Resources a. Questions and findings One question each was selected from the Counterpart, Trainer and Provider questionnaires. Counterparts: Are there (sufficient) human resources so that the activities will be sustained? About half said yes, the other half noted deficiencies in auxiliary nurses, rural health and the health institutions. Trainers: Are there (sufficient) human resources so that the activities will be sustained? For example, to retain the personnel trained in training and supervision? Most of the trainers responded yes, but with significant qualifications. It cannot be done without adequate financial resources, enough trainers and lower turnover. The personnel at the health centers, for example, are insufficient to meet the demand. Providers: Are there (sufficient) resources to enable the activities to be maintained? All 11 respondents answered yes, but they mentioned different sources: IGSS, health centers, NGOs, 29 MSP technicians and the private sector. b. Conclusions In general, or in theory, there are adequate human resources to maintain the program. However, there are gaps, especially at the health center and promoter levels, but also at the counterpart and trainer levels. Turnover will require recruiting and training replacements. Migrants, in particular, increase the seasonal need for additional human resources, especially those who can speak their languages. c. Recommendations If the program continues there will be a need to identify, recruit and train personnel at all levels: counterparts, trainers, providers and promoters. Expansion of the program and personnel turnover will require that. One trainer suggested a data bank of trained trainers so that replacements can be identified quickly. Continuing education will also be required for all staff. To serve migrants better, each plantation should have at least one promoter or assistant promoter who can speak the migrant’s language. 5. Financial Management a. Questions and findings Two questions were selected from the Counterpart questionnaire and one from the Promoter questionnaire. Counterparts: Have the costs of the services that you offer been estimated so that they can be budgeted annually? Were these costs included in the work plan/annual programming for last year? All but one of the eight respondents answered yes, with some limitations. For example, the Health Area estimates costs, the health center only estimates the costs of medicines and the costs are in the work plan but the plan is not executed. Have you considered whether the technical and financial resources exist that are needed to sustain the program in the future? Surprisingly, all but one of the eight respondents responded yes, with certain caveats, such as depending on priorities, coordination and management. One respondent said no because the MSP frequently cuts the budget. Promoters: How are you covering the cost of medicines, (through the) Ministry of Health, the plantation, the patient’s family, donations from HOPE, other? Fifteen of the 19 respondents mentioned the Ministry of Health/Health Centers; six mentioned HOPE, two mentioned the plantations (and HOPE) and one mentioned Agrosalud. Only one did not manage pharmaceuticals and two others did not respond. b. Conclusions The counterparts are optimistic about being able to continue to cover the costs of the program in the future. However, one respondent mentions a key issue with respect to government financing, which is that it is unreliable. The funds may be in the budget, but that does not mean that they 30 will be spent as budgeted. The promoters are also heavily reliant on MSP and HOPE to cover the cost of medicines. Funding from the former is unreliable and time-limited from the latter. The plantations and families are not significant sources of funding at the moment. c. Recommendations A prudent recommendation is to look for alternative sources of funding, especially for medicines. It would also be helpful to calculate the unit costs of each component so that alternative budget scenarios can be generated. Local government units may be a source of funding under the decentralization initiative. 6. Logistics a. Questions and findings Two questions were selected from the Counterpart questionnaire and one each from the Trainer, Provider and Promoter questionnaires. Counterparts: What impact has logistics had (provision of medicines, equipment and vehicles) on the implementation of this project? Have the in-kind donations provided by HOPE resolved problems; created some others? All but one of the eight respondents responded positively, noting that medicines have been a significant contribution of HOPE. The only critic said that the impact has been little because the contributions do not arrive on time. Is the provision of essential medicines now sustainable? If no, what should be done to assure that? Six of the eight respondents said yes, it is sustainable. The other two did not respond. Trainers: What impact has logistics had (particularly transportation, educational materials and medicines) on the implementation of training and follow-up supervision? All but one of the six respondents said that the impact has been good. They identified transportation, educational materials, medicines, in particular. One person mentioned some problems with getting enough IEC materials. Another thought that logistics has been too limited in general. Providers: What impact has logistics had (resources, medicines, transportation, educational materials) on the implementation of the project? All but three of the 11 respondents praised the logistics component for making medicines, educational materials and transport available. One did not answer and two remarked about the lack of these inputs. One person noted that the plantations have accepted the program, but they do not provide vehicles and some don’t provide any medicines at all. HOPE and its associates were praised for providing needed commodities and transport. Promoters: Is the provision of essential medicines now opportune? What should be done to ensure this? The views were split on this. Four of the 19 respondents did not answer the question. Of those that did, seven were positive and six negative. The complaints were that: not enough medicines are provided; there are stock outs at the Health Center and supplies are irregular. 31 b. Conclusions The general view is that HOPE and its associates have done a good job in making essential medicines, educational materials and transportation available. Only the counterparts were asked if the provision of essential drugs is sustainable and most believe that it is. Nevertheless, it is clear that at least some sites are not getting the assistance they need, do not enough and/or it is not consistent and timely. c. Recommendations The achievements made to date need to be maintained and the gaps in logistical support filled. During the group discussion a recommendation was made to involve the associates more in logistics planning and that strategies be developed to ensure true sustainability, such as sales outlets and rural drug stores. 7. Information Management a. Questions and findings Two questions were selected from the Counterpart questionnaire, one from the Provider questionnaire and two from the Promoter questionnaires. Counterparts: How has the information system helped assess progress toward objectives; is this information displayed in the health situation room? All eight responded yes, but it appears they were responding to the second question, rather than the first. Do you know if the monitoring and evaluation information of this project has been shared outside of this project (e.g., with the central MSP, collaborating institutions, other NGOs, the academic sector, health councils)? All but two of the respondents said yes, but their answers seem to refer to routine dissemination to Area superiors, HOPE and other program participants. One mentioned an annual meeting with (participating) plantation owners. Providers: Is there a system in your organization for measuring the achievements of activities supported by HOPE? Is this information displayed in your health situation room? All 11 of the respondents responded yes to the first question. Those who responded to the second question said that it is included in overall data charts, such as vaccinations, but not compiled or displayed separately. Promoters: Do you use forms or tables to keep track of your activities? Is this information helpful to you in some way? Two of the 19 promoters did not answer the question. The remaining 17 all said that they keep track of their activities. Nine of those described various ways that they used the information: as background information about the patient; to report on types of illnesses found; and to keep track of medicines. Two noted that they used MSP forms and several said that they sent the information to the MSP health centers. None mentioned that they use the information to assess improvements in health or health service utilization. 32 Did you go to school? Until what year? Three of the promoters did not answer and one did not attend school. Of the remaining 15, the average was about 3 years, clustering around the second and third grades. The highest was sixth grade and the lowest first grade. b. Conclusions Counterparts contend that data is collected, displayed and shared with other agencies, but this seems to reflect routine dissemination practices. All of the providers said that data is collected, but it is integrated into the health center activity data. It is not compiled or displayed separately. Promoters collect and use the information largely for reporting and monitoring stocks of medicine. From these responses it does not seem that data is being used to assess achievement of project health objectives. It is largely incorporated into MSP activity reports. c. Recommendations The discussion group suggested strengthening the information system and its use. There were suggestions to coordinate with the plantation information systems, find ways to involve communities and to “socialize” the information in meetings to make them more useful. However, from the responses obtained, it seems that the data is merely incorporated into the larger MSP database. It would seem appropriate to take a closer look at how this data is compiled and used, especially if it is used to assess project effects on health utilization and health status. The data needs to be disaggregated so that the effects of the project can de measured. Otherwise it will not be possible to determine if the project is having any impact on disease prevention and health￾seeking behavior. 8. Technical and Administrative Support a. Questions and findings Two questions each were selected from the Counterpart and Provider questionnaires. Counterparts: Describe the type of external technical assistance that your institution has received to date from Project HOPE and if it has been timely and useful. All of the eight respondents received several forms of technical assistance. The most common was training (6), followed by, financial support, planning assistance, supervision, information systems, logistics (5 each) and educational materials (4). Three respondents suggested that more assistance is needed in supervision, vehicles, training and nutrition. What technical assistance that the project needed was not available? Two did not respond. The others said none. That is, they project received all of the technical assistance it needed. Providers: Discuss the external technical assistance that your institution has received from Project HOPE. All 11 of the respondents had received multiple kinds of assistance, with training and educational materials at the top of the list (10 each). They were followed by: planning assistance (9), financial assistance, information systems, logistics/medicines (8 each) and supervision (7). Two suggested other needed assistance: travel allowances and medicine outlets. What technical assistance that the service needed was not available? Two did not respond. One 33 mentioned the need for audio-visual materials and transportation for supervision. The others thought that nothing was overlooked. They received all of the assistance that they needed. b. Conclusions The assistance provided by HOPE was important and valuable to the providers and counterparts, most of who said that all of their needs were met. The additional needs identified by a few were in medicines, transportation, nutrition and training. c. Recommendations The project needs to maintain its technical and administrative assistance to current providers and counterparts. This will also be a need in the project extension. HOPE will need to identify and respond to the needs of the new counterparts and providers so that they can carry out their responsibilities. E. OTHER ISSUES IDENTIFIED BY THE TEAM The Evaluation Team members discussed the findings from the Qualitative Survey in their subgroups and later in a plenary session. Team members interjected findings and insights from their own experiences. In addition, preliminary results from the KPC survey and the most recent report on benchmarks were also introduced and raised other issues. As a result, the following additional conclusions and recommendations have been made. 1. Constraints and Facilitating Factors. As part of the discussion exercise, the Evaluation Team members were asked to identify factors outside of the control of the project that affected performance, either positively or negatively. The most important of these are listed below. · Economic: The fall in international coffee prices has had a significant effect on livelihood. Many people are out of work and that limits their ability to remain healthy. · Financial: Probably the greatest constraint is the lack of resources at the MSP for supporting this program. Health Center staff is usually quite willing to help but is constrained by lack of funds for travel to the plantations, for medicines, for supervision of the promoters and follow-on training, and so forth. The MSP itself has extremely limited funds. The plantations are very supportive of the project and are willing to contribute to it – up to a point. · Cultural: The migrant Mayan population, in particular, is often reluctant or unable to understand or comply with healthy behaviors due to poverty, male dominance, lack of Spanish, limited education, etc. The project finds it difficult to deal with these cultural issues. Neither the project nor the MSP/IGSS health systems are set up to follow migrant populations. · Environmental: Many plantations are in remote areas. They are difficult to reach, which 34 limits access to health care. Seasonal rains turn roads into muck and isolate residents from health facilities. Altitude and topography affect health in many ways: respiratory infections, exposure to mosquitoes (dengue and malaria), nutrition, etc. · Political: Migrants are a high priority for the government. However, Guatemala has had four Ministers of Health in the past three years and policies have changed rapidly, as have budget priorities. The MSP should be providing all of the basic medicines (including contraceptives) to migrants (a “high priority” group), but that is not happening. 2. Access to Health Services a. Conclusions From a qualitative perspective, the major achievement of the project seems to be that it has increased access to basic health services for thousands of plantation workers, both resident and migrant. This has resulted in increased utilization of priority maternal and infant care services. And that has resulted in improved health for both mothers and their children. Quantitatively, it is more difficult to determine how many plantation workers have been served. The only data we have at the moment is summarized in Table 10, which shows the number of visits made in the last quarter of 2000 to plantation health units by migrant children under age 5. Table 10: Visits to Plantation Health Units by Migrant Children < 5 years of age, October - December 2000 Department ARI Diarrhea Malnutrition Anemia Total Quetzaltanango 1,747 1,080 31 43 2,901 San Marcos 2,441 1,385 65 117 4,008 Suchitepéquez 1,455 616 4 14 2,089 Total 5,643 3,081 100 174 8,998 Source: Project HOPE, Programa Supervivencia Infantil, Septiembre 2001, p. 9. These data do not include 1,979 vaccination visits made during this same period.4 If we add the vaccination visits, the total for 150 plantations is almost 11,000. Although this appears to be a large number, it averages out to 73 visits per health unit over the three-month period, or about one visit per day per health unit. In addition, it is important to keep in mind that these are data on visits, not the number of individuals served. That is likely to be at least half the number of visits, or about one migrant child served every other day by a plantation health unit. Unfortunately, we do not have pre-project data to make comparisons. We do not have data for visits by resident children, maternal visits, or visits by adults for emergency or routine care. These would add to the health unit caseload by an unknown quantity and would possibly quadruple the workload to about 4-5 visits per health unit per day, or even more. Hopefully, details on this type of information will be provided in the other evaluation components. 4 Same source, p. 14. 35 b. Recommendations HOPE should update its utilization data to show: 1) the number of visits to plantation health units each month by mothers, children under five and others, for migrants and for residents; 2) the number of individuals served by the plantation health units in 2001, broken down the same way; 3) the number of plantation residents and migrants served by Health Center staff during their visits to the plantations; and 4) the number of referrals made to the Health Centers by plantation promoters and midwives. 3. Project Effects on Health Behavior a. Conclusions Preliminary data are now available on some of the key health indicators, such as fully-immunized children and diarrhea cases seeking care (see Table 11), but only for residents. In general, the changes seem modest and fall far short of project objectives. Overall, the project has exceeded EOP targets on only four of the 11 indicators for which there are data. Two of those indicators are not significant improvements (+ 2-4% change). On the positive side, there have been improvements over the baselines in 11 of 13 indicators for which there are both baseline and final data. However, six of those are not significant changes (+2-8%). Compared with the baseline, there have been significant improvements in five indicators: families that have child health cards (+23 percentage points); mothers who are exclusively breastfeeding (+32); children who are eating three or more meals daily (+23); and two of the most important indicators, mothers who have had TT2 vaccinations (+27); and births attended by trained providers (+24). The three that are underlined exceeded end-of-project objectives. There were modest gains in four other indicators. One of the most important indicators (fully￾immunized children) increased from 35 to 43 percent, far below the EOP target of 80 percent. Children immunized against measles increased from 40 to 48 percent; diarrhea cases seeking care increased 6 percentage points; and women not wanting a child in the next two years who were using a modern contraceptive method increased from 21 to 25 percent. There were declines in three of the indicators. Mothers knowing they should get measles vaccinations for their infants at 9 months dropped from 40 to 22 percent; mothers who are giving complementary feeding to their infants went from 85 to 58 percent; and mothers who seek care for children with a cough or difficult breathing dropped 13 points. The primary project indicator (utilization of MCH services by migrants) was not collected because the survey was limited to residents. It had not been possible to collect adequate data on this indicator in the baseline or in a second migrant baseline that was carried out in 1998. 36 Table 11: Project Performance Indicators Indicator EOP Target Baseline Final Final-EOP Final-Base Conf. Int. 1.1 Utilization of MCH services by migrants 2.1(a) Fully immunized children 12-23m 80% 35.1% 43% -37 +7.9 ± 9.2 2.1(b) Measles immunized, 12-23 m. NP 40.4% 47.9% +7.5 ± 9.2 2.2 Mothers know to get measles vaccine @ 9m 50% 39.7% 22.1% -27.9 -17.6 ± 4.9 2.3 Families with child health cards 60% 48.1% 71.2% +11.2 +23.1 ± 5.3 3.1(a) Infants breastfed within 1hr of birth NP NR 62.5% ± 5.7 3.1(b) Exclusive breastfeeding, 0-3.9 m 60% 47.8% 79.2% +19.2 +31.4 ± 11.8 3.2 Complementary feeding, 5-8.9 m =20 pts 85.1% 57.8% +27.3 ± 15.1 3.3 Three or more meals previous day 90% 20.6% 43% -47 +22.4 ± 9.5 3.4 Children 6-23 m receive Vitamin A NP NR 15% ± 5.1 3.5 Children <2 weight for age <-2Z NP NR 4.1 Diarrhea cases (0-5 yr) seeking care 60% 51.8% 57.1% -2.9 +6.7 ± 14.5 4.2 Dehydration cases (0-5 yr) using ORT 30% NR 27% -3 ± 11.6 4.3 Mothers maintain/increase BF during/after diarrhea 60% ~60% 62% +2 ~+2 ± 12.6 4.4 Wash hands with soap/ash before food prep. NP NR 94.6% ± 2.7 5.2 Mothers who recognize signs of pneumonia 40% NR 5.3 Care seeking for cough or difficult breathing 60% 69.9% 56.6% -3.4 -13.3 ± 11.1 5.4 Can name two signs of illness that warrant care NP NR 5.5 Caretakers who sought care for ill child NP NR 5.6 Children <23m who slept under insecticide net NP NR 41.1% ± 5.7 5.6 Caretakers giving same/more liquids during illness NP NR 6.1 Mothers seeking prenatal care 70% 58.8% 6.2(a) Able report = 2 danger signs of pregnancy/PP NP NR 6.2(b) Mothers with TT12 70% 4.1% 30.8% -39.2 +26.7 ± 5.4 6.3 Births attended by trained provider 40% 28.8% 53.8% +3.8 +2.5 ± 5.8 6.4 Mothers with at least one PP visit NP NR 36.4% ± 17.3 7.1 Women who use FP method to space 2 yrs. NP 20.7% 24.5% +3.8 ± 8.5 7.2 Children 0-23m born = 24 mo after last child NP NR 7.3 Able to name = 2 symptoms of STI NP NR 7.4 Able to name = 2 ways to avoid HIV infection NP NR Positive changes in bold. Confidence interval of Final Survey data @ 95%. Significant changes are highlighted. 37 These results are not final and data on 12 other indicators have not yet been processed. However, overall performance is well below expectations and should raise questions about the viability of the project design as well as its implementation. Although there are no data on the prime target group, migrant mothers and their children, achievements are likely to be even lower than these figures for residents. b. Recommendation HOPE should step back and reexamine both the project design and its implementation before expanding to new sites or expanding the service package. The “model” does not appear to be ready for replication as yet and should not be expanded prematurely. 4. Target Groups a. Conclusions The project has had a very difficult time reaching its primary target group, migrant women and their children. It has been difficult even getting data about the health status and needs of this group. The current project strategy is not conducive to serving migrants and, given the recent KPC results, it is unlikely that the project will be able to demonstrate any significant improvement in the health of this group. That raises a fundamental question: should the project keep this as its priority objective? The initial intent of the project was to improve the health of migrant women and their children.5 However, the focus changed early in the project from migrants to residents.6 The initial design included pre-migration preparation and services in the migrants’ home communities. The revised design was based on a traditional community facility-based set of activities. No attempt was made to find ways to follow migrants or in other ways to ensure comprehensive and continuous care throughout the year. The baseline survey was done very early in the project in order to capture information on migrants while they were in the plantations. No attempt was made to collect data at their home communities or anywhere else except at the project plantations. Thus, the baseline collected very little information about migrants. A special migrant baseline was then designed, but again data were only to be collected at the plantations. Therefore, the project waited a year until the next migration cycle to conduct the “migrant baseline survey.” By that time the project implementation plan was already set – as a plantation-based project. Even then data were very hard to collect and only 165 women were interviewed. Because the survey team had so much trouble collecting data from the initial sample, the design was changed to a convenience sample. That is, only those who volunteered were interviewed. This means that the data were not 5 This is still the title of the project: “Improving the Health of Guatemala’s Most Vulnerable Population – Migrant Women and their Children in the Boca Costa of Guatemala,” December 1996. The project application speaks almost exclusively of this as the target group. 6 See the “Detailed Implementation Plan” of April 1998, in which 84 percent of the beneficiaries are residents and were to receive 100 percent of the resources for eight months of each year (January-August) and 30 percent for the remaining four months (the migrants were to receive 70 percent of the resources during this latter period). 38 representative of the migrant population. In addition, the survey pictured the migrants as a very small, very difficult group to reach, culturally as well as physically. The final KPC did not even attempt to collect data on migrants. The project activities emphasized developing capacity to serve residents through volunteer promoters, traditional birth attendants and health units located in or near their communities. Migrants were to be encouraged to use the same facilities when they came to the plantations. No significant activities were undertaken to fit the migrants’ needs outside of these facilities. The project has found it difficult to work with the migrants because of such obstacles as language, culture, limited education, the short period of contact and the lack of time the migrants have after work. The baseline also showed that this is a very small target population (22,000) that is expensive to serve. In fact, since the average period of time in a plantation is only 2 months, the effective exposure is only 3,670 person years (2/12 * 22,000).7 The current model is not appropriate for a migrant population, especially this one. It is not likely to have any significant effect on health knowledge and behavior, even in the long run. There are too many constraints and the cost of overcoming them would be exorbitant. The project has proposed a number of additional activities, but they would not alter the basic model, which is to provide services only during the time that the migrants are working in a plantation. b. Recommendation HOPE should conduct a careful analysis of the current situation and feasible options. The following options have been identified so far: 1. Scale back. Provide access to education and services for migrants when they are in the plantation. Eliminate all other special activities. 2. Tracking. Develop a migrant tracking system so that all migrant women and their children can be located and provided education and services year round. 3. SIAS/MSP. Turn the responsibility for care of the migrants over to SIAS/MSP, which would have the complete responsibility for the migrants, even when they are in the plantations. 4. Combinations. Combine 1 and 4. SIAS/MSP could be responsible for the migrants when they are in their home communities and the project would be responsible when they are in project plantations. Coordination between the project and SIAS/MSP would be needed to ensure continuity of care. 7 Based on DIP estimates. The latest progress report (September 2001, p. 4) estimates the target population to be even smaller (8,366 children < 5 years and 2,608 pregnant women). Effective exposure is only 1,829 person years. 39 5. Interventions a. Conclusions Both the project proposal and the DIP list five health interventions: immunizations, nutrition diarrhea disease control, pneumonia and maternal care. The project performance indicators also conform to this set of interventions. At the time of the mid-term evaluation two other “interventions” were added: access to services and capacity development. A significant change was made in 1999. The third annual report8 identified the project interventions as: ß Training in diarrheal disease control, immunization, pneumonia control, nutrition and maternal care; ß Implementation of minimal health units on the coffee plantations; ß Development of master trainers in the main partner agencies of the project; ß Improvement of supervision systems and logistics; and ß Coordination with radio stations to disseminate health messages in the local Mayan languages. The latest progress report has a similar list.9 During the evaluation, HOPE management stressed that it provides no direct health services and that it’s primary intervention is to facilitate the involvement of the MSP, IGSS and Anacafé, who would provide the direct services. The point was also made that this facilitation model is what it hopes will be replicated elsewhere. AID guidelines require the assessment of the results of each intervention. This shift in the definition of the “interventions” is not only confusing, it also gives the impression that HOPE wants the project to be assessed on the basis of its success in facilitation (e.g., training, coordination, capacity development) instead of its success in improving health and health service utilization. There is no doubt that HOPE’s role is capacity development, as is the case for most development projects. However, capacity development is not an end in itself. Rather it is a means to an end, which in this case is improved health and health services utilization. b. Recommendations The effectiveness of HOPE’s model should be judged in terms of its effectiveness in achieving the health objectives of the project. If the facilitation activities do not improve health and health service utilization, the model cannot claim to be a success and should not be replicated. 6. Design and Implementation Issues As of September 2001 the project has met most of its benchmarks, with the exception of four of the training targets (see Table 12). From comments made to the Evaluation Team during the field visits, it is clear that project staff have been very active and productive. The following issues 8 Improving the Health…Annual Report – October 1999 to December 2000, p. 1. 9 Programa Supervivencia Infantil, Septiembre, 2001, p. 4 40 were raised by members of the Team during the analysis phase and are meant to raise points not made in the analysis of the evaluation questionnaires. a. Health Units Conclusions: HOPE planned to establish 200 plantation health units. This was acknowledged to be an unreasonable number and was reduced to 150. Most of the units had been set up by the second year (see B. of Table 12). This involved working out agreements with the coffee plantation owners and administrators over a period of time, which, undoubtedly, must have been a long, time-consuming process. According to the latest project progress report, the participating plantations agreed to construct a health unit, pay the salaries of promoters, purchase basic equipment for the volunteers; and make time available, largely in the afternoon and at night, for migrants to get health care.10 The Evaluation Team noted that many of the plantations visited had not met all of these requirements. Some plantations constructed new facilities. But from the observations made during the field visits, it seems that many plantations just refurbished an existing health unit or converted a spare room into one. Some of the units are stand-alone structures with separate examining rooms, but many are small, one-room facilities that are poorly equipped. The quality of the facilities varies significantly. Some that were visited are painted, orderly and clean. Others are run down, unkempt and dirty. Most do not have running water or toilets. Most have an examining table and a makeshift screen for privacy, but they are of poor quality. Several team members noted that the scales used for weighing infants were mostly bathroom scales, which are not recommended for weighing infants because they are imprecise and it is difficult to maintain the correct calibration. Recommendation: 1) carry out an inventory and quality assessment of the health units; 2) develop a set of standard requirements for health unit facilities, including furniture and equipment (recommend Salter scales); 3) conduct an analysis of the history of the units (how many have been established, how many have complied with the membership requirements, how many have dropped out and why, what is the average turnover rate and duration of membership?) to determine if there are any significant characteristics that should be taken into account when selecting new project sites. b. Promoters Conclusions. The promoter is essential to the strategy. Without one the health unit is useless and medicines cannot be dispensed. There is no current information as to the number of health units with active, trained promoters. At one meeting three of the eight Plantation administrators said they did not have promoters. Sometimes the promoters leave for better jobs, especially after they have been trained and gained experience, in other cases they tire of the work and resign. According to the benchmark figures (see A.5, Table 12), 150 promoters have been trained in infant health, 150 in IMCI and 650 in IEC. It is likely that at least some of the 150 trained in health care are no longer working as promoters and would need to be replaced. 10 Programa Supervivencia Infantil, Septiembre, 2001, p. 5. 41 Table 12: Project Benchmarks, 1997-2001 Activity/Benchmark Plan Year 1 Year 2 Year 3 Year 4 Total A. Training 1 Training of master trainers 4 2 2 4 2 Training of IMCI and maternal care training teams 1 6 4 11 3 Training of IMCI and maternal care volunteer teams 15 100 150 150 150 4 Training of Health Center providers in IMCI 180 15 26 45 50 136 Training of Health Center providers in IEC 265 25 65 80 68 238 5 Voluntary promoters trained in infant health 250 15 90 20 25 150 Voluntary promoters trained in IEC 250 50 200 150 250 650 6 Traditional midwives trained in maternal care and breastfeeding 250 74 201 250 154 679 7 Mothers groups that meet to be trained in infant health care 200 10 20 20 10 60 B. Health Units Established in Coffee Plantation 1 Coordination agreements with plantations 200-150 15 100 15 20 150 2 Health units established 200-150 15 100 15 20 150 C. Planning, Coordination and Implementation Structure 1 Counterpart agreements 10 9 9 2 Department health councils formed and functioning 3 3 3 3 Department training teams formed and functioning 3 4 4 4 Department plantation (owners and administrators) committees 3 4 4 5 Permanent education commissions 3 4 4 6 Municipal monitoring and supervision plans 20 1 plan applies to 20 districts 1 7 Immunization plans developed 1 standard plan calls for 3 health trips/year to each plantation, including vaccinations 8 Mothers groups formed and meeting monthly 15 100 150 150 150 D. Logistical Support 1 Vaccines available in the Ministry (infant vaccines and TT) (1) MSP MSP Area POA POA 2 Antibiotics available (2) MSP MSP District POA POA 42 Activity/Benchmark Plan Year 1 Year 2 Year 3 Year 4 Total 3 Materials scales 10 90 150 150 150 timers 10 90 208 208 208 E. Activity Information 1 Design and implementation of a tool to collect data about migrants (3) Census design 15 plan. 100 plan Dropped 2 Health Information System (HIS) integrated into MSP at district level 20 dist. 20 20 20 20 20 3 Baseline survey (residents, migrants, health care providers) 1 KPC 97 Migrants 98 4 Training needs assessment 1 Completed for IGSS and San Marcos 5 Family health cards provided by MSP and plantations Mother's card Infant card Child card F. Health Message Broadcasting 1 Development of messages in Spanish and local languages 69 69 69 69 69 2 Broadcasting by radio stations in Spanish and local languages 10 5 5 12 12 12 3 Health units with loudspeakers (for health messages) 15 40 60 60 60 G. Technical Assistance 1 Assistance in Standardized Case Management and maternal care Training, guides, logistics for health staff and volunteers 2 Information systems (3) Source: Project HOPE, October 2001 (1) in the first two years the MSP supplied the vaccines; in the third year they were obtained through a requisition to the Health Area; and in the fourth year the requisitions for the migrant population were included in the Plan of Action (POA) (2) in the first two years the MSP supplied the vaccines; in the third year they were obtained through a requisition to the districts; and in the fourth year the requisitions for the migrant population were included in the Plan of Action (POA) (3) assistance provided at the Area and district levels in registration forms. HOPE does not operate a parallel information system. It uses the official MSP system. The MSP has changed the system 3 times in 4 years, which makes it difficult to have a stable information system. 43 Recommendation: 1) update the database on promoters to determine how many are active, the average turnover rate and length of service, how many have been trained, how many are paid and how much they are paid; 2) examine the characteristics of promoters who resign to identify possible predictive criteria that can be used in the future to select promoters. c. Visits by Health Center Staff to the Plantations Conclusions. The link between health services at the plantation health unit and at the municipal Health Center is a very important element in the strategy because it ensures greater access to trained health providers. Theoretically, the link works both ways: Health Center doctors and nurses go to the plantation to provide services to residents and migrants; and promoters and traditional midwives refer clients to the Health Centers. When this works it is very effective. It has become even more valuable since the project has provided IMCI and maternal care training to Health Center providers. Although vaccinations seem to be the primary reason for the visits, sometimes the visits are more comprehensive, especially if a physician is included. Growth monitoring, examinations for ARI and treatment of minor illnesses are among the services provided in these extended visits. However, this appears to vary from place to place and depends greatly on the composition of the team as well as the motivation of individual team members. There does not seem to be any standard set of health activities for the community visits, with the exception of vaccinations. Some teams give the impression that they are just carrying out a required task. Some of the vaccination teams, in particular, limit their work to screening, vaccinating and recording. Many clients are screened out because they do not have the vaccination cards or their children do not need any more vaccinations. The Evaluation Team observed clients turned away because the time was reserved solely for vaccinations, even though the vaccination team was not busy. During this down time many team members did not take advantage of it to do other things (attend to other clients, conduct a health education session, carry out home visits to find children who need to be immunized or treated, train/supervise the promoter, etc.). Several members of the Evaluation Team commented on the need to “humanize” these sessions. Others commented on the inefficiency of the process, especially the scheduling. Teams often arrive late morning and leave before 5:00 pm, the end of their traditional workday. Others are sensitive to the needs of the migrants, in particular, and stay on through the afternoon and evening to provide services when the clients are available. This unfairly penalizes the providers, because the MSP does not have a flex-time system that allows health staff to adjust their work hours to fit client needs. Recommendations: HOPE needs to develop a strategy for ensuring that the link between the health centers and the health units is ensured and strengthened. A standard set of activities and procedures should be developed to maximize the productivity of each team visit. HOPE should also work with the MSP to set up a flex-time system for health staff who work in coffee plantations where clients are more likely to come for services in the afternoon and evening. d. Transportation Conclusions. The lack of reliable transportation is a critical concern. It affects interaction between the plantation health units and the municipal health centers. It affects the availability of essential drugs. One of the principal factors limiting training and supervision of volunteers by the 44 health center providers is the lack of funds within the MSP for this kind of activity, especially transportation and related costs. Recommendations: The plantations should provide transportation for the health center providers to get to the health units to provide services as well as to supervise and train the volunteer promoters and midwives. Funds should also be sought from the municipal governments under the new decentralization program so that the health staff can fulfill their obligations to protect community health. e. Communication for Behavior Change Conclusions. There are some doubts that the BCC/IEC campaign is having any effect on the target groups. The 69 messages that have been developed would seem to be too short and simplistic to have much effect on behavior, especially over radio and broadcast from the project vehicle. One radio station that broadcasts the project messages claims to cover 60 percent of the population in its Department. However, a project study that included a question about where the target groups heard certain project messages identified six stations, none of which was the station in question. At each plantation visited the project vehicle was parked in front of the health unit broadcasting messages over its loudspeakers. It would seem to be more effective to broadcast the messages near the galeras and other spots where workers and their families congregate. The education activities could have more impact if they took the expectations of the residents and migrants into account. One mother mentioned that she liked to watch films. Another approach could be cooking demonstrations of healthy recipes that use local ingredients. Staff could then monitor cooking practices to see if the demonstrations had results. There does not seem to be any special emphasis on adolescents and youth. Family planning (or birth spacing), which is clearly needed, is not promoted. Contraceptives are only available in some health units, especially where APROFAM has established outlets (some project promoters are also APROFAM promoters). Recommendations: HOPE should take a close look at the BCC/IEC strategy to see if it is having any effect. BCC/IEC strategies should be adapted to fit the expectations and information needs of the target populations. Adolescents and youth should be priority targets, especially among the migrant populations. Family planning should be included in the BCC/IEC strategy. The project should work more closely with APROFAM to make contraceptive information and services available in the health units. F-1 F. ATTACHMENTS 1. Qualitative Evaluation Participants JHPIEGO Dr. Gustavo Barrios Licda. Alicia Ruano Licda. Demitrio Margus San Marcos Health Area Dra. Elisa Barrios Enf. Vilma Veláquez Enf. Adriana Castillo de Méndez Enf. Galvez Reyes Suchitepéquez Health Area Dra. Miriam Pastor Enf. Jovita Morales R. IGSS Dr. Renato Umaña Enf. Juanita Xuruc HOPE Central Dr. Luis Benavente Dr. Bettina Schwethelm Dr. Jack Reynolds (consultant) HOPE Guatemala Dr. Victor Calderón Dra. Anabela Aragón Dr. Ronald Alvarado Dr. Edgar López Enf. Brenda Elizabeth Yes Orozco Licda. Karina Mancroz Licda. Gabriela Peldéz F-2 2. Field Work Schedule Date Team Location Activity 1 Oct/Mon HOPE office Project overview Review of studies Review IEC materials Review/revise evaluation questionnaires 2 Oct/Tues HOPE office Review/revise evaluation questionnaires Review/revise field work schedule, select teams (A and B) 3 Oct/Wed A San Marcos San Rafael Malacatán Interviews: Health Area Team, Promotion, Radio, Finca Merceditas: owner, promoter, health unit B San Marcos San Pablo Malacatán Interviews: Board/Health Council, Master Trainers, Radio, Finca Buena Vista: vaccination session, promoter, mothers, administrator 4 Oct/Thu A Malacatán Siloe & La Montañita Com. Health Center; midwife training; IEC event; interviews: PRS, mothers, Finca San Luis: interviews, galeras B San Pablo El Tumbador Malacatán Health Center: staff, facilitator, MEC activity, finca promoters, owners/administrator meeting Finca San Luis; medicine shop; Finca El Edén, interviews: administrator, promoters, providers, mothers 5 Oct/Fri A El Rodeo Health Center staff, IEC training, MEC activity, promoter supervision, facilitator meeting, Finca San Jerónimo, Finca Pomarosa: vaccination session, prenatal session, health unit, interviews: owner, administrator, promoter B El Quetzal Health Center; midwife training, administrator meeting, HC staff meeting, Finca Oná, Finca Chiquilá: vaccination, prenatal, morbidity sessions, interviews: promoters, administrators, mothers, promoters, visit galera 6 Oct/Sat A Colomba, Rio Negro com., Santa Eulalia Medicine shop, mothers training session, health guides training; Finca Victoria Chuvá, Finca Batavia: morbidity session, migrant mothers, galeras, administrator, promoters B Chicutzan, Chicacao Mothers IEC session, Finca El Refugio, Finca Las Camilias; Finca Valle de Oro, Finca La Vega: vaccination, health units, interviews: administrators, health unit staff, promoters, owner, mothers, visit galeras 8 Oct/Mon A Colomba Coatepeque, Miramar HC staff, administrator meeting, MEC training, Sta Rosa: sala situacional presentation, Finca San Carlos, Finca San Francisco: morbidity session, prenatal, vaccination, interviews: promoters, providers, mothers, administrators B Quetzaltenango, Matatenango, Sta Barbara IGSS staff, facilitator meeting, MEC and maternal care, IEC promoters; Finca Panamá, Finca Mocá: galeras, morbidity session, prenatal, vaccination: interviews: administrators, F-3 promoters, providers, mothers C HOPE office Compile questionnaire data 9 Oct/Tue A Flores, Colomba, Coatepeque, Flores, Génova HC staff, interviews: facilitators; maternal care session, midwives; promoter training in IEC, Nueva Esperanza community: IEC for mothers, Las Mercedes: medicine shop; Finca Morelia: Finca San Antonio Morazán: vaccination, prenatal, galeras, interviews: administrators, trainers B Suchitepéquz, Mazatenango, Zapotitlán Area staff, MEC-AEIPI training for providers, radio station, Health Post Ssan Francisco: midwife training; Finca Margaritas; Finca La Providencia: vaccination, prenatal, interviews: owners, promoters, galeras C HOPE office Compile questionnaire data 10 Oct/Wed 1,2,3 HOPE office Review compiled data, summarize findings, conclusions and recommendations (48 questions) 11 Oct/Thu 1,2,3 HOPE office Review summaries; Group meeting and discussion; prepare final versions of summaries; collect available KPC and benchmark data; core team to Guatemala 12 Oct/Fri Guatemala Meeting with MSP/AIPI re project extension; briefing at USAID re KPC and Qualitative Eval findings; develop plan for evaluation reports (KPC, qualitative, processes). 13 Oct/Sat Guatemala AA returns to Quetzaltenango; VC/LB/BS to Millford; JR to Houston-Honolulu 15-21 Oct Honolulu Consultant prepares report 22 Oct Honolulu Draft report submitted to HOPE central F-4 3. Documents Improving the Health of Guatemala’s Most Vulnerable Population – Migrant Women and Their Children in the Boca Costa of Guatemala, Project HOPE/Guatemala 1. Proposal, December 13, 1996 2. Detailed Implementation Plan, April 1998 3. Annual Report (Summary), October 1997-December 1998, January 1999 4. Annual Report (Full report), October 1997-December 1998, January 1999 (Appendixes include the baseline questionnaire, sample, tabulations, training needs assessment and training plans, among other relevant documents). 5. Baseline Survey: Knowledge, Practices and Coverage of the Migrant Population in the Intervention Areas, November 1998 6. Epidemiological study of migrant mothers and residents with children less than five years and focus groups for the evaluation of the diffusion of basic health messages, October/November 1999 7. Midterm Evaluation, December 30, 1999 8. Mini-survey of migrant mothers’ knowledge of child survival components, (undated), 2000. 9. Extension Proposal, December 20, 2000 10. Annual Report, October 1999-2000, February 21, 2001 11. Annual Report, October 1999-December 2000, March 2001 12. Trimester Report (untitled), September 2001