Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti GLOBAL HEALTH ACTION Healthy Mothers, Healthy Children: A Child Survival Initiative in Petit-Goâve, Haiti Final Evaluation Report (November-December 2010) CA #: GHS-A-00-04-00021-00 October 1, 2004 – September 30, 2010 External Evaluator: Erlantz Hyppolite M.D. Submitted to USAID Date of Submission: December 24, 2010 Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 2 Evaluation Report drafted and finalized by Erlantz Hyppolite M.D. Report reviewed by Margaret McCann PhD, Deputy Editor, Epidemiology Starry Sprenkle, M.S., Program Manager, Albert Schweitzer Hospital, Haiti Girija Sankar, M.S., Program Manager, Global Health Action Robin C. Davis, R.N., M.N., Executive Director, Global Health Action Final Evaluation Report Prepared For: Global Health Action P.O. Box 15086 Atlanta, GA 30333 USA & 1902 Clairmont Road, Decatur, Georgia 30033, USA This Final Evaluation report has been made possible through support provided by the U.S. Agency for International Development, under the terms of the USAID Cooperative Agreement # GHS-A-00-04-00021-00 between USAID and Global Health Action, Inc., for the Child Survival Project in Haiti. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 3 TABLE OF CONTENTS FIGURES & TABLES ……………………………............................................... 4 ACRONYMNS……………………………………………………………….. 5 ACKNOWLEDGEMENTS………………………………………………….… 6 A. PRELIMINARY INFORMATION Executive Summary………………………………………………………. 7 B. OVERVIEW OF THE PROJECT Project Location and Population…………………………………………… 9 Project Design………………………………………….................................... 11 Partnerships & Mission Collaborations……………………………………… 15 Project Implementation Summary…………………………………………… 16 C. PRESENTATION OF PROJECT RESULTS Quantitative Indicators Review……………………............................................ 21 Data Quality: Strengths and Limitations…………………………………… 27 Presentation of Project Results based on Qualitative Evaluation…………… 28 D. DISCUSSION OF RESULTS……………………………………………… 37 E. CONCLUSIONS & RECOMMENDATIONS……………………………. 40 F. REFERENCES……………………………………………………………… 43 G. ANNEXES Annex 1. Results Highlights…………………………………………………… 44 Annex 2. List of Publications & Presentations related to the project…………… 45 Annex 3. Project Management Evaluation…………………………………….. 46 Annex 4. Workplan table……………………………………………………… 47 Annex 5. Rapid CATCH Table………………………………………………… 49 Annex 6. Final KPC Report…………………………………………………… 49 Annex 7. CHW Training Matrix………………………………………………… 50 Annex 8. Evaluation Team Members and Titles……………………………….. 52 Annex 9. Evaluation Assessment Methodology……………………………….. 53 Annex 10. List of Persons Interviewed and Contacted………………………… 54 Annex 11. Special Reports Annex 11A. Note on Project Inventory……………………………………. 55 Annex 11B. Other Pertinent Information related to the Final Evaluation….. 56 Annex 12. Project Data Form……………………………………………….… 65 Annex 13.Grantee Plans to Address Final Evaluation Findings………………… 72 Annex 14.Grantee Response to Final Evaluation Findings……………………… 74 Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 4 FIGURES & TABLES Item Title Page Figure 1. Map of the Commune of Petit-Goâve 10 Table 1. Population by Section and Setting in Petit-Goâve 11 Table 2. Health Facilities of Local Partners in Petit-Goâve 12 Table 3. Specific indicators for outcomes under Objective 1 22 Table 4. Specific indicators for outcomes under Objective 2 24 Table 5. Specific indicators for outcomes under Objective 3 24 Table 6. Specific indicators for outcomes under Objective 4 25 Table 7. Specific indicators for cross-cutting activities that benefitted multiple project objectives 26 Table 8. Distribution of focus group participants 29 Table 9. Theme One: group constitution and awareness of the project 30 Table 10. Theme Two: Training 31 Table 11. Theme Three: Service Delivery 32 Table 12. Responses on Immunization and Maternal Mortality 33 Table 13. CHW/TBA Responses on training, & maternal health practices 34 Table 14. Services offered by TBAs 34 Table 15. CHW services and outreach efforts 35 Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 5 ACRONYMS AIDS Acquired Immune Deficiency Syndrome CHW Community Health Worker COD-EMH Coordination Office for Development￾Methodist Church of Haiti CS Child Survival DIP Detailed Implementation Plan GHA Global Health Action HFA Health Facility Assessment HIV Human Immuno deficiency Virus HQ Headquarters IHSI Institut Haïtien de Statistique et D’informatique (Haitian Institute for Statistics and Informatics) KPC Knowledge, Practice, Coverage (Survey) MCH Maternal and Child Health M&E Monitoring and Evaluation MOH Ministry of Health (MSPP in French) MSPP MSPP or Ministère de la Santé et de la Population (MOH in English) PF Planning Familiale (FP or Family Planning ) PM Program Manager PMTCT Prevention of Mother-to-Child Transmission of HIV PRR Projet de Réhabilitation Rurale (Rural Rehabilitation Project of the COD-EMH) RA Rapid Assessment STI Sexually Transmitted Infection TBA Traditional Birth Attendant USAID United States Agency for International Development Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 6 ACKNOWLEDGEMENTS The author wishes to recognize the efforts of Global Health Action and its partners (Coordination Office for Development of the Methodist Church of Haiti, and the Ministry of Health/Unité Communale de Santé) and those of the community health workers for their participation in the planning, implementation, and supervision of project activities that have contributed to improved delivery of health services in Petit-Goâve, Haiti. The author expresses sincere appreciation for the cooperation and support received from GHA field office in Haiti and would like to thank Rachelle Etienne GHA’s Field Office Coordinator and Schmidt Brewster Desir, GHA’s field mobilizer. The author would also like to give a special thanks to Ms.Mirelaine Deshormeau, Head Nurse-Administrator at the Olivier Clinic, for her irreplaceable help with sharing project reports and vital strategic planning information. A very special thank you also goes to Ms. Girija Sankar, Program Manager, GHA-HQ, for making the trip all the way to Haiti, helping in the organization of the focus groups, and sharing major and vital information that helped in the final evaluation writing. The attention of the author turns to Mrs. Robin Davis, GHA’s Executive Director for her advice and for being available all along the report-writing process. In addition, the author extends his appreciation to the focus group participants for their time and for the honesty of their response, and hopes that this document captures the true meaning of their views and feedback. A very special thanks goes to Ms. Margaret McCann and Ms. Starry Sprenkle for reviewing the document. The author would like to mention at the outset that due to various difficulties that will be presented in the document, the USAID’s “Guidelines for Modified Final Evaluation Report” could not be respected in all is specificities. Due to the lack of information on some project activities over the years the author was compelled to produce a narrative document of the project implementation history, perform focus groups, and present modified tables. No formal final results could be presented due to the lack of data and no final KPC (same survey that set baseline data) could be conducted due to the modified circumstances caused by the earthquake of January 12, 2010. The modified evaluation activities and requirements were part of USAID’s modified operations and reporting guidelines (For Projects Where Activities Were Significantly Disrupted). Petit-Goâve, the Project location, was at the epicenter of the January 12, 2010 earthquake. The evaluator takes this opportunity to wish success to all those who are participating directly and indirectly in the planning and implementation of the project activities for GHA. Even though Healthy Mothers, Healthy Children reached its full-term, GHA is still working in Petit-Goâve to improve the health system in these difficult times that have further weakened the health system and increased demand for quality projects and programs. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 7 A. Preliminary Information Executive Summary Haiti is the poorest country in the Western Hemisphere, and has the worst maternal and child health indicators in the Caribbean. There is also a very low utilization (less than 30%) of available health services, especially by the poor majority. This low utilization is a symptom not only of poverty but also complex social and educational problems that the population faces, as well as inadequate infrastructure and transportation. Finally, no health services are available at all in many remote, rural areas of Haiti. It is within this context that (Mere an Sante, timoun an Sante) Healthy Mothers, Healthy Children, a five-year Child Survival Project (09/01/04 – 09/30/09, with a no-cost extension till 09/30/10) was implemented in Petit-Goâve. The project was implemented by Global Health Action (GHA), a non-profit international health organization with a registered field office in Haiti, and its local partners: the Bureau for Coordination & Development of the Methodist Church of Haiti (COD-EMH) and the Ministry of Health/Unite Communal de Santé (MOH/UCS). Together they worked to improve the health and well-being of women in childbearing age and their young children. Project Goals and Objectives The project’s goal was to contribute to a sustainable reduction of maternal and infant mortality in the Petit-Goâve region of Haiti. Broadly, the project undertook to improve the availability and quality of key child survival and maternal and newborn health services; and educate and mobilize the target population to increase the demand for and the utilization of these services. The overall project objectives were: 1. Seventy-five (75) percent of mothers of children aged 0-23 months in the Petit-Goâve region of Haiti will have received/utilized high quality pre- and post-natal/infant services during their most recent pregnancy by September 2009. 2. By September 2007, 75% of persons of reproductive age in Petit-Goâve will be correctly informed about the prevention of sexually transmitted infections (STI), their impact on the outcome of pregnancy, and the availability of treatment in the UCS. 3. By September 2006, 40% of men of reproductive age in the district of Petit-Goâve will have completed a “Responsible Fatherhood” course. 4. Establish a local network of referral services that provide quality maternal and neonatal care in Petit-Goâve region by September 2009. Activities GHA and its partners aimed to reach these goals and objectives by putting in place a system that could facilitate certain key interventions that have direct impact on the project objectives. Interventions were in the fields of: Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 8  Maternal and newborn care  Immunization coverage for pregnant women and women in reproduction age.  Breastfeeding promotion/children spacing family planning (PF)  HIV/AIDS and STIs Key Outcomes The project objectives were more quantitative than the status of the monitoring systems implemented by MOH or GHA will allow us to determine definitively. That is why this evaluation gathers qualitative evidence from focus groups and interviews. Considering the range of data available, here is what can be said about project achievements: Objective 1: Maternal and Child Health: Maternal and child health generally improved during the project, although most deliveries still occur at home with traditional birth attendants (TBAs) and maternal mortality is still a concern. Immunization levels improved due to the project. Objective 2: STI Awareness and Treatment: Laboratory and treatment capacity have improved, significant community education efforts took place, and prevention of mother-to child transmission (PMTCT) services are offered throughout the area. Objective 3: Responsible Fatherhood: Father’s groups were formed and educated, and numerous and varied community education activities took place Objective 4: Network and Referrals: A communication system and medical protocols have been established, and the staff has received management/leadership training. But, the database for referral and clinical registers are not organized or implemented to date. Challenges During implementation, numerous external challenges were posed by weather/natural disasters, instability of partner organizations, political instability/insecurity, insufficient integration of and by the MOH, and frequent personnel changes, all of which delayed or prevented some project activities. Moreover, GHA was not able to implement monitoring and evaluation tools due to the weakness of partner organizations and GHA’s own lack of supervisory/administrative presence in Petit-Goâve. Recommendations GHA must improve monitoring and evaluation tools and supervision. It should strengthen its administrative structure in Haiti both in Port-au-Prince and farther afield, in order to provide the necessary structure for supervision and follow-through. Conclusion Despite the challenges faced and some shortcomings, the project was useful and saved lives. It was apparent from the evaluation that GHA provided much-needed community-based health training, education, and mobilization, along with resources, that were not present in Petit-Goâve prior to the project’s inception. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 9 B. Overview of the Project Project Location and Population Haiti is the poorest country in the Western Hemisphere, and has the worst maternal and child health indicators in the Caribbean. Available estimates of health statistics indicate that the under-five mortality rates are at 120/1000 live births (12%) and maternal mortality ratio is the highest in the Caribbean at 624/100000 (0.6%) live births (Emmus IV 2005-2006). In addition, the utilization of available health services, especially by the poor majority, is estimated at a mere 28.5%. This low utilization is caused by a number of factors. It is a symptom not only of poverty but also complex social and educational problems that the population faces, including a lack of basic public health education provision by the government, and traditional beliefs in spiritual, non-medical treatment of disease. Furthermore, the country’s lack of infrastructure (i.e. paved roads) and means of transportation (few Haitians own a motorized vehicle, and mass transport is run by private entrepreneurs) make it difficult for people to access the existing health centers. Finally, there is a complete lack of services available in many areas. It is within that context that Healthy Mothers, Healthy Children, a five-year Child Survival Project (09/01/04 – 09/30/10) was implemented in Petit-Goâve, one of the poorest and most mountainous coastal areas of Haiti’s Ouest (Western) department. The project was implemented by Gobal Health Action (GHA), a non-profit international health organization with a registered field office in Haiti, and its local partners: the Bureau for Coordination & Development of the Methodist Church of Haiti (COD-EMH) and the Ministry of Health/Unite Communal de Sante (MOH/UCS). Together they worked to improve the health and wellbeing of women in childbearing age and their children with an emphasis on infants aged 0 – 23 months. To understand better the project context, strategies, and achievements it is imperative that we present the Petit-Goâve region. The commune of Petit-Goâve is located sixty-eight kilometers (68km) south-west of Haiti’s capital, Port–au–Prince, located in the same department as the capital (Ouest). It is a mostly mountainous coastal region divided into twelve (12) communal sections which are further subdivided into five hundred and seventy (570) localities (communities) and approximately 158 landholdings. A detailed breakdown of the sections, landholdings, and communities is given in Table 1 of Appendix 1. The coastal town of Petit-Goâve is the administrative center of the region. A map of the sections, their populations, and health worker coverage is on the following page. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 10 Figure 1. Map of Petit Goâve commune divided by communal section, with population and health worker coverage for each section. A total of 19 trained CHWs were dispatched in all the sections by the project. A total of 8 health centers are distributed in the northern half of the district. The southern half is served only by CHWs and mobile clinics. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 11 According to the National Haitian Institute of Statistics (IHSI), the region of Petit-Goâve has an estimated population of 143,191 of which 11% ( 16,373) are under five years of age and an estimated 27% ( 38,742) are women of childbearing age. Table 2 in Annex 11B describes the age structure of the population, with age groups divided in five year increments and detailed by sex. Twenty-two percent (22%) (32,748) of the population live in the urban area and 77% (110,443). in the rural areas. Table 1 below gives the population breakdown by rural section and type of living environment (urban or rural). Table 1. Population by section and setting in Petit-Goâve. Rural section Total Setting Urban Rural 1st section Bino 20208 5547 14661 2nd section Delatre 14703 - 14703 3rd section Trou Chouchou 7349 - 7349 4th section Fond-Arabie 6639 - 6639 5th section Trou-Canari 2392 - 2392 6th section Trou-Canari 9500 - 9500 7th section des Platons 3181 - 3181 8th section des Platons 18426 - 18426 9th section des Palmes 7801 - 7801 10th section des Palmes 4810 - 4810 11th section Ravine Seche 35276 23503 11773 12th section des Forques 12906 3698 9208 Summary 143191 32748 110443 Source: Direction des Statistiques Démographiques et Sociales, IHSI- 2005 The health system in Haiti breaks the country into geographic regions called UCS or the Unité Communale de Santé. The commune of Petit-Goâve is part of a UCS named ‘Goavienne,’ which groups Petit-Goâve along with the neighboring commune of Grand Goâve. According to the 2009 IHSI and MOH data, twenty-seven (27) health establishments were identified both in the city and in the rural sections. The referral center for the UCS, Hospital of Notre Dame, is located in the city of Petit-Goâve. Project Design The Healthy Mothers, Healthy Children is a five-year child survival project (09/01/04 – 09/30/09, with a one-year no-cost extension) supported by USAID and implemented by GHA in close partnership with the Bureau for Coordination & Development of the Methodist Church of Haiti (COD-EMH) and the Ministry of Health/Unite Communal de Sante (MOH/UCS). Health statistics indicate that the major public health problems that contribute to high mortality rates among children and women of reproductive age include: malaria, diarrhea, acute respiratory infections (tuberculosis and pneumonia), measles and childhood communicable diseases, HIV/AIDS/STIs, and pregnancy complications. Healthy Mothers, Healthy Children’s overall strategy was to lower the rates of maternal and child mortality by working both at the community and the facility level to reduce the severity and frequency of those factors. For coverage and sustainability purposes the project planned to Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 12 work with eight health facilities in the region (seven clinics and one hospital). Six of the facilities are managed by the MOH through the UCS office and the two remaining ones are operated by the COD-EMH. These facilities are listed below. Table 2. Health Facilities of Local Partners in Petit-Goâve No. Facility Affiliation 1. Notre Dame Hospital MOH 2. Arnoux Health Center MOH 3. Vialet Health Center MOH 4. Madeliene Health Center MOH 5. Vallue Health Center MOH 6. Cadet Trou Chou Chou MOH 7. Olivier Health Center COD-EMH 8. Platon Trou Chou Chou COD-EMH Note: Ministry of Health [MOH]; Methodist Church [COD-EMH] The project strategy was to make available basic infant/neo-natal and maternal health services for the targeted populations in facilities that were managed by two major health providers of the region-the UCS and the COD-EMH. The provision of key services for women of childbearing age and children under five at the clinics was one step towards the project’s goals and objectives, but not enough to cover the desired fraction of the population because a considerable part of the population would not be able to access those services owing to difficulties in transportation. This was why a second, critical part of the project strategy was to work at the community level with a network of trained community health workers (CHW’s), traditional birth attendants (TBA’s), and community leaders to help project staff implement outreach activities such as mobile clinics and community-based health education. According to the project goals and objectives specific interventions were planned throughout the five years of implementation to facilitate the establishment of the project. Objective 1- By September 2009, 75% of mothers in the district of Petit-Goâve, having a child 0 to 24months, will have benefited from adequate pre- and post-natal care during their last pregnancy. One of the major difficulties impairing this objective is that pregnant women rarely seek pre￾natal care in Haiti, and when they do, it is usually very late in the pregnancy. Therefore, education and mobilization were critical steps in meeting this objective. The project proposed as a specific intervention to train and mobilize CHWs to educate women regarding the importance of early pre-natal check-ups at regular intervals, and referring to MOH guidelines that require every pregnant woman to have three pre-natal check-ups. This would help to increase health-seeking behavior among all women in the district. GHA with its partners did not stop at encouraging pregnant women to access services, but also gave women the opportunity to access pre-natal care closer to their homes through mobile clinics. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 13 The project hoped to have the following impacts in terms of increased pre-natal care:  Increase the number of pregnant women living in Petit-Goâve region who have visited a clinic and who hold an official MOH maternal health card.  Increase the number of pregnant women receiving a first and a second pre-natal check￾up before the end of their sixth month of pregnancy, and the number of pregnant women having their third pre-natal check-up before the end of the eighth month of pregnancy.  Increase the number of pregnant women receiving two doses of tetanus toxoid vaccine before their ninth month of pregnancy. The MOH guidelines also recommended that every mother is to be tested for HIV and Syphilis. Project staff were responsible for identifying HIV positive mothers and referring them to appropriate medical facilities for follow-up, not only at the facility level but also from mobile clinic visits. TBAs were to be trained and supervised for adequate delivery practices, and were to inform the local CHWs of all births. The CHWs in turn were to keep a register of pregnant women in their area and were to be trained to assess the new mother’s health status during early home post-natal visits which occurred no less than 72 hours after delivery. Auxiliary nurses were to be trained for this service, as well as in communication skills at the health facility level to aid new mothers. Local women’s groups were to receive incentives for referrals and facilitating post-natal visits of mothers. As an incentive, mothers completing pre-natal visits would receive a newborn kit (containing hygiene products and some baby clothing), while mothers completing the first post-natal visit would receive a “winning mother” kit (containing towel, soap). The project worked towards the following post-natal outcomes:  Increase the number of births taking place in the presence of a TBA.  Increase the number of mothers receiving post-natal visits at home by trained CHWs within 72 hours of delivery.  Increase the number of new mothers presenting for a post-natal check-up at a health facility in their area within five days of delivery.  Increase the number of infants who begin breastfeeding immediately after delivery. Local women and rural groups in the district were to be educated and encouraged to plan their families with a two-year period between pregnancies. CHWs were to be trained to educate new mothers on the benefits of adequate child spacing. These interventions had the goal of increasing the spacing between pregnancies. As mentioned before, Healthy Mothers, Healthy Children particularly focused on children up to 23 months of age. A set of outreach strategies (mobile clinics, rally posts for weighing and immunization, home visits by CHWs, and distribution of Vitamin A, in keeping with the MOH guidelines) were to be put in place in communities in order to increase coverage of immunization for the this age-group of children, and also to ensure proper follow-up of their nutritional statuses. The health worker network, mother’s groups, and the father’s groups would all be involved in this effort. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 14 Objective 2- By September 2007, 75% of persons of reproductive age in Petit-Goâve will be correctly informed about the prevention of sexually transmitted infections (STI), their impact on the outcome of pregnancy and the availability of treatment in the UCS. STI knowledge and prevention was to be promoted through trainings of local women’s groups, outreach efforts by CHWs, the initiation of ‘Responsible Fatherhood’ Clubs, health fairs and weekly radio announcements. These interventions were to be continuous and would increase awareness amongst all people of reproductive age about STIs and their impact on pregnancy. COD-EMH would also seek to adequately manage patients who come to the dispensaries with STIs. Most clinics could run tests for most STIs, and if necessary they would provide a referral to a facility that had the proper capacity (i.e. the hospital). The goal was to make available essential laboratory equipment and pharmaceuticals for STIs at UCS and COD dispensaries. These efforts were expected to achieve the following results:  Members of community groups, youth and literacy centers are able to identify symptoms, methods of prevention, and understand the impact of STIs on pregnancy and infant health.  The health facilities staff will have participated in training sessions on the MOH norms and protocols for diagnosis, counseling and treatment of STIs, and will have adequate equipment and facilities to treat them. Objective 3- By September 2006, 40% of men of reproductive age in the district of Petit-Goâve will have completed a “Responsible Fatherhood” course. Gender inequalities were to be addressed to change attitudes and behavior in families and at the district level for women’s and children’s health. To do this, an education module for “Responsible Fatherhood” was to be developed in Creole by COD-EMH. The module would include the importance of pre-natal and post-natal care, breastfeeding, immunization, paternal displays of affection and guidance. A series of educational radio spots were to be aired to promote the concept of “Responsible Fatherhood”, and “Responsible Fatherhood” Clubs would also be established. The project sought to reach not only the male partners of women’s club members, but also provide similar educational sessions for secondary school students and youth clubs in Petit-Goâve. The ultimate goal of these efforts was to foster a sense of responsibility amongst males in the project area for maternal and child health and also mitigate fathers’ neglect towards their young children. This would leave both mothers and children less vulnerable to malnutrition and behaviors that are detrimental to health. Objective 4- By September 2008, a local referral network for quality maternal care will have been established. As part of management capacity building, GHA would assist the UCS to design a maternal health referral plan for obstetrical complications and emergencies. Standards for reporting and care were to be established at every level: dispensary, district, and hospital. In Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 15 collaboration with MOH, key staff in each dispensary were to be trained for the adequate management and/or referral of emergency obstetrical/neonatal cases. Mobile/Radio telephones and an ambulance were to be acquired for use by five dispensaries. This system would also facilitate supervision and management of the project. These interventions were to establish a sustainable basis for the continuation of quality care across the district and referral between the health facilities. It would foster trust at the community level in the various health facilities, improve access to quality care and decrease deaths due to major obstetrical problems. Putting in place a successful CS project requires effective integration of interventions that have a direct impact on key CS indicators. This is why GHA and its partners chose to design a variety of interventions addressing different levels of the community. Bringing education to key actors (CHWs, active community members) who interact daily with the targeted population would lead to behavior change that would then positively impact the project indicators. This included working with the mothers' clubs and fathers clubs to train them on appropriate health-seeking practices in time of pregnancy, the importance of pre- and post￾natal care, the importance of immunization, and the importance of breastfeeding. As part of this core strategy project partners also sought to put in place a system that would enable people to practice those very behaviors on which they received training. Sustainability was also the motivating factor for GHA’s decision to work with two of the biggest health providers in the region: the MOH through the UCS and the COD-EMH. The project maintained regular communication with the Haiti mission. Over the years, GHA has approached the technical advisor for facilitating meetings; participation in another CS project final evaluation in Haiti. Partnerships Please refer to section titled “Discussion of Results” for a detailed discussion on partnerships. Mission Collaborations In year five, the GHA Field Office Coordinator participated in evaluation meetings sponsored by the USAID/Haiti mission for the final project evaluation of the Haitian Health Foundation. This meeting brought together recipients of CSHGP funding to discuss successes and challenges. Réginalde Massé of USAID/Haiti facilitated GHA’s staff involvement in the meeting. GHA field staff participated in three quarterly meetings organized by the USAID/Haiti mission throughout years three and four. These meetings brought together recipients of CSHGP funding from throughout Haiti to discuss successes and challenges. The project met the USAID Haiti Mission’s overall health objectives as well as their specific objectives regarding immunization rates, prenatal visits, family planning and breastfeeding. The project’s interventions - immunization of pregnant women and women of reproductive age, promotion of breastfeeding and maternal and newborn care - target Haiti’s most vulnerable groups: women and children. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 16 Project Implementation Summary Year One: October 1, 2004 to September 30, 2005 Work in year one included networking with various stakeholders, assessing the availability of resources, and introducing the Healthy Mothers, Healthy Children project to the beneficiaries of the project. Despite the political instability at the time, GHA was able to hire a Program Manager to work closely with the MOH/UCS and COD-EMH to reach the project goals and objectives. The main accomplishments of year one were:  Conducting a health facility assessment (HFA) of the COD-EMH /MOH-UCS.  Assessing the resources in place in terms of staff and materials.  Identifying the needs of the staff for training.  Knowledge practice coverage (KPC) survey to set baseline for indicators.  Conducting the detailed implementation plan (DIP) workshop and the DIP assessment. One important observation of GHA staff in year one was that, even though the project was supposed to work with the eight clinics listed in the project design section, it would be very difficult to involve all of these clinics initially. This was primarily due to the degree of political instability at the time, which rendered the MOH facilities under-funded and under￾staffed. They could not be fully functioning partners, and the resources that would be necessary to keep them operational were not planned in the project’s budget. Healthy Mothers, Healthy Children had no other choice in terms of working with fixed points, and to focus this effort toward the two Methodist clinics that had better support at the time: Olivier Health Center where a full-time doctor had been put in place and Siloe (which was not on the list originally, but was added later, given the circumstances). Extensive outreach activities (more mobile clinics, more rally posts) were to be planned for coverage purposes in the other communities where the project could not guarantee a functioning fixed point clinic. For instance, the Platon Trou Chou Chou dispensary (managed by COD-EMH) which used to be open once a week was now functional three times a week (Monday through Wednesday); two auxiliary nurses provided their services instead of one. This is particularly important given that this dispensary is in an extremely remote area of Petit-Goâve, accessible by foot and located three hours from the community of Vialet. At the community level the CHWs were identified and their needs for materials and training assessed. In year one there were six CHWs in the project region, one in Platon Trou Chou Chou, one in Cadet Trou Chou Chou, one in Petit-Goâve , two in Arnous and one CHW attached to the Olivier clinic. Plans to identify more CHWs and train new ones were also made. Eighteen (18) TBAs who worked for the COD clinic of Olivier were also identified. No other TBAs worked with any of the other project clinics. The Project Manager discussed the project with the TBAs, since GHA/COD-EMH and MOH planned to train the TBAs in Year 2. Staff Training Needs: The Field Program Manager (PM/field) was able to assess staff training needs by direct observation of their skills during patients’ visits. Staff had also expressed interest in STI diagnosis and treatment. Besides this training, GHA, COD-EMH Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 17 and the MOH planned trainings on the implementation of Maternal and Child Health (MCH) protocols. The COD-EMH staff unanimously showed interest in GHA’s leadership and management training which was planned for year two. A baseline KPC survey was conducted in 2005. The results of this survey were used to establish baseline indicators that were to be formally monitored starting the second year of the project. For this purpose the GHA Headquarters Program Manager (PM/HQ), PM/field, COD-EMH Medical Director –Dr. Paul, Dr. Salomon (full time at Olivier Health Center), Dr. Toussaint (Medical Director of the Notre Dame Hospital) and the statistician worked together to create and implement a form based on the MOH monthly reporting form that will enable them to collect data concerning the project’s fieldwork. The results of the KPC survey also helped with the elaboration of the DIP in a more comprehensive way. Year Two October 1, 2005 to September 30, 2006 Year two was also marked by a series of social and political instabilities that had direct impact on the project implementation because of the strong links of the project to the Ministry of Health (MOH) which was directly affected by the situation. Despite being directly and indirectly affected by this unrest and lack of security, the project was able to reach several of its targets during the second year. The following are the five major accomplishments for year two of the project:  Qualitative Rapid Assessment  Distribution of medical supplies and instruments to local hospital and clinics  Development of Healthy Mothers, Healthy Children curriculum  Pilot test of Healthy Mothers Healthy Children curriculum  Training of Community Health Workers The qualitative rapid assessment (RA) was conducted in October 2005, the results of which helped develop the Healthy Mothers, Healthy Children curriculum as well as in the planning and implementation of project activities which aimed to reduce maternal and child mortality in the region of Petit-Goâve. Several key barriers to women’s health in Petit-Goâve were identified from the assessment: transportation, availability of materials and delivery of services, and education. The project took immediate action to remedy the lack of supplies, taking into consideration the limitations of the Ministry of Health and their difficulty in providing supplies and medical equipment on a regular basis. The project successfully distributed needed supplies to the UCS who had the responsibility to distribute the needed materials to the MOH dispensary network. Supplying the UCS was also a strategy used by the project to prepare the beginning of other project activities. One of the key missing elements identified by GHA was staff training. Twenty-one CHWs were selected from various sectors in the Petit-Goâve region to participate in a seven-week intensive training (September 25 to November 10, 2006) which took place at the COD-EMH Rural Rehabilitation Project (PRR) center. Upon completion of the training, every CHW received a first-aid kit with basic medical supplies. CHWs then Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 18 returned to their communities with basic knowledge on maternal and child health and primary health care protocols approved by the MOH. One of the key accomplishments in year two was the Development of the Healthy Mothers, Healthy Children curriculum. Designed to assist the community education strategies of the child survival project, the goal of this curriculum was for women to use the information to enact behavior change to improve maternal and infant health within their families as well as in their communities. The same year, the curriculum was pilot tested in the communities with the help of the health workers. Based on the pilot test the curriculum was adapted to respond better to local realities. Year Three: October 1, 2006 to September 30, 2007 Year three was a very important year for Healthy Mothers, Healthy Children as it marked the beginning of full-scale activities at the community level. The health workers were trained and had returned to their communities to deliver services. Monitoring their activities to measure the project progress was a very important step at that time. It had been agreed by GHA and its partners that since the main purpose of Healthy Mothers, Healthy Children was to support the UCS (a government division) of the MOH (government ministry), monitoring tools would hence be used to keep track of progress. The UCS had the responsibility to distribute official pregnant women charts, child growth monitoring charts, and family registers to allow the participants to follow MOH monitoring protocols. The main accomplishments in year three were:  expanding vaccine coverage for children under-five and women of reproductive age (through rally posts and mobile clinics)  providing facility-based pre-natal and post-natal care in the focus areas of Petit-Goâve The midterm evaluation was also conducted in year three and showed that the project was on a good path to achieve its objectives. Year Four: October 1, 2007 to September 30, 2008 The first three years of project implementation in Haiti were marked by delays due to staff turnover, political, and economic insecurity, and natural disasters (as will be further detailed in the section on barriers to implementation). Despite the rocky start, year four showed a lot of progress in terms of activity implementation. Training for TBAs occurred during year four, which provided the Petit-Goâve health district with a team of trainers from their local partners, i.e,, the Bureau COD-EMH and the UCS￾MOH, who would continue to strengthen the capacity of local TBAs even after the end of the project. These trainers conducted the 16-week-long training for 50 TBAs in Petit￾Goâve, bringing the number of trained TBAs in the project to 62. The TBAs started participating in monthly meetings after they completed the training. CHW outreach activities increased significantly during year four. The majority of the 21 CHWs who were active in the project were assigned to health facilities in the area and were integrated in their monthly activities. During year four, they completed their training in the Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 19 six modules of the Healthy Mothers, Healthy Children curriculum and applied their training by teaching the modules in the 21 women’s community groups, recording radio messages that were broadcast throughout the district, and participating in other educational sessions sponsored by the project. Additionally, the CHWs participated actively in monthly meetings and submitted regular reports on their activities. Making up for significant delays in project implementation since the beginning of the project, community mobilization activities were in full force as of the end of year four. These included monthly meetings which took place with groups of women, youth, community leaders, religious leaders, and traditional healers. These meetings covered the main modules used in the Healthy Mothers, Healthy Children curriculum developed during year three. Larger community-oriented activities which were implemented included events around World AIDS Day, International Women’s Day, Mother’s Day, as well as other local holidays. Clinical outreach activities expanded throughout year four. Although there were some constraints due to travel accessibility and access to funds in the field, 37 mobile clinics took place out of the 48 that were originally planned. Each of these clinics brought a multi￾disciplinary team of a doctor, a nurse, auxiliaries, a lab technician, and a pharmacist to areas of the district that are outside of easy access to health facilities. In some areas, regularly scheduled mini-clinics were established and conducted on a bi-monthly basis. These mini￾clinics brought a more streamlined team to provide primary health care to residents. Two international medical mission trips also took place through sponsorship from the Methodist Church of Haiti. In addition to the trainings of TBAs, TBA trainers, and CHWs, 20 project personnel from the COD-EMH and the MOH participated in a training workshop on logistics management of pharmaceuticals and medical supplies in March 2008. Year Five: October 1, 2008 to September 30, 2009, and No-Cost Extensions October 1, 2009 to September 30, 2010 Year five of the project implementation in Haiti was marked by intense efforts on behalf of the team to mobilize the community, resulting in good progress in terms of activity implementation. However, it was not an easy period considering the four hurricanes and storms that hit the country in August and September 2008, causing some activity implementation delays. Consequently, Healthy Mothers, Healthy Children applied for and was awarded a no-cost extension from October 1, 2009 – through March 31, 2010 to enable the project to achieve its goals. Monthly educational meetings (sessions), which began late in year four, continued through year five with different categories of groups covering the main modules used in the Healthy Mothers, Healthy Children curriculum developed during year three. These modules covered all subjects related to infant and maternal care including personal hygiene, family planning, STI/SIDA (AIDS), breastfeeding and responsible fatherhood. In total, the sessions covered seven months, and despite all the obstacles, many people were trained through mothers clubs, fathers clubs, youth clubs and religious clubs. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 20 The Community Health Workers (CHWs) trained during year four were active in the Massive Awareness Campaign for Community Organizing, including educational sessions with groups of more than 30 people each, as well as other awareness meetings within their communities and participation in other educational sessions. The year five report states that, “the CHWs participated in monthly training sessions (a total of 12 during year five) that allowed them to increase their capacity to serve the population. The CHWs have become a point of service in their communities for immunization and nutritional monitoring,” the results from the focus groups cast some doubt on this statement (see below). The 62 TBAs trained by the project participated in monthly meetings focused on continued training with the head nurses in order to improve their knowledge and ability to assist pregnant women within their communities. Approximately 143 community leaders from all over the Petit-Goâve district, including holders of influential positions such as town council chiefs, community council administrators, and legal representatives of the community councils, were trained to become educators and to promote new and better health care practices within their communities. Religious leaders were another target for the project’s outreach. To incorporate health care practices in their daily living and preaching habits, approximately one hundred (100) religious leaders, from Catholic preachers to voodoo priests, were trained. Traditional healers, known as “Hougans,” were a very important group for outreach activities, and they were successfully trained during year five. Trusted by the population, the traditional healers were given key information to better assist their clients by promoting the use of health services in the hospital. Seventy-five (75) traditional healers participated in the training sessions in the last six months of year five. Main accomplishments of year five:  Large community-oriented activities were implemented including events around World AIDS Day, International Women’s Day, and Mother’s Day, as well as other local holidays.  Live broadcasts were continued this year helping to spread positive lessons concerning health practices. Two-hour broadcasts three times per week were scheduled allowing the Program Manager and other guests (doctors, nurses) to talk to the population on different aspects regarding the health project. This was also an effective way of increasing the project’s visibility.  Clinical outreach activities expanded throughout year five. Forty mobile clinics took place out of the 48 that were originally scheduled.  The Referral system was established in September 2010. GHA provided the facilities with a radio communication system to discuss case complications and send referrals. They also did a massive community outreach campaign to get the communities to bring people to health centers in a timely fashion. Due to the earthquake of the 12th of January and the logistical challenges that followed, a lot of activities that had been planned in the first no-cost extension period were delayed. In order to better achieve the Healthy Mothers, Healthy Children goals, GHA applied for another no-cost extension and was granted an extension from April 2010 thru September 2010. These no-cost extensions were used to reinforce project activities and ensure the project came closer to meeting its goals. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 21 C. Presentation of Project Results Quantitative Indicators Review The following tables present a review of the status of original indicators chosen to monitor project success (Tables 3-7) for Objectives 1-4 and for specific indicators that serve multiple objectives (cross-cutting activities). Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 22 Table 3. Specific indicators for outcomes under Objective 1 Outcomes Indicator Method Baseline Value Target Goal reached? Comments Objective 1: 75% of mothers of children 0-23 months of age in the Petit-Goâve Region of Haiti will have received/utilized high quality pre￾and post-natal/infant services during their most recent pregnancy by March 31, 2010 Outcome 1: Increased use of Pre-natal Care % of mothers in group of interest who attend pre￾natal services during most recent pregnancy -KPC Survey 65% 80% unknown Due to the project’s many difficulties to collect data, the loss of data that had been gathered and the absence of a government monitoring system, we cannot ascertain if the goal has been reached. Outcome 2: Increased capacity of CHW’s for serving pregnant women % of CHWs trained who recognize 2 signs of pregnancy complication and make proper referrals (goal: train 40 CHW’s) -CHW records -Clinic referral records Training Pre-test results 100% No. 21/40= 53% of goal Twenty-one CHWs were selected to participate in a seven-week intensive training (September 25 to November 10, 2006) which took place at the COD-EMH PRR center. Outcome 3: Increase number of TBA’s associated with clinics number of TBAs attached to health clinics (Eight total facilities require a minimum of 80 TBAs) -TBA reports -Clinic referral records Training pre-test results 80% No. 62/80= 78% of goal The 62 TBAs trained and active in the project area are attending monthly meetings for continuing education and give reports on their activities. Outcome 4: Increase tetanus vaccinations for pregnant women % mothers of children 0-23 months who received 2 TT injections before birth of youngest child -KPC Survey 6.7% 65% card verified unknown Vaccination activities took place but poor recordkeeping does not allow us to quantify them to know the impact. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 23 Table 3. Continued. Outcomes Indicator Method Baseline Value Target Goal reached? Comments Outcome 5: Increased utilization of health centers for deliveries % of children 0-23 months whose birth was attended by health personnel -KPC Survey 66.5% 80% unknown Activities took place but it is very difficult to quantify them or their impact. (See information from focus groups and key informant interviews.) Outcome 6: Increased postpartum care % of mothers of children 0-23 months who had at least one postpartum checkup after most recent delivery -KPC Survey 35.9% 80% unknown See above Outcome 7: Increase checkups for children during postpartum consultation % children 0-23 months who received check-up at the time of mother’s first postpartum consultation -KPC Survey 56.6% 75% unknown See above Outcome 8: Increase incidence of breastfeeding % of children 0-23 months who were exclusively breastfed at birth -KPC Survey 79.8% 90% unknown See above Outcome 9: Increase immunizations during first year of life 1. % children aged 12-23 months who are fully immunized by first birthday -KPC Survey 25% 80% unknown See above Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 24 Table 4. Specific indicators for outcomes under Objective 2 Outcomes Indicator Method Baseline Value Target Goal Reached Comments Objective 2: By September 2007, 75% of persons of reproductive age in Petit-Goâve will be correctly informed about the prevention of sexually transmitted infections (STI), their impact on the outcome of pregnancy and the availability of treatment in the UCS. Outcome 1: Increase in HIV testing, PMTCT for pregnant women % of HIV+ pregnant women registered and entered into PMTCT project -KPC Survey 0 100% unknown Mobile VCT services are offered during the mobile clinics and community outreach activities, while VCT services are systematically offered in project clinics. Outcome 2: improved laboratory and treatment capacity availability of laboratory services Interview records and reports 0 60% Yes Lab personnel participated in the pharmaceutical logistics management training. Shipment of medical supplies valued at US$139,000 arrived in Haiti, December 2008. Two shipments of medical supplies valued at US$149,153 and US$132,082 arrived in Haiti, in 2005 & 2008, respectively, for a total value of US281235, as GHA cost share. Table 5. Specific indicators for outcomes under Objective 3 Outcomes Indicator Method Baseline Value Target Goal reached? Comments Objective 3: By September 2006, 40% of men of reproductive age in the district of Petit-Goâve will have completed a “Responsible Fatherhood” course. Outcome 1: Creation of responsible fatherhood clubs # of responsible fatherhood clubs created n/a 0 1/clinic yes* Men’s groups met on a monthly basis in Y5. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 25 Outcomes Indicator Method Baseline Value Target Goal reached? Comments Outcome 2: Participation in responsible fatherhood clubs # of men registered in responsible fatherhood clubs n/a 0 35/ club Yes* Men’s groups of 50 men each were organized to meet on a monthly basis in Y5. *Indicates that the goal was achieved in one or more years but not all project years. Table 6. Specific indicators for outcomes under Objective 4 Objectives/Outcomes Indicator Method Baseline Value EOP Target Goal reached? Comments Objective 4: Establish a local referral network for quality maternal care in Petite Goâve region of Haiti by September 2008. Outcome 1: 2-way radio communication system links health centers with hospital # of health facilities with operating radios and communication with hospital Installation/ communicatio n records 0 7 Yes, in 12 sites Communication system in place and functional. Outcome 2: Health centers have feasible referral and medical evacuation plans in place and resources for implementation % of health facilities with written plan and resources in place -HFA reports 0 100% yes Emergency evacuation protocol training done in the Extension Period. Outcome 3: Develop standards for obstetric emergencies and other women and infant health emergencies Availability of protocols and procedures -Protocol document ation n/a 100% yes Reported as implemented during the extension period, but documentation not provided Outcome 4:Up to date database of pregnant and post-partum women available in health centers # of health facilities with up￾to-date database with all pregnant and post-partum women in the catchment area -Review of database 0 7 No No register was found in the field, either family registers that are carried by CHWs or health center registers. Please refer to “Discussion of Results” Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 26 Outcome 5: Improved skills of health facility personnel in management and leadership % of identified health personnel with defined skills -Training reports vari able 80% yes Management and Leadership training held for 20 project staff members during Y4, followed by supportive supervision. Table 7. Specific indicators for cross-cutting activities that benefitted multiple project objectives Activity Indicator Method Baseline Value Target Goal reached? Comments Cross-Cutting activities that serve multiple project objectives Youth groups # of community youth groups established and trained n/a 0 5/ clinic/yr yes* 6 youth groups of 50 youth each were created and trained during Y5. Theatre # of community theatre skits organized n/a 0 1/month yes* Theater skits were organized at the community level using local theater groups during Y5. Radio # of radio spots produced and aired -Project activity reports 0 No target yes Radio spots for outreach activities took place. Educational sessions were recorded and diffused. Peer Leaders # of peer leaders identified and trained to deliver health messages -Project activity reports 0 10/ clinic/yr yes* 50 peer leaders were chosen and trained to deliver health messages during Y4. Women’s groups # of women’s groups created/trained n/a 0 5/ clinic/yr yes* 24 women’s groups of approximately 50 women each were created and trained Community education fairs # of health related community fairs organized n/a 0 2/ year yes* Community health fairs were organized and conducted during Yrs 3, 4, & 5 Community leader workshops # of community leader workshops organized n/a 0 1/yr yes Eight workshops of approximately 50 community leaders each were organized Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 27 Activity Indicator Method Baseline Value Target Goal reached? Comments Increased outreach by CHW’s % of CHWs providing counseling /education on pregnancy, family planning and STI prevention -CHW records and reports Training Pre-test results 80% yes All of the active CHWs are providing this education in their communities and when they work on specific project activities. Health facility capacity building number of health facilities participating in project -Health Facility Assessme nt (HFA) 0 100% No. 2/8=25% of goal Difficulties in the government led to poor functioning and partnership with the MOH clinics, resulting in only two participating in the project. Health facility pharmacy support % facilities with no deficiencies recorded in essential drugs and supplies for key services Pharmacy inventory records 0 100% unknown Records not available. Pharmacy logistics management training given to 20 project staff. Supervision and monitoring continued into Y5. National level stock outs, especially for contraceptive supplies, continue to affect all project clinics. *Indicates that the goal was achieved in one or more years but not all project years Data Quality: Strengths and Limitations As the preceding tables show, it is difficult to quantify the real impact of the project’s activities in the community, because of which so many responses to the ‘goal reached?’ question are ‘unknown,’ particularly for Objective 1: maternal and child health. No quantification of the number of people actually reached by Healthy Mothers, Healthy Children can be made due to the weak monitoring system and data loss due to natural disasters. No registers were found in the hands of the CHWs or at the facilities. Therefore the only way to really have an idea of what happened is to ask the communities how they lived during the duration of Healthy Mothers, Healthy Children. The qualitative approach is the only way that we can have a real perspective on what the project brought to the population of Petit-Goâve- so we conducted focus groups. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 28 Presentation of Project Results based on Qualitative Evaluations --Focus Groups and Key Informant Interviews Focus Group Methodology The Healthy Mothers, Healthy Children final evaluation relied on open-ended interviewing techniques in both discussion groups and key informant interviews. Discussion groups were conducted in French and creole by the GHA consultant evaluator, Dr Erlantz Hyppolite, with the presence of GHA Program Manager, Ms. Girija Sankar. Both of them took detailed notes on the discussion, and as soon as possible after each discussion group (within 24 hours in all but one case), they met to debrief on the findings, most often accompanied by Schmidt Brewster, GHA’s community mobilizer in Petit-Goâve, who helped to organize the groups along with Nurse Mirlaine Deshormeau. Discrepancies and remaining questions were discussed among the three team members in order to confirm findings and refine questions for subsequent groups. In all, nine focus group discussions were conducted. Seven of them were reuniting community groups (Fathers club- 1, youth clubs- 2, mothers’ club- 1, community leaders- 1, traditional healer/voodoo priest “hougan”- 1, and religious leaders- 1) that had participated in the project, and two of them were held with CHWs and TBAs who were also project participants. In total, 335 people took part in the group discussions. The group discussions occurred in community buildings such as, offices, churches, and a school. The questions that we asked each groups were subdivided in specific themes: group constitution and formation, trainings, and service delivery. More specific questions were asked about immunization and maternal death for certain groups. Other themes were covered with the CHWs and TBAs specific to their activities: 1. Average numbers of deliveries held in a month by the TBAs 2. Supervision techniques (both TBAs and CHWs) 3. Data collection (both TBAs and CHWs) Each participant in the focus group was asked to identify the section that they came from. Presented below (Table 7) is a distribution of focus group participants by area of origin. That strategy allowed us to have a broad idea of how the project was accepted and implemented in the different sections of Petit – Goâve, at the least to know which sections were and were not represented in the focus groups. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 29 Table 8. Distribution of focus group participants Section Youth groups Church groups Voodoo priests Mothers groups Fathers groups Leaders groups CHWs TBAs Total Fraction of total 1 11 2 1 1 7 1 7 30 0,09 2 2 4 4 1 1 1 1 11 25 0,07 3 2 1 3 4 2 2 6 20 0,06 4 4 3 6 1 1 5 1 21 0,06 5 2 1 1 3 1 2 10 0,03 6 11 2 1 3 2 1 1 21 0,06 7 3 1 6 1 11 0,03 8 1 2 1 2 2 1 2 2 13 0,04 9 11 1 1 4 8 1 26 0,08 10 1 2 1 2 2 8 0,02 11 24 9 5 5 9 12 3 7 74 0,22 12 27 7 7 8 7 15 2 3 76 0,23 Total 99 31 27 27 37 59 19 36 335 100 From the table of attendance we are able to say that 56% of the participants came from the three sections that are primarily urban (1, 11, and 12), which contain 48% of the commune’s total population. Representation from the other 9 sections that are completely rural totaled 44%, which is roughly comparable to the population breakdown of those sections. The distribution of CHWs was very well spread among the 12 sections, but TBA’s were only present from half of the sections (see Table x). Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 30 Summary of focus group responses by theme and sections Table 9. Theme One: group constitution and awareness of the project Question /Subject Youth groups (99 people) Church groups (31 people) Voodoo Priest (27 people) Mothers groups (27 people) Fathers Groups (37 people) Leaders Groups (59 people) Total (280 people) % Have you heard of the Healthy Mothers, Healthy Children project? 76% say yes 80% say yes 55% say yes 40% say yes 59% say yes 73% say yes 192 69% When was the group founded? 2007 2007 2009 2009 2009 2008 N/A N/A How many times did the group meet per month? 3 times 2 times 3 times 2 times 2 times 3 times N/A N/A How were the group members chosen? youth group AQUISHA Church Comm-- unity Mobilizer Church church Church N/A N/A When was your last meeting? 2010 2010 2009 2010 2010 2009 N/A N/A On average, 69% of focus group participants had heard of the project, but this percentage varied greatly by the type of group. Knowledge of the project was relatively high in youth groups, church groups, and leader groups (all above 73%) and relatively low with traditional healers (voodoo priests) and father’s groups (55 and 59%, respectively). Alarmingly, knowledge of the project’s existence was lowest in the mother’s groups (40%). The oldest groups we found had started meeting in 2007, which was year 3 of the project, indicating a slow start to implementation. All the groups meet 2-3 times per month, indicating that they are a good platform for community health education. Most of the groups had been formed after a church announcement, which suggests that they might have used informal protocols to create those groups. Groups created by affinity such as these are easier to integrate and often have more success with message transmission and acceptance. For example a “Responsible Fatherhood” group that is taught the importance of breastfeeding can give direct support to a mothers group of breastfeeding women if the participants are pulled from the same pool. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 31 Table 10. Theme Two: Training Question /Subject Youth groups (N=99) Church groups (N=31) Voodoo Priest (N=27) Mothers groups (N=27) Fathers Groups (N=37) Leaders Groups (N=59) Total (N= 280) % Training(s) given 100% 100% 100% 100% 100% 100% 280 100% What themes were covered? STI/AIDS,teen pregnancy, hand washing STI/AIDS,teen pregnancy, hand washing Risk factors in pregnancy, hygiene STI/AIDS, teen pregnancy, breastfeeding, hand washing Responsible father hood, risk factors in pregnancy, breastfeeding STI/AIDS, teen pregnancy, hygiene N/A N/A Which theme was the most important? hygiene hygiene hygiene Hygiene hygiene hygiene 280 100% The training method was appropriate 24 % 39% 26% 40% 59% 58% 110 39% The trainings changed your behavior 100% 100% 100% 100% 100% 100% 280 100% The CHW followed up on the lessons 13% 67% 19% 22% 22% 22% 66 23% The project organized trainings for all the clubs, which all the participants found to be effective in that they changed their behavior (see table above). This is the strongest positive result that we have. The trainings covered a range of topics, from STI/AIDS, teenage pregnancy, pregnancy risk factors, breastfeeding, responsible fatherhood, and the importance of hygiene and hand washing. All groups received a variety of messages. Some suggestions and feedback for improvement included providing a written information sheet during trainings (all participants) and using audio/visual aids (youth groups). Interestingly, all groups considered the hand washing messages as most important, perhaps influenced by the focus groups being conducted during the height of Haiti’s tragic Cholera epidemic in late 2010. The degree of follow-up by the CHW’s varied as well- the CHW’s seem to be especially active in church groups (67% follow-up) and much less active in all other groups 13%-22%. This might be explained by the fact that Petit-Goâve is a very religious city and it might had been more easy for the CHWs to find people through the church and follow up with them after church hours. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 32 Table 11. Theme Three: Service Delivery Question /Subject Youth groups N=99 Church groups N=31 Voodoo Priest N=27 Mothers groups N=27 Fathers Groups N=37 Leaders Groups N=59 Total N=28 0 % Know the local CHW 22% 77% 81% 55% 56% 57% 138 49% Know the local TBA 65% 83% 100% 55% 89 % 64% 203 73% Mobile clinics or rally posts held in your region 31% 46% 14% 48 % 32% 42% 102 36% There is a working health center in your region 21% 39% 11% 29% 16% 40% 74 26% What can you say about the services provided since the project’s start?** Improved 100% Improved 23% Improved 36% Improved 31% Improved 38% Improved 70% 59 79% Note: ** In the last question the calculation was made considering using only the pool of people that answered positively to having a working health center where they lived (preceding question). TBAs were generally better known in their regions than CHWs, with 73% of focus group participants knowing their local TBA as opposed to 49% knowing their CHW. Notably, 100% of traditional healers (voodoo priests) knew their local TBA mainly because some of them were the actual TBAs. It seems that health outreach activities did not have great coverage, with only 36% of respondents having seen a mobile clinic or rally post in their area, and only 26% lived near a working health center. Global Health Action Final Evaluation of a Child Survival Initiative in Petit-Goâve, Haiti 33 Table 12. Responses on Immunization and Maternal Mortality Question /Subject Youth group s N=99 Church groups N=31 Voodoo Priest N=27 Mothers groups N=27 Fathers Groups N=37 Leaders Groups N=59 Total N= 280 % CHW does regular follow-up visits in your community 24 % 39 % 33% 26% 35% 39% 88 31 % CHW checks your child’s growth chart on a regular basis N/A 64% 44% 30% 43% 39% 79 43% CHWs do screening for malnourishment 18 % 13% 7% 26% 21% 15% 48 17 % Your children are completely immunized N/A 6% Declined 3% 2% 11% 25 13% Immunization happened more often with project presence 75% 70% 62% 63% 70% 71% 204 72% Most women deliver with a trained TBA N/A 93% 96% 74% 56% 57% 130 71 % Women often die when delivering with a TBA 54 % 70% 33% 63% 35% 29% 132 47% Maternal deaths with TBAs have decreased since the project started 100% 100% 100% 100% 100% 100% 280 100 % CHWs visited you after delivery N/A N/A N/A 93% N/A N/A 25 93% The community groups did not have a strong perception of CHW presence for follow-ups, child growth monitoring and malnutrition screening, with less than half of the participants agreeing that these services were provided regularly. This indicates a weakness in the supervisory component of not only the project, but the MOH. The results also show that most deliveries happen at home with a midwife (on average 71% of respondents agreed to this statement) and there still must be a very significant number of women dying while delivering a baby (47% agreed that women ‘often’ die delivering with a TBA). Despite these grim circumstances, all of the participants agreed that in general the situation improved after GHA did training for the TBAs. Also an average of 72 % of people admitted that the immunization situation got better with the project presence, in terms of increased availability of services, even though many could not say that their children were completely immunized (many did not know for sure, only 13% affirmed that they were immunized). 34 Table 13. CHW/TBA Responses on training & maternal health practices Questions/subject CHWs TBAs When were you trained? 2006 2007 What were the most important themes in the trainings? Immunization, Nutrition, Women’s health and care Immunization for pregnant women, safe delivery, referral of candidates for difficult deliveries Did the training change the way you provide certain services? 100% say yes 100 % say yes Do you check the pregnant woman’s medical records (chart)? 100% say yes 100% say yes How often do you do home visits after delivery? Always always What advice you give after delivery? Immunization, Nutrition, Post-natal consultation Immunization, Nutrition, Post-natal consultation Note: 55 CHWs and TBAs interviewed in total. The TBAs and CHWs received training from Healthy Mothers, Healthy Children in the project’s early years, and the trainings were effective in positively changing at least some of their practices. The themes that they mentioned as the most important part of their training (immunization, nutrition, women’s health and care, referrals) suggest that the women’s health and child survival components were strong. Anecdotal evidence suggests that CHWs and TBAs check the woman’s medical chart and carry out home visits after delivery, providing a general post-natal consultation and advice on immunization and nutrition. Table 14. Services offered by TBAs Question Response How many times do you visit a pregnant woman before delivery? An average of 3 Do you use your birthing kit to perform each delivery? Always How many women do you deliver in a month? An average of 3 How often do women die giving birth? Rarely to never TBAs see pregnant women an average of 3 times before they deliver, deliver an average of 3 women per month, and always bring their project-provided birthing kit (which helps to ensure proper hygiene for mother and child). They responded that women die ‘rarely to never’ while giving birth, which is probably an underestimation according to common knowledge and compared to the community perception of maternal death (‘often,” as reported in Table 11). TBAs are perhaps in denial of incidences of maternal death, which is understandable because it would negatively impact their reputation. This situation is not only true for the TBAs in Petit-Goâve, but for most of the TBAs in Haiti. This shows clearly that the strategy to fight maternal deaths has to be revised. The MOH is currently recommending strongly that every women deliver in a health facility, but it will not be that easy to take the TBAs out the system as there is not enough coverage in terms of delivery centers all over Haiti to handle the demand. 35 Table 15. CHW services and outreach efforts Question Response Is every pregnant women registered in the MOH women’s register? 100% use personal notebooks for registry How often you do rally post in a month? Once How often were you supplied with vaccines? Rarely Do you do home visits monthly? When possible Do you do nutritional monitoring for the children under five in your communities? 100% said yes, but they had no scales CHWs did not do enough outreach activities, and most of them said they did only one rally post per month, when the standard is 2-4 times per month. Possible reasons for fewer outreach activities include lack of supervision and also the fact that there are not enough CHWs. Referring back to Figure 1 and Table 1 from the Introduction, the primarily urban sections 1, 11, and 12, have a total of 68,390 inhabitants, 48% of the total population of the commune. Within these three sections, the population is 48% urban and 52% rural (see Table 1). According to MOH guidelines, there should be 27 CHWs in those three sections alone due to their population. But since those sections are a part of Petiti-Goâve urban area with more access to transportation that would allow each CHW to cover more territory, the minimum number of CHWs for adequate coverage could be reduced to 12. However, the project could only support six, which was not sufficient for proper coverage. Considering the importance of the rural populations of Petit-Goâve (77% of total population, Table 1), rally posts once a month were certainly not enough to reach the project’s coverage goals. Rally posts upto 3-4 times a month and spread out strategically in the different communities might have addressed coverage issues. The lack of a proper M&E system on the national and project level is obvious in the fact that none of the CHWs had any official MOH women’s register and that they all had to use their personal notebooks instead. The family registers are supposed to be provided by the MOH, and the project planned to use the MOH M&E forms. Instead, the absence of official registers seriously weakened the project’s data collecting system. Besides registers, there was a noticeable lack of materials and supplies including scales and rare vaccine provisioning. But, project staff noted that CHWs were supplied with arm circumference bands for monitoring nutritional status in young children. 36 Key Informant Interviews Key informant interviews were conducted with MOH/UCS representatives and with project staff, including health services personnel working in two facilities (total of three). In most cases, these interviews occurred spontaneously during facility visits (Hospital Notre Dame and Olivier clinics). They served to evaluate and confirm preliminary findings from group discussions, and to fill in and clarify after group discussions. Following is a summary of the three interviews. 1. Joel Charles, Hospital Notre Dame of Petit-Goâve Administrator Notre Dame is the reference hospital for the entire region, the central and most advanced point for healthcare delivery, and also a crucial point of service for the project. Its administrator Joel Charles did not have much information to give about the Project since he had not been involved for a long time with Healthy Mothers Healthy Children (since 2007), and, according to him, there had never been a proper transition from the last administrator. The only thing he had to say about the project was that the referral system had been put in place in September 2010, and that he had participated in the management training in 2007. This situation show clearly how much the project was affected by staff turnovers and how the MOH did not integrate the project activities 2. Roland Francois, UCS Administrator The Unite Communal de Santé (UCS) is the Ministry of Health’s district level health system, responsible for coordinating and overseeing all health related activities in their region for the government. The UCS-Goavienne was a major partner for the project. The interview with UCS administrator Roland Francois revealed a similar lack of general knowledge of and integration with the project as with Joel Charles. The only valuable information that he had to share was in support of the newly instated referral system. He informed us that there were health committees (revitalized by the Project) in five sections of Petit-Goâve, and each committee had nine members leading to a total of 45 people. The committees serve as a platform for health outreach in the communities (they are very useful for the current cholera epidemic, for example) and they are also responsible for medical evacuation coordination. 3. Nurse Mirelaine Deshormeau, Head-Nurse-Administrator at Olivier Health Center The Olivier clinic is a small clinic run by the COD-EMH with one consultation room, an observation room with two beds to monitor patients, a pharmacy, a laboratory, and a waiting room. It is the only clinic that was involved with the project from the beginning, and benefited the most from project activities. The clinic has a refrigerator to maintain the cold chain for immunization. The laboratory can perform all the basics blood test specially HIV and syphilis that are strictly done for every pregnant women. There is no separation between the pregnant woman and the other patients in the waiting room, which poses a risk to the health of the women. Nurse Mirelaine Deshormeau is responsible for the health center and had been involved with the project since the beginning. The interview of Nurse Mirelaine revealed that from the very beginning it has been very difficult for her to carry out activities and report them. 37 The clinic had no monitoring tools, they had to record data in their notebooks (and those records are only available for 2010, examples given in Appendix 4 and 5), they had no means to do supervision in the field, and they had administrative problems that delayed activities. It is her opinion that the COD-EMH is responsible for the confusion that impaired project implementation at the clinic level. Despite those difficulties, she thinks that Healthy Mothers, Healthy Children was very important for the Petit-Goâve commune and has improved the lives of a lot of children and pregnant women, and hopes that GHA continues activities in the region. D. Discussion of Results Since the implementation of Healthy Mothers, Healthy Children, the project has faced a lot of difficulties in terms of implementing activities, which have also made it difficult to evaluate project success. This was due to external factors and factors directly linked to the project, all of which are summarized below. The impeding factors that were external to GHA can be subdivided into the categories of weather/natural disasters, instability of partners, political instability/insecurity, insufficient integration of and by the MOH, and personnel changes, which are detailed in the section below. The situation generated by all those difficulties was one of the first reasons why GHA applied for the first no-cost extension from October 1, 2009 – through March 31, 2010 that would give the project more time to achieve its goals. The situation became more complicated after the earthquake in January 2010 presented difficulties and new priorities, leading to a second no-cost extension from April 2010 through September 2010. Weather/natural disasters Due to ongoing rain during the last two months of year one, there were some activity delays and the health facilities had fewer patients than usual. Also, during the last quarter of year four, Haiti was struck with multiple hurricanes and tropical storms that caused massive damage throughout the country to homes, health facilities as well as infrastructure. Although there was limited loss of life in the project area, the main building where meetings and trainings took place for project activities was destroyed along with project data (registers and data collection forms). Additionally, access to some parts of the district for mobile clinics and other outreach activities was severely limited due to the destruction of roads and paths. To address these issues, mobile clinic schedules were modified due to the changed road conditions, and accessibility to isolated areas was being monitored regularly by field staff. Petit-Goâve was at the epicenter of the January 2010 earthquake which caused major destruction and disruption in the entire project area, and required a new set of health and development priorities and actions. People were killed, injured, or displaced; communication and transportation systems were severely disrupted; and health facilities collapsed or were damaged to the point of being non-functional. Instability of partners The Bureau for Coordination and Development of the Methodist Church of Haiti (COD￾EMH) underwent dramatic restructuring throughout the project, which eliminated most of the high-level positions that existed on the original project management structure. This led 38 to confusion regarding roles and responsibilities both within COD-EMH and in the project partnership. Additionally, COD-EMH lost nearly 100% of their other sources of funding. This has severely affected the project in terms of the cost-share of the original agreement, particularly regarding the payment of staff salaries, and operation and maintenance of equipment. For example, in order to implement project activities successfully, GHA was forced to include certain expenses in the budget that normally were covered by the COD cost share component. The Haitian Ministry of Health (MOH) was also a very weak partner due to the political instability during the project period and could not sustain most of its responsibilities. Political instability/insecurity Haiti suffered from political turmoil for the first two years of the project. During this time period there was heightened civil unrest, which resulted in road blocks, keeping project staff based in Port-au-Prince from commuting to the project site regularly. Moreover, the U.S. Department of State issued various travel warnings throughout year two, inhibiting GHA’s US-based staff and medical teams from traveling to Haiti. This caused some planned trainings to be cancelled. Kidnappings were rampant in spring of 2005 and impacted the project. The kidnapping of Dr. Roussel Toussaint, Medical Director of Notre Dame Hospital, who was employed 25% by the project, was a big setback for Healthy Mothers, Healthy Children. Although he was not seriously harmed, and was able to escape, he decided to leave the area afterwards. Dr. Toussaint had been instrumental with not only managing the Notre Dame Hospital, but also consulting and referring patients at the UCS in Olivier, where he had worked one day a week; his departure reduced project capacity. Insufficient Integration of and by the Ministry of Health Certain project activities were not possible due to insufficient integration of the Ministry of Health into the project design. For example, MOH officials resisted taking responsibility for active supervision of the Healthy Mothers, Healthy Children staff’s work in the MOH clinics because the officials felt that they were not receiving any direct financial support from the project for the clinics. That situation led to the project not being able to work with six clinics, and put GHA and COD-EMH in a position to reconsider their coverage strategy with lesser support from the MOH. In the revised strategy, the MOH was still included in project planning, but not implementation: GHA and COD personnel met regularly with MOH staff to provide updates on activities, and project personnel emphasized the collaborative nature of the project in order to encourage active participation of Ministry officials. Some essential materials for monitoring and follow-up, which were supposed to be provided by the MOH, were very difficult to obtain (specifically, mothers charts and child growth monitoring charts). The MOH and UNICEF were the only vaccine providers as well as the only suppliers of propane tanks to keep those vaccines refrigerated and usable. As is true for health intervention projects throughout Haiti, it was a struggle throughout the duration of Healthy Mothers, Healthy Children to acquire vaccines and propane tanks in a timely manner. 39 Personnel Changes Both GHA/Headquarters and GHA field partners experienced changes in staff throughout project implementation. At the beginning of the project, the COD director who had worked with GHA in the planning of the child survival initiative resigned. The new director, Reverend Marco Depestre, after a year and a half of working on the project was asked to resign by the president of the COD-EMH in September 2006. However, the conflict between the president and the director began as early as May 2006 and was seriously impairing the project implementation. Personnel changes slowed progress on activities, since new personnel needed time to catch up on the project, and made it very difficult to maintain consistent work and action plans and strategies. This was especially complicated because there were three large partner organizations involved. Consider the number of key staff that changed throughout the life of the project: at the beginning of the second quarter of year four, the GHA/HQ technical backstop and GHA/Field managers were new, the COD-EMH point person was newly appointed due to restructuring, and the Director of the UCS (the Community Health Unit, which is the local Ministry of Health office) was new as well. At the end of year four, the Director of the UCS was replaced again. To strengthen stability and continuity of the project, project partners met regularly in person and through phone conversations to facilitate teamwork and learning, and GHA/HQ Program Staff and the Haiti Project Manager participated in the USAID Technical Development Meeting to learn from other Child Survival grantees. Impeding factors that were directly linked to the project included monitoring and evaluation of the field activities, staff supervision, transportation, and management presence for supervision, which will be described below. Monitoring and Evaluation of field activities A statistician assigned part-time on the project in year one by the COD was not paid full￾time by the COD as an employee in year two and quit, and was never replaced. Attempts by GHA to replace him specifically for the project were not successful. Throughout the project’s implementation, GHA headquarters had difficulty getting written activity reports or monitoring data, despite regular requests to field staff and receiving their promises to deliver. Eventually it became clear that, partially due to staff turnover, the knowledge of the planned Monitoring and Evaluation (M&E) structure of the project was seriously limited and the newly appointed staff did not have access to appropriate forms for data collection. To try to solve the problem, GHA prepared clear, easy-to-use M&E forms to use in the field. During field visits and teleconferences, staff was trained on the M&E techniques, but the project staff did not successfully adopt the techniques. Staff Supervision The responsibilities for field staff supervision were naturally left in the hands of the Field Program Manager. This person was assisted in this role by the GHA-HQ Program Manager and the regional MOH/UCS officials to ensure effective coordination of monitoring and 40 supervision activities at the communities and at the project-assisted facilities levels. From document review, the final evaluation assessment team noticed that the number, roles, and workload of personnel and supervisory visits appeared to be insufficient for meeting the technical and managerial needs of the project. Transportation The project had very limited means of transportation, since only one 4-wheel drive vehicle was donated to the project by another NGO in January 2009 (year five) given to GHA for project use. Although arriving late, the vehicle was very useful for the field staff to realize activities in areas that were very difficult to access. However, there was confusion regarding ownership of the vehicle between COD and GHA, which led to many difficult situations that resulted in the postponing of a lot of activities. However, the project also funded two motorcycles which GHA purchased for field activity use with the clinics and community group activities. Through-out the project, GHA also paid the operating expenses and repairs for a separate COD-EMH vehicle/truck as well. The COD did not maintain its own vehicles, and relied on external funds for this purpose. Management Presence Even though GHA had a project manager (PM) on the ground, the fact that there was not an official GHA office in Petit-Goâve and the PM had to work with two weak partners made the implementation of Healthy Mothers, Healthy Children very difficult. The lack of an office in Petit-Goâve fed the confusion that was existing throughout the project implementation between GHA, COD and MOH. The Project Manager was essentially the only management person on the ground for GHA. He had to work with two weak partners that did not embrace all of the aspects of the project, which made his mission more difficult. If GHA were to have an official office with the basic management team to support him in Petit-Goâve, a lot of the difficulties that the project had encountered could have been managed more efficiently. E. Conclusions and Recommendations Considering all those difficulties and delays at different levels, especially in the monitoring system, it is difficult if not impossible to estimate the project impact and effectively determine if the objectives were reached at a quantitative level. Most of the data that had been collected were lost during the 2008 hurricane and the rest were lost after the 2010 earthquake. The GHA headquarters did not have backups of the data because of insufficient reporting. However, to have an overall idea of Healthy Mothers, Healthy Children’s achievements, we will examine the objectives, outcomes and indicators by comparing the KPC results at baseline with the final year report, in the following tables. The comment column in the far right side of the table, explains the status of each indicator. 41 Overall achievement of objectives The project objectives were more quantitative than the status of the monitoring systems implemented by MOH or GHA will allow us to determine definitively. That is why this evaluation gathers qualitative evidence from focus groups and interviews. There were cross￾cutting activities that supported multiple goals in terms of community education and mobilization on key issues but are impossible to describe in a quantitative manner given the data at hand. The activities included focused youth groups, community theatre, radio spots, health fairs, peer and community leader groups, and women’s groups. Special trainings were held with the community groups, and changed their behavior. The activity and capacity of the CHW’s were increased, and the capacity of some health facilities was improved. Considering the range of data available, here is what can be said about achievements towards each of the four project objectives: Objective 1: Seventy-five (75) percent of mothers of children 0-23 months of age in the Petit-Goâve region of Haiti will have received/utilized high quality pre-and post-natal/infant services during their most recent pregnancy by September 09. Accomplishments: a. 19 CHWs received comprehensive training on community-based health outreach, including maternal and child health b. 62 TBAs trained c. Community education on the importance of seeking care during pregnancy d. Focus group results indicate that: i. In general, maternal and child health improved during project ii. Most deliveries still occur at home, with TBAs 1. maternal mortality is perceived as high 2. TBAs always use birthing kits supplied by the project 3. TBAs follow up after delivery iii. Immunization levels increased due to project Objective 2: By September 2007, 75% of persons of reproductive age in Petit-Goâve will be correctly informed about the prevention of sexually transmitted infections (STI), their impact on the outcome of pregnancy, and the availability of treatment in the UCS. a. PMTCT services offered in mobile clinics and project clinics b. Laboratory and treatment capacity improved c. Community education took place. STI/AIDS were themes included in sessions with youth groups, church groups, mother’s groups, and leader’s groups Objective 3: By September 2006, 40% of men of reproductive age in the district of Petit￾Goâve will have completed a “Responsible Fatherhood” course. d. Men’s groups were formed and covered the curriculum, but we cannot quantify the number of people reached by this initiative e. Community education also included this curriculum. 42 Objective 4: Establish a local network of referral services that provide quality maternal and neonatal care in Petit-Goâve region by September 09. a. A two-way radio communication system was installed b. Medical evacuation protocols established c. Management and leadership training given for 20 staff d. But, project database is poor, making any meaningful outcome/impact evaluation difficult. Even though the focus groups revealed some gaps and weaknesses, the project had a certain impact that was recognized by both caregivers and beneficiaries. This is despite numerous external challenges posed by weather/natural disasters, instability of partner organizations, political instability/insecurity, insufficient integration of and by the MOH, and frequent personnel changes. Trying to work with the UCS and the COD, who were the most important providers in health care in the area, was a good strategy, and the best way to guarantee sustainability of the activities, but the local partners’ weak administration was also a liability to the project. On the project’s side, there was insufficient monitoring and evaluation of the field activities, staff supervision, transportation, and management presence for supervision. In conclusion, everyone (beneficiaries, employees, caregivers) says that the project was useful and saved lives. As evidence, take the story of a TBA who was also a mambo (voodoo priestess), who said that before her training, when a woman had seizures and convulsions during pregnancy she would keep her and try to treat her, but now that she has learned they are symptoms of eclampsia, she has already referred nine cases. Eclampsia causes the death of mother and child, so just with that one TBA there is the potential that eighteen lives were saved. Recommendations I recommend better supervision to improve data collection by making sure the tools are in place (GHA should develop its own system of data collection, since the MOH system has always been weak and will probably continue to be so.) The reporting system would had been more effective if, since the start, there had been a statistician working full time on strengthening the project’s monitoring system and making sure that the correct data were collected. In the future GHA should consider involving itself more administratively, both in Petit￾Goâve and in Port-au-Prince, to allow for more supervision and more continuity of project activities. Having an office with a strong administration would certainly have made the dialogue with MOH and COD easier. The lack of GHA administrative structure in the field could be felt throughout the project implementing period as was especially evidenced in interviews with key partners Joels Charles and Nurse Deshomoreau. If GHA had an office in the Petit-Goâve region (even if it was integrated in the office of a partner), most of the management and administrative misunderstanding that occurred during the project implementation time would had been identified since year one and proper actions could have been taken. This also would have facilitated staff supervision greatly. GHA needs to be both better connected to its partners, and more independent- to not rely heavily on partner funding and activities for achieving outcomes, but to help them progress and improve in tandem with GHA’s efforts. 43 F. References Davis, Robin, Rachelle Etienne, et al. 2009. GHA Child Survival Annual Report for Year Five. Submitted to USAID 2 November 2009. EMMUS 2005-2006. Enquete Mortalite, Morbidite et Utilisation des Services IV. Prepared by the Institut Haitienne de l'Enfance, Petion-Ville Haiti. Sponsored by USAID, CDC, UNICEF, UNFPA, and others. Calverton, Md. IHSI. 2005. Inventory of Resources and the Potential of Communes. (Inventaire des Ressources et des Potentialités des Communes). Haitian Institute of Statistics and Informatics (IHSI). Informational CD. Lattke, Lynda, Yolanta Melamed, et al. 2005. GHA Child Survival Annual Report for Year One. Submitted to USAID 3 November 2005. Mc Swegin, Melissa, Robin Davis, et al. 2008. GHA Child Survival Annual Report for Year Four. Submitted to USAID 31 October 2008. MSPP. 2005. Plan Stratégique National pour la Reforme du Secteur Santé 2005 – 2010. Bureau of Coordination and Organization of Services, MSPP. Wynne, Leigh, Yolanta Melamed, et al. 2006. GHA Child Survival Annual Report for Year Two. Submitted to USAID 1 November 2006. MSPP. 1995-96. Paquet Minimum de Service 1995-1996. Bureau of Coordination and Organization of Services, MSPP. MSPP. 2003. Manuelle pour le Système des Statistiques, Normes et Procédures. Bureau of Coordination and Organziation of Services, MSPP. Issue : June 2003. Thompson, Angela & Lynda Lattke. 2005. GHA Child Survival Project Baseline KPC Survey Report.Submitted to USAID March 2005. Utshudi, Armand L., Leigh Wynne et al. 2007. GHA Child Survival Midterm Evaluation. Submitted to USAID 31 October 2007. 44 G. Annexes Annex 1: Result Highlights The Healthy Mothers, Healthy Children project was implemented from 10/01/04-09/30/10, and it benefited greatly from the innovative idea of partnering with the major players in healthcare in the region, for greater project sustainability. We highlight this innovation even though its results are not directly quantifiable, because we believe it is a critical approach to improving healthcare in Haiti. We also describe the promising practice of improving the capacity of TBAs and traditional healers. Innovative Ideas: Working with the COD-EMH and the MOH was certainly the best possible way to accomplish the intended sustainability. Those two institutions run the health system in Petit-Goâve-the MOH has 6 health centers spread throughout Petit-Goâve sections plus the referral hospital, while COD-EMH has two clinics. The initial strategy of working with the MOH and COD health centers did not work very well, so the project adapted and modified its plan and intensified the community outreach programs more, while still working very closely with its partners. After six years of working with two very difficult partners, GHA gained experience that will benefit the organization in the future. There is no way to quantify the results, other than to say that without this approach and the adaptability that GHA showed, it is unclear whether the program would have had any results at all. Promising Practice: The Healthy Mothers, Healthy Children project in Petit-Goâve, Haiti, had the goal of increasing access to and demand for quality health services in the district. In the Haitian context, the majority of women deliver at home with TBAs. This was confirmed during our focus groups. Many women believe in supernatural causes of illnesses during pregnancy, and this causes the population to doubt the effectiveness of the clinical health services supported by the project. The TBAs recognize the importance of both the spiritual and medical causes, and are trusted to treat both. GHA focused on increasing their medical capacity. Through its lifetime the project trained 62 TBAs in immunization needs for pregnant women, safe deliveries, and the importance of referring women with or at risk for complications. The final evaluation focus groups revealed that almost all TBAs agree that the trainings changed the way that they provide services, and that they now work with the pregnant women’s medical records as stipulated by the MOH. The focus groups also revealed that they always conduct home-visits after delivery and provide information on immunization, nutrition, and general post-natal consultations. This approach was an innovative promising practice and certainly helped the project to reach its goals. 45 Annex 2. List of Publications and Presentations related to the project Davis, Robin. 2010. “Building Health Capacity at District and Community Levels in Haiti.” Panel presentation at Haiti 2020: Haiti’s Public Health System -- Today, Tomorrow, and in 2020. One-day symposium, Dec. 2, 2010. Georgia Institute of Technology, Atlanta, Ga. Davis, Robin. 2009. “Global Health and the Role of Faith Partnerships.” Panel presentation at auxiliary event sponsored by the Faith & Global Health Caucus of the Global Health Council at the Annual Global Health Council Conference. May 26, 2009. Washington, D.C. Etienne, Rachelle, Sankar, Girija. 2010. “Healthy Mothers, Healthy Children Project: Pre- and Post-Earthquake in Rural Haiti, 2004-2010.” Panel presentation at the annual conference of the National Haitian American Health Association. Oct 20-22, 2010. Port-au￾Prince, Haiti. Kramer, Kathryn. 2007. Healthy Mothers, Healthy Children: A Mother Child Survival Curriculum for Petit-Goâve, Haiti. MPH thesis & Poster presentation. Rollins School of Public Health. Emory University. April 2007 . Sankar, Girija. 2010. Manman an Bòn Santé Timoun an Bòn Santé- Healthy Mothers, Healthy Children Project: Pre-&Post-Earthquake in Rural Haiti, 2004-2010.” Poster presentation at the Global Maternal Health Conference, Aug.30-Sep. 1, 2010, New Delhi, India. 46 Annex 3: Project Management Evaluation Throughout the implementation years, Healthy Mothers, Healthy Children had management challenges at different levels. The final evaluation helps explain and identify the challenges and situations that affected project management. The essential project management structure was never present in the field among the partners, and therefore even basic management activities of planning, supervision, staff management, and reporting were very difficult. Planning: Although the project had a very comprehensive DIP which outlined a plan and a good strategy to apply it, the weakness of the other implementing partners, and the fact that GHA did not have a strong administrative presence on the ground, made execution of activities very difficult. Therefore, even though GHA always sought to integrate the MOH and the COD-EMH, a lot of the activities were planned and supported by GHA alone. This situation limited project outreach efforts. Supervision of project staff: Trying to work with partners that do not have the means to implement proper supervision systems combined with GHA’s weak supervisory presence in the field, led to no consistent supervisory system. It is a situation that definitely had negative impacts on the project’s overall objectives. GHA should learn from this experience and take actions to avoid such shortcomings in the future. Human resources and staff management: The project had frequent personnel changes at different levels and different offices, as detailed in the section titled “Discussion of Results” in the main document. There were three administrations involved: GHA, COD-EMH, and MOH. These were three independent administrations managing their own staff and their own issues, which made it difficult to maintain consistency in activities, reporting, supervision etc. For instance the COD administration was in conflict with one of its directors, who happened to have good collaboration with the Healthy Mothers, Healthy Children project, GHA had no power to convince COD to keep him. This situation might have been worked out if there was an administrative mechanism to jointly discuss decisions regarding key project staff. The same problems arose with the MOH office. The confusion created by this situation greatly affected the project components that depended on strong administration to work well. Symptoms included difficult logistics (transportation and the acquisition of supplies were difficult) and expenses were not properly tracked in the field due to a weak financial structure locally. Reporting: The weak administration and lack of a supervisory network made reporting very difficult. GHA tried multiple times to get reports, adapting their guidelines and training staff, but they could not get satisfactory results. 47 Annex 4: Workplan Table Strategy 1: Improve the quality of pre- and post-natal services available in the region Objectives /activities Objective Met Activity status Health Facility Access and Management Analysis of data from exit and key informant interviews to develop actions to improve the utilization of services yes Initiated in year one (Y1). Additional qualitative research took place in Y2, completed Carry out survey of key informants on barriers to health service utilization yes Completed by Y5 Assess current work and effectiveness of mobile health clinics yes Completed Improve availability and quality of mobile health services yes Began in Y3, completed Increased community outreach activities at clinics, vaccination campaigns, and mobile clinics yes Completed Train appropriate personnel on follow-up and counseling methods and strategies yes Completed CHWs and TBAs Training for CHWs and follow up yes 19 trained with full curriculum, completed Explore and implement models to address CHW incentives and sustainability questions no Discussion with partners to provide support after the project ends not conclusive Monitor and supervise TBA activities yes Monthly meetings & visits, completed Support/facilitate post-delivery home visits by trained health agents yes Trained CHW are active in the communities, completed CHWs and TBAs educate women on importance of pre-natal, attended delivery and post-natal care yes 21 mother’s groups trained, completed. Provision of Key Services 48 Provide counseling & referral for HIV testing for all pregnant women who attend the project’s clinics and the Notre Dame Hospital yes Routine availability of testing for all pregnant women established in Y3, completed Immunization services for pregnant women and children, including TT vaccinations yes Supported by the project since Y3 at clinics and mobile clinics, completed Strategy 2: Build/strengthen referral network and channels within existing health facilities and community Objectives /activities Objective Met Activity status Develop communications protocol linked to referral and medical evacuation protocols. no Discussion ongoing Develop plan for referral and medical evacuation b/w health facilities & hospital yes In place, started late September 2010 Train personnel in implementation and use of referral protocols ? Not yet done, but part of 2011 plans Create database of pregnant women visiting health centers and follow-up for HIV no No Database found to date Post-partum visits carried out by staff/personnel ? Unknown. TBAs and CHWs say that they do postpartum visits, but this cannot be verified by any register or monitoring system 49 Annex 5: Rapid CATCH Table. Due to modified circumstances owing to the earthquake of January 12, 2010, and with approval from the CSHGP team at USAID, we did not carry out a final KPC survey. Hence there are no Rapid CATCH final estimates. Annex 6: Please see explanation provided above. 50 Annex 7: CHW Training Matrix Commune Number of CHWs Type Training period Subjects Petit-Goâve 19 Training Sep.-Nov.2006  Community assessments and census  Home visits  Growth monitoring  Hygiene & the environment; waste management.  Sanitation-clean and safe drinking water etc.  Diarrhea, ORS preparation, prevention  Nutrition, causes of identification of malnutrition etc  Vaccination, techniques for vaccination  Malaria and prevention; TB  Male/female reproductive systems  Maternal and child health  Danger signs during pregnancy, labor and delivery, post partum period and in the newborn  Importance of breast-feeding  Importance & methods of Family Planning  STI/ HIV/ AIDS education  BCC activities (Behavioral Communication and Changes)  First-Aid  Participatory methodologies for health education and promotion  Modules from project designed Healthy Mothers, Healthy Children curriculum: 51 -Hygiene -Baby care -Pregnancy -Birth -Mother care Petit-Goâve 21 Refresher training September 2010 Hygiene, Nutrition, Breast-feeding, Pregnancy, Pre￾and post-natal care, family planning, HIV/AIDS; Petit-Goâve 18 Refresher training for newly identified health workers September 2010 Basics of health, identifying problems and solutions within community, census of community members nutrition, hygiene, diarrhea, human reproductive system, pregnancy, danger signs during pregnancy, pre-and post-natal care, delivery, breast feeding, HIV/AIDS. 52 Annex 8: Evaluation team Members and their Titles Name Title Erlantz Hyppolite M.D., Team leader/Lead Evaluator Schmidt Brewster Community Mobilizer, GHA-Haiti Robin C. Davis Executive Director, GHA-HQ Mirlaine Deshormeau Head Nurse-Administrator at the Olivier Clinic, COD-EMH Rachelle Etienne Field Office Coordinator, GHA-Haiti Margaret McCann Deputy Editor, Journal Epidemiology Girija Sankar Program Manager, GHA-HQ Starry Sprenkle Program Manager, Albert Schweitzer Hospital 53 Annex 9: Evaluation Assessment methodology Due to major disruptions cause by the earthquake (facilities that were destroyed and the population that moved because of the fear caused by the frequent aftershocks), a final KPC was not possible. The final evaluation of Healthy Mothers, Healthy Children started with two weeks of desk reviews of the major documents that were produced during the project’s lifetime. It also included a qualitative approach by conducting nine focus groups covering different themes to give a broad idea of the project implementation and impact on the community. Please refer to the section titled “Presentation of Project Results based on Qualitative Evaluation” in the main document for more details. To summarize, the objective of the final evaluation was to  Determine accomplishment of project goals and objectives as defined in the DIP. The evaluation process included:  Desk review of documents/reports, including record reviews and service observations at two health centers/posts.  Qualitative interviews with communities, staff and partners  Focus group discussions with stake-holders The following focus group discussions (FGD) were carried-out by the lead evaluator and other team members during an eight-day field visit:  Community Health Workers---19 FGD participants  TBAs--36  Mothers’ group members --27  Fathers’ group members--37  Youth Club members—99  Church group members/leaders—31  Voodoo priests—27  Community leaders--59 The evaluation audiences are:  USAID (CSHGP, Haiti mission)  Grantee  Local partners (MOH, district health team, local organizations)  Other local stakeholders 54 Annex 10: List of persons interviewed and contacted during Final Evaluation Name Function/Affiiliation Schmidt Brewster Community Mobilizer, GHA-Haiti Joel Charles Administrator, Notre Dame Hospital Robin C. Davis Executive Director, GHA-HQ Mirlaine Deshormeau Head-Nurse/Administrator, Olivier Clinic, COD-EMH Lucito Edouard CHW Rachelle Etienne Field Office Coordinator, GHA-Haiti Lesd Feris Hougan (Traditional Healer) Roland François Administrator, UCS-Petit-Goâve Teddy James Physician, Olivier Clinic Miss. Loudenir Nurse, Siloe Clinic, COD-EMH Serge Lubin Accountant COD-EMH, Petit-Goâve Sony Noel Animator/Mobilizer, UCS-Petit-Goâve Renette Olivier Nurse-Trainer, UCS-Leogane; GHA’s consultant trainer Henri Pierre CHW Girija Sankar Program Manager, GHA-HQ Johane Baptiste Local partner liaison, COD-EMH (contacted several times by email & phone, but did not receive a response) Réginalde Massé Program Manager, USAID, Haiti Mission(contacted several times by email & phone, but did not receive a response) 55 Annex 11: Special Reports Annex 11A: Note on Project Inventory The project did not acquire any inventory items over USD 5000 using USG funds. 56 Annex 11B: Other Pertinent Information related to the Final Evaluation Table 1. Distribution of large landholdings and communities in the 12 rural sections of Petite Goâve. Rural section Number Landholdings Communities 1st section Bino 12 40 2nd section Delatre 10 88 3rd section Trou Chouchou 18 45 4th section Fond-Arabie 17 14 5th section Trou-Canari 10 41 6th section Trou-Canari 4 35 7th section des Platons 29 16 8th section des Platons 4 31 9th section des Palmes 13 61 10th section des Palmes 22 39 11th section Ravine Seche 10 63 12th section des Forques 9 97 Summary 158 570 Source: IHSI-2005. Table 2. Distribution of the population by age and sex in Petit-Goâve Age Group Total Sex Male Female All ages 143191 70364 72827 0-4 years 16373 8129 8244 5-9 years 16588 8330 8258 10-14 18273 9423 8850 15-19 15910 8133 7777 20-24 12114 5802 6312 25-29 9924 4631 5293 30-34 8441 4014 4427 35-39 8370 4014 4356 40-44 7587 3667 3920 45-49 6832 3319 3513 50-54 5916 3050 2866 55-59 3534 1720 1814 60-64 3815 1882 1933 65-69 2868 1301 1567 70-74 2843 1326 1517 75-79 1440 654 786 80-84 1269 538 731 85 and over 1094 431 663 Source: IHSI- 2005 57 Evaluation Instruments/Tools: Instrument 1: Interview Guide for Community groups Focus group topics: Program awareness, the benefit of the trainings conducted by the program; availability, coverage and quality of services offered by the program; knowledge, awareness or understanding of the interventions and perceived effect on the lives of women and infants within the community. Characteristics of focus group participants Key community stakeholders divided in specific groups (mother’s groups, father’s groups, church groups and youth groups) representing the main program beneficiaries. Introduction/Warm-up  Thank you for coming  Explain the purpose of the focus group  Break the ice: participants introduce themselves, facilitator puts them at ease Topic 1: Program awareness/Group constitution (Haitian Kreyol/English) 1. Nou te kon tande pale de pwogwamm GHA pou fam enceinte yo? / Have you heard of the Healthy Mothers, Healthy Children program? 2. Ki le klub ou a te forme? / When was the group founded? 3. Kobien fwa nou te kon rencontre nan yon mwa?/ How many times did the group meet per month? 4. Koman yo te chwasi moun kin an klub la? / How were the group members chosen? 5. Ki denye fwa nou te rencontre?/ When was your last meeting? Topic 2: Training benefits/ Behavior changing 1. Eske yo te kon fe ti formation pou noun an klub yo? / Had you been educated on specific healths topics within your clubs? 2. Ki kalite message yo te kon fe passé? / What themes were covered? 3. Ki les ladan yo kit e pi important selon ou? / Which theme was the most important? 4. Eske nou te remen jan yo te kon fe formation yo? / Was the training method appropriate? 5. Eske formation a te aide ou change kek nan jan out e kon agi? / Did the training change your behavior regarding some specific health practices? 6. Eske Agent de sante bo lakay out e kon encourage ou nan pote ti message pou ou ki te ka permet ou songe formation a ?/ Did the CHW follow up on the lessons? Topic 3: Services and delivery 1. Eske nou kon agent de sante kap travay bo lakay nou? / Do you know the health worker serving your area? 58 2. Eske nou kon matron kap travay nan zon ou a? / Do you know the local Traditional Birth Assistant (TBA)? 3. Eske nou te kon gen activite kilinic mobil ak post rassamblement nan zon nou? Were mobile clinics or rally posts held in your region? 4. Eske gen yon sant sante tout pre lakay ou? / Is there a working health center in your region? 5. Ki sa out a ka di de service nan sant la depi pwogram la komanse? / What can you say about the services provided since the program started? Topic 4: Specific Indicators 1. Eske agent sante kon fe visit souvent nan zon lakay nou? / Do the CHWs do frequent follow-up visits in your community? 2. Eske agent kon habitué kontrole Kat chemin la sante timoun yo souvent? / Does the CHW check your child’s growth chart on a regular basis? 3. Eske agent kon vin pese timoun ki ap viv nan kominote a? / Do CHWs conduct screenings for malnourishment in the community? 4. Eske timoun ou pran tout dose vaccine li? / Are your kids completely immunized? 5. Eske nou te gen activite bay vaccin pi souvent an zon la depi pwogram la te komense ? Did Immunization happen more often with the program’s presence ? 6. Eske pifo fam accouche lopital o sinon lakay yo ? /Do most women deliver in a health facility or at home? 7. Eske pi fo medam enceinte yo accouche avec yon matron ak boite ? / Do most women deliver with a trained TBA? 8. Eske sa rive souvent ke fam enceinte mouri le matron ap accouche yo? / Do women often die when delivering with a TBA? 9. Eske ou ka di kef am ki kon mouri nan accouchement ak matron diminue apre pwogram la te komance? / Could you say Maternal deaths with TBAs have decreased since the program started? 10. Eske agent de sante te kon fe visit pou fam ki fek accouche? / Do the CHWs visit women that have just deliverd ? Closing  We have had a very good discussion on interesting topics. (Summarize the ideas which emerged from the group, noting where there was consensus and where there was not consensus.)  Is there anything anyone would like to add before we close? (Probe: go around the group, giving each participant a chance to respond)  Thank you for your time. To spur discussion within the group, we used the following phrases:  Why?  Why not?  I don’t understand that point. Could you explain it again?  Do you agree with her opinion, (name)?  What do you think about that, (name)? 59 Instrument 2: Interview Guide for CHW’s and TBA’s Focus group topic: Trainings, program implementation, supervision and data collection, maternal health practice. Characteristics of focus group participants: Community Health Workers (CHWs) and Traditional birth attendants (TBAs) that were implementing the program. Introduction/Warm-up  Thank you for coming  Explain the purpose of the focus group  Break the ice: participants introduce themselves, facilitator puts them at ease Training and service delivery of CHWs and TBAs (Haitian Kreyol / English) (Unless otherwise indicated (next to question, in parenthesis) questions posed to both groups-CHWs and TBAs.) 1. Ki leu yo te fe formation pou nou? / When were you trained? 2. Ki sa ki te pi important nan formation a selon ou mem? / What were the most important themes of the trainings? 3. Eske formation a te aide ou sou kijan out e kon travay? / Did the training change the way you provide certain services? 4. Eske ou enregistre fam enceinte ak manman nan registre MSPP? / Do you register pregnant woman and moms in their respective registers given by MOH? (CHWs) 5. Eske nou kontrole kat maman yo souvent? / Do you check the pregnant women’s chart often? 6. Eske ou fe visit pou fam enceinte yo avant et après yo acouche? / Do you do home visits for pregnant women before and after they deliver? 7. Ki konseil ou ka bay yon fam enceinte apre li fin accouche? / What advice could you give after delivery? 8. Eske nou itilize kit matrone le nap fe accouchement? / Do you use your birthing kit to perform each delivery? (TBAs) 9. Kombien fam ou accouche chak mwa? / How many woman do you deliver in a month ?(TBAs) 10. Kombien fwa yon fam kap accouche mori nan men ou ?/ How often do a women die giving birth ?(TBAs) 11. Kombien fwa ou fe post rassemblemen nan yon mwa? / How many times do you do rally posts in a month? (CHWs) 12. Eske out e kon jwen vaccine facil? / How often were you supplied with vaccines? 13. Eske ou fe visit domicilie chak mwa? / Do you do home visits monthly? (CHWs) 14. Eske no fe pese nutritionel pou timoun ki pi piti pase cinq l’anne nan zone nou travay yo? / Do you do nutritional monitoring for the children under five in your communities?(CHWs) 60 Closing: please see instrument 1. Phrases used to spur discussion within the group: please see instrument 1. Instrument 3: Key Stakeholder interviews with GHA and Partner staff Questions for Joel Charles Administrator Hospital Notre Dame (MOH) and Roland Francois Administrator for the MOH-UCS (Haitian Kreyol / English) 1. Ki sa ki te relation ou ak projet a? / What has your relationship been with the Child Survival Project? 2. Eske ou konen bien objectifs projet a? / Are you familiar with the project goals and objectives? 3. Eske ou pense ke projet a te contribuye a renforce pratique sante reproductive yo Petit-Goâve? / What are the contributions you see the project has made to the reproductive health and infant health in the region? 4. Ki sa ou te pense ki te force projet a? poukisa? / What do you think are the greatest strengths of this project? Why? 5. Kisa ki te pi gros defies nan projet a koman nou te aborde yo? / What are its greatest challenges? Why? How were they addressed? 6. Koman projet te travail avec institution ou a, koman role yo te partage? Ki pi gwo dificlute partenariat sa te geyen? / How has the project worked with their partners, including you? What were the relative roles and responsibilities? What difficulties were encountered in the partnership and how were they addressed? What was the benefit of the partnership? 7. Eske support projet tap pote pou institution a te suffisant? / Was the COD/GHA staffing appropriate (in quantity and technical competency) for the job? Why or why not? How might it have been better? 8. Eske system administrative la te present pou gere proje a? / Were the necessary management systems in place (human resources, logistics, supervision, etc.) to effectively implement the project? How might they have been strengthened? 9. Eske selon ou projet a te bien organize pou lit e ka jwen bon resultat? / Was the project appropriately designed to meet the project goals? Why or why not? How might the design have been improved? 10. Eske selon ou projet a atteindre objectif li? / According to you did the project reach its goals? Closing: please see Instrument 1. Questions for Nurse Mirlaine Deshormeau Olivier Clinic head nurse (GHA and COD-EMH supported) 1. Depi ki leu wap travay pou projet a? / How long have you been working for the project? 2. Kisa ou ka di de projet en general? Force ak faiblesse ? / What can you say about the project in general ? Weaknesses , strengths? 3. Si out e ka change yon bagay nan proje sa ou tap change? / If you could change something in the project what would it be ? 61 4. Koman collaboration a te ye avek GHA e COD-EMH? Was the collaboration easy between GHA and COD-EMH ? 5. Kisa ou ka di de clinic mobil, post rassemblement ak vaccination ki pwogwamm te fe ? / What can you say about the mobile clinics, rally post and immunization that occurred because of the program? 6. Kisa out a ka di pou travail matron yo ak agent santé pendan pwogwamm la? / What can you say about CHWs and TBAs work during the program? 7. Eske ou ta di ke avec projet a servce la vin pi bon? Would you say in general that services improved with the project’s implementation? 8. Eske ou ka di ke moun yo pi konnen et key o bousque la sante plis apre formation GHA yo? / Would you say that the population is more aware of the importance of the specific services that were promoted by the project and utilize them more? 9. Eske ou gen commente special pou nou ? Do you have any special comments? Closing: please see Instrument 1. 62 Scanned copy of GHA- MOH Partnership Agreement 63 Example of handwritten reporting from mobile clinics (date, location, number of children, pregnant women, adults, and total people served, recorded by Nurse Deshormeau for Olivier Clinics; records available only for 2010) 64 Example of handwritten reporting from the monthly TBA meetings (Summary statistics for entire group, recorded by Nurse Deshormeau for the Olivier Clinic; records available only for 2010) Format: Month Number of TBAs present Number of Deliveries Number of Newborn Deaths Number of Hospital References 65 Annex 12: Project Data Form (downloaded PDF file presented here as images) 66 67 68 69 70 71 72 Annex 13: Grantee Plans to Address Final Evaluation Findings GHA will develop and implement plans to further strengthen local capacity building, project monitoring and evaluation, consistent data collection & analysis, reporting, field supervision, and local partner coordination and collaboration in the Petit-Goâve region through the following plans and actions:  Recruit and hire a fulltime Program Manager, based in the Petit-Goâve region, to provide strong program oversight, including supervision, monitoring and evaluation, and partner coordination.  Establish a GHA field office in the Petit-Goâve region close to program sites for stronger administration, field supervision, and regular communication and follow up with field staff, GHA health providers, partners, and local resource groups.  Set realistic and meaningful goals, objectives and health indicator targets over a 3-5 year horizon, taking into consideration, availability of resources, skilled personnel, and providing for unforeseen changes (such as natural events or political turmoil).For instance, set specific, realistic and time-bound targets for trainings, health promotion outreach efforts, number of community members trained, number of referrals by health providers and other related outcomes.  Develop simple monitoring and evaluation tools, in consultation and collaboration with MSPP, for consistent data collection by clinic staff, CHWs and TBAs.  Supply clinic staff, CHWs, and TBAs with standard monitoring and evaluation forms, client health cards, immunization cards, and other materials to record and track health interventions and status.  Supply the GHA-trained CHWs and TBAs with the additional equipment and supplies needed to monitor health status, such as portable spring scales for weighing babies.  Establish an easy-to-use, field-tested electronic data collection back-up system to protect and preserve program data from loss due to natural disasters and local turmoil.  Conduct coordination meetings, at least monthly, between the GHA field-based Haiti Program Manager, officials of the MSPP, and other partner organizations in the Petit-Goâve area.  Continue to support and strengthen the community health committees of the MSPP-UCS, which GHA helped to revitalized during the project 73  Train and support staff supervisors for monthly field supervision of GHA-trained CHWs and TBAs.  Increase collaboration and conduct GHA field-based health programs with MSPP as the primary partner and with the COD-EMH as a secondary partner.  Continue to raise additional program funds on a multi-year basis to support GHA’s local partners and to strengthen GHA’s program presence, strategies, and presence in the Petit-Goâve area of Haiti.  Provide ongoing and consistent capacity building for GHA field staff, health providers, and local partners (including MSPP) in the areas of monitoring and evaluation, recordkeeping, reporting, community outreach, field and home-visit skills, referral protocols and procedures, electronic communication, disaster planning & response, and financial management & recordkeeping. 74 Annex 14: Grantee Response to Final Evaluation Findings GHA agrees with the final evaluation findings: 1) that GHA needs strong, full-time program management staff and an office near the program sites in the field; 2) that a consistent monitoring and evaluation system needs to be put into place and utilized for regular data collection, analysis and feedback; 3) that local partner capacity building needs to be strengthened with existing local resources in mind; 4) that the field-based structure for supervision, administration and project oversight needs to be strengthened; 5) that clinic staff, CHWs and TBAs need consistent and strong supervision in the field; 6) that increased collaboration and partnership with the MSPP at the departmental, district, and local levels needs to be fully operationalized within GHA’s programs in Haiti; and 7) that building capacity of community-based leaders for health promotion and care activities strengthens the local health response and status. GHA remains committed to improving community-based health promotion and care services in the Petit-Goâve area, and is continuing priority health program activities there, especially related to maternal and child health. Working with weak local partners has been very challenging, but we recognize the importance of these existing local partners especially in a resource-poor area, and we have learned lessons through this experience to strengthen our own future strategies and approaches. In addition, we have benefitted from the lessons learned, innovative ideas and practices of other child survival grantees in Haiti, such as the Haitian Health Foundation in Jeremie. The guidance and technical assistance of the USAID Child Survival and Heath Grants Program team has helped GHA build its own capacity as a child survival project grantee and implementer. MSPP officials in Petit-Goâve have directly acknowledged that GHA is the only organization helping to revitalize and support the functioning health committees and helping to build a communication, referral and emergency response system in the Petit-Goâve area. We have a base to build from for a strong program that involves community mobilization and the commitment for a continuing partnership between GHA and MSPP in Petit-Goâve, Haiti.