CONCERN WORLDWIDE USAID Child Survival & Health Grants Program GHS-A-00-05-00018-00 October 2005 – September 2010 Midterm Evaluation Report The Urban Health Project for Five Disadvantaged Neighborhoods of Metropolitan area of Port-au-Prince, Haiti Delmas Commune: St. Martin and Cite Okay-Jeremie Petion-Ville Commune: Jalousie and Bois de Moquette in Port-au-Prince Commune: Descayettes A Partnership of Concern Worldwide, FOCAS, and GRET with the Ministry of Health West Department Submitted on October 31, 2008 Prepared by: Marcie Rubardt, Consultant In collaboration with: Michelle Kouletio, Health Advisor, Concern Worldwide Guerda Debrosse, Program Manager, Concern Worldwide Haiti Helen Spraos, Assistant Country Director, Concern Worldwide Haiti ACKNOWLEDGEMENTS I would sincerely like to thank all of the people who participated in this evaluation. It was sometimes difficult to pull together and it isn’t always easy to deal openly and honestly with the challenges and difficulties such as this project has faced. I appreciate your good will and commitment to the beneficiaries we are all working for. I hope this effort leads to a stronger and more effective project, and improved health for its target children. Marcie Rubardt 1 ACRONYM LIST BC Bureau Communal of the MSPP CAMEP Centrale Métropolitaine d'Eau Potable CBO Community based organization CDO Community Development Officer CHO Community Health Officer C-IMCI Community Integrated Management of Childhood Illness CMAM Community Management of Acute Malnutrition CSHGP USAID-funded Child Survival and Health Grants Program DHS Haiti Demographic Health Survey DIP Detailed Implementation Plan DSO West Health Department of the Ministry of Health FH Food for the Hungry FOCAS Foundation of Compassionate American Samaritans GENESIS A Haitian public health management and technical consulting firm GRET Groupe de Recherche et d'Echange Technologique HC Health Center HMIS Health management information system IMCI Integrated Management of Childhood Diseases IR Intermediate Results KDSM Federation of CBOs operating in St. Martin “Kowodinasyon pou Devlopman Sen Maten” KPC Knowledge, Practices and Coverage survey MEI Local NGO “Mission Evangelique Internationale” working in Bois Moquette MNC Maternal Newborn Care MSH Management for Science and Health MSPP Ministre de la Santé Publique et de la Population (Ministry of Health) MUAC Mid-Upper Arm Circumference measurement OBDC Local NGO « Oeuvres de Bienfaisance et Développement Communautaire » working in Jalousie ORS Oral Rehydration Salts PM Project Manager PROMESS Essential Medical Supply Store PSI Population Services International SNELAK Local CBO operating in Descayettes, “SOSYETE NEG LAKAY” TBAs Traditional Birth Attendants YV Youth Volunteers 2 TABLE OF CONTENTS Executive Summary.......................................................................................................... 4 Assessment of Progress - General................................................................................... 7 Overview..................................................................................................................... 7 Constraints / Recommended DIP Adjustments........................................................... 9 Highlights from M&E Table..................................................................................... 10 Work Plan Status....................................................................................................... 11 Assessment of Progress – Technical Approaches......................................................... 13 CDD.............................................................................................................................. 13 Nutrition........................................................................................................................ 14 HIV/AIDS..................................................................................................................... 15 ARI / EPI....................................................................................................................... 16 MNC ............................................................................................................................. 16 Assessment of Progress – Cross-Cutting Approaches................................................. 17 Consortium of Three Partners....................................................................................... 17 Community Mobilization.............................................................................................. 19 Behavior Change........................................................................................................... 23 Capacity Building ......................................................................................................... 24 Health Systems Strengthening ...................................................................................... 25 Health Facilities........................................................................................................ 25 Policy and Advocacy .................................................................................................... 26 Scaling Up..................................................................................................................... 26 Equity............................................................................................................................ 27 Sustainability................................................................................................................. 27 Mission Collaboration .................................................................................................... 27 Contextual Factors.......................................................................................................... 28 Conclusions and Recommendations.............................................................................. 28 Action Plan ...................................................................................................................... 29 Annexes: 1. Results Highlight 2. Project Presentations 3. Management Review 4. Monitoring and Evaluation Table 5. Rapid Catch Indicators 6. Midterm Knowledge Attitudes and Practices Survey 7. Health Facilities’ Assessment – midterm 8. Evaluation Team Participants 9. Evaluation Methodology 10. List of Persons Interviewed 11. Project Data form 12. Compiled Recommendations 13. Proposed Revision of Objectives 14. Work Plan 3 Executive Summary Concern Worldwide Haiti (Concern) has been implementing a five-year USAID Child Survival funded project called, “The Urban Health Project for Five Disadvantaged Neighborhoods in Port-Au-Prince, Haiti” since 2005 in partnership with the Ministry of Health West Department, FOCAS, and GRET. In St. Martin it also works with Kowodinasyon pou Development Sen Maten (KDSM), a federation of six Community Based Organizations (CBOs) in Saint Martin. The strategic objective of the project is: “sustained improvements in the health status of mothers, children and youth in five disadvantaged urban neighborhoods of Port au Prince” reaching about 10 percent of the city’s population. The total estimated project population includes 218,490 residents including 32,555 children under five years of age and 53,967 women of reproductive age (15-49 years). The Intermediate Results encompass the strategies and activities required at the household, neighborhood, health service and political level: 1. Empowered communities with increased knowledge and interest in maternal, child and youth health promotion. 2. Enhanced availability of and access to reproductive and child health services for disadvantaged households in urban areas. 3. Increased quality of reproductive and child health services in selected government and private non-profit health centers. 4. Improved policy environment for the urban populations, putting emphasis on protection for the poorest people. Priority interventions and relative total levels of effort targeted are: HIV & AIDS (20%), maternal & newborn care (20%), control of diarrhea disease (25%), nutrition (15%), pneumonia case management (10%); and immunizations (10%). The project is implementing interventions at both the health facility and community levels. At the facility level it has trained providers in the Integrated Management of Childhood Illness (IMCI) as well as the Community Management of Acute Malnutrition (CMAM). It has also strengthened management systems for essential drugs and supervision. At the community level, the project has recruited and trained of over 500 youth volunteers who are reaching 15 households apiece for health education, promoting care of sick children, and tracking vaccination drop outs, as well as promoting condom use. 27 CBOs are actively providing support to the volunteers as well as assisting with broader community mobilization for child survival. The project has also trained 26 community facilitators in more depth on both child survival and dialogue facilitation, and has established 70 community distribution points for ORS and condoms, although restocking these remains problematic in some zones. These have been achieved despite significant skepticism both that the household level could be reached and that it could be done with volunteers. 4 With the exception of condom use, the project has not yet made significant progress towards its objectives. Its focus has been on the establishment of the structures more than on clearly prioritizing and promoting the desired behaviors. Its actual coverage at the household level remains limited in approximately half of the zones, while beginning to improve in zones where the CBOs and youth volunteers are more active and working together. In addition to the predicted challenges of working in an urban environment, the project has faced significant violence and insecurity. Staff were unable to go into Descayette for eighteen months and concern for staff security has curtailed activities in general. With three different implementing partners, four different community cadres, and six interventions, developing a coordinated vision with all the components moving together has been an additional challenge. Other constraints have been the lack of senior leadership due to difficulty in recruiting the Health Coordinator in Concern, and the lack of stability and security in the country at large. The global food crisis and recent hurricanes have also greatly exacerbated the extreme poverty in these communities, making it even more difficult to develop a mobilization structure based on volunteers. The overall conclusion of this evaluation is that this project has made significant progress in developing a model for child survival implementation based on volunteers in the extremely difficult conditions of urban Port-au-Prince. However, it has fallen short of achieving cohesive implementation of that model across partners, or among the different community-level cadres. It has also not yet focused adequately on specific messages to achieve the desired impact at the household level. Key recommendations focus around: 1. Strengthen the model for community mobilization by focusing on the strong zones where both CBOs and youth volunteers are working well and by improving the coordination between the different community workers. 2. Focus the behavior change strategies by prioritizing and limiting the health promotion messages and by focusing on the key child survival interventions. 3. Focus health service strengthening efforts on the two health centers where both IMCI and CMAM are being implemented. 4. Reinforce the teamwork of the three implementing partners in developing a vision for the next two years of the project, a belief that it is possible, and commitment to the cause of access to health for the urban poor. A complete list of compiled recommendations is included in Annex 12. 5 Inputs Activities Outputs Outcome Goal BEHAVE strategies and Reflection Circle Dialogue Modules Orient staff and partners to use of BEHAVE Framework and develop strategies to address 13 emphasis practices 2 monthly reflection circles by each of the 26 trained facilitators Youth volunteers Training 1136 youth (to date 558 trained, 34 more recruited and 474 active)and organizing monthly sessions in all 5 neighborhoods Youth volunteers screening and referring sick and malnourished children for 15 HHs Mobilization of existing community based organizations Establish multiple distribution sites accessible to parents and youth for ORS, Jif, condoms (to date 70 distributers) CBOs implementing community health promotion activities Essential commodities of ORS, JIF, condoms, essental drugs, RUTF Capacity building of CBOs (planning, communications, collaboration) At least 20 distributors stocked with essential commodities per nghood IMCI training curriclum, algorithms, essential drugs management, quality improvement . CMAM training Trained 14 providers in IMCI SWOT analysis to quality improvement in 5 HCs, COPE approach in St. Martin II. Health workers providing correct integrated case management for sick and malnourished children, and treatment for malnourished children. Support for MSPP DSO and Bureau communale IMCI training, HC supervision, Reliable supply of essential meds and vaccines at five health facilities. Regular supervision KPC and HFA data Establish regular links with UNICEF, WHO, UNFPA to ensure coordination of programs and subsidized services Urban health learning and action exchange IR 1: Empower ed communit ies with increased knowledg e and interest in health promotion IR 2: Enhanced availabilit y of and access to health services. IR 3: Increased quality of health services. IR 4: Improved pro-poor urban health policy environme nt. Improved preventive child health practices: Exclusive breastfeeding / Full vaccination by first birthday Improved care for sick child Careseeking for pneumonia Maintained feeding and increased fluids for sick children Improved maternal and newborn care ( recommended awareness of danger signs and birth planning) Enhanced youth HIV/AIDS protection Contraceptive use by youth Consistent condom use by sexually active youth A complete review of the work plan is included in Annex 14. 6 Assessment of Progress - General Overview Concern Worldwide Haiti (Concern) has been implementing a five-year USAID Child Survival funded project called, “The Urban Health Project for Five Disadvantaged Neighborhoods in Port-Au-Prince, Haiti” since 2005 in partnership with the Ministry of Health West Department, FOCAS, and GRET. In St. martin it also works with Kowodinasyon pou Development Sen Maten (KDSM), a federation of six Community Based Organizations (CBOs) in Saint Martin. The goal of the project is to reduce maternal and childhood mortality through improved health service provision and usage within five slum areas of Port-au-Prince, reaching 24,000 children under-five and 52,500 women of reproductive age. Priority interventions and relative total levels of effort targeted are: HIV & AIDS (20%), maternal & newborn care (20%), control of diarrhea disease (25%), nutrition (15%), pneumonia case management (10%); and immunizations (10%). The major project strategies include: • Strengthening the quality and range of government and non-profit health clinic services • Building family and community capacity to prevent unnecessary illness and death • Increasing the capacities of key Ministry of Health structures and of partners to implement, integrated, community-based health projects in urban settings The diagram below represents the different project elements that act on parents’ and youth’s attitudes and practices. The project uses these elements to provide a range of reinforcement for the priority behaviors as well as aiming to enhance coordination between them. The community based organizations become the link that can provide coherence despite the difficulty of achieving a sense of community in the urban setting. 7 Parents / Youth Participatory Education (dialogues) Youth volunteers Distributors (access) Radio/Media Mass mobilization – cultural activities Provider Advice CBOs The project area comprises five urban slum areas of Port-au-Prince, in Haiti, with a total population of 218,490. The number of total direct beneficiaries is estimated to be 85,169 (24,000 children under five years of age and 52,500 women of reproductive age). The beneficiaries live in slums characterized by unplanned urbanization. Table 2 summarizes population estimates for the five slums. Concern is implementing this project with two partners: 1) GRET which is responsible for Descayette and is working with one CBO partner (SNELAK) and implementing all of the project interventions; and 2) FOCAS which is responsible for Jalousie and Bois Moquette and is only implementing the HIV/AIDS interventions. As of October 1, 2008, the FOCAS project will be turned over to Food for the Hungry(FH) which will complete the interventions. Concern is responsible for all of the interventions in St. Martin and Cite Okay. 8 Table 2: Estimated project population Project Area Partner Commune Total Population Intervention 1 St. Martin Concern Delmas 75,000 All 2 Cite Okay Concern Delmas 25,000 All 3 Descayettes GRET Port-au-Prince 50,000 All 4 Jalousie FOCAS Petion-Ville 54,758 HIV/AIDS 5 Bois Moquette FOCAS Petion-Ville 13,732 HIV/AIDS Total 218,490 Source: Project proposal, 2004; Population of Dekayet – GRET estimate 2006. Constraints / Recommended DIP Adjustments In thinking about project progress, it is essential to first remember the considerable contextual constraints this project has faced. The environment in Haiti is significantly more unstable than it was at the time the project was designed. Initially and continuously, security has been an overwhelming problem. In Descayette, the project was unable to go into the neighborhood from October, 2005 – March, 2007. Experiences with gunfire during surveys have also led to security concerns such that staff are limited in their ability to circulate in any of the neighborhoods. In addition, the global food crisis has hit these communities hard. There were food riots in April, 2008, malnutrition is increasing, and other health priorities are falling behind the urgent concerns around food security. In addition to these problems, four hurricanes swept through the country in the two weeks prior to the evaluation, the government had just appointed a new prime minister after 5 months without such leadership, and the project has had significant difficulties in recruiting and maintaining senior health staff to provide leadership and vision for the work in the communities. As several people pointed out in the course of the evaluation, if project indicators are even maintained at the same level, it is a significant accomplishment given the deteriorating conditions in the project communities. Despite these constraints, the project has managed to stay focused on its goal. They have made significant progress in the mobilization and training of community structures, have provided IMCI training and management support in selected health centers, and introduced seventy community distribution points for essential C-IMCI products. The midterm evaluation is recommending a couple of changes in the current DIP. The first is to decrease the emphasis and expectations for the MNC interventions, and the second is to focus the community mobilization efforts on those sub-zones that are working well in order to refine and consolidate the model. This will necessitate adjusting some of the project targets downwards. The MNC recommendation is discussed in more detail in the section on MNC, and focus on sub-zones is discussed in the section on community mobilization. The adjustments in the objectives are discussed in the 9 conclusions section and a table with the proposed revisions for the objectives is included in Annex 12. Highlights from M&E Table Note: The complete table is attached in Annex 4 Baseline, Jan-Mar 2005 Midterm, June 2008 Indicators % % Midterm Target End of Project Target Explanation/ References Maternal and Newborn Care** Proportion of mothers of children age 0-11 months who know at least 2 signs during delivery and postpartum requiring immediate medical attention* 34% 42% 40% 50% On track. Enhanced youth HIV protection Proportion of sexually active youth aged 15-24 years, who are not in a stable relationship of one or more years who use a condom consistently for past three months 12.6%(1) 47% 15% 20% Surpassed target. Baseline not specific to project area. Proportion of female youth aged 15-24 years who are using a modern contraceptive* (baseline Aug 2007) 28% 40% 40% 55% Change is significant and on target. Preventive Child Health Proportion of mothers of children age 12–23 months who purify drinking water 31% 27% 40% 50% No significant change. Economic barriers. Proportion of children age 12–23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday 49% 56% 65% 80% No signficant change and off target. Proportion of children age 6–9 months who received breast milk and complementary foods during the last 24 hours 45% 47% 50% 65% No signficant change and off target. Care of Sick Child Proportion of children less than two years old with diarrhea in the past two weeks who received oral rehydration solution 55% 53% 60% 70% No signficant change and off target Proportion of mothers of children age 12–23 months who know at least two signs of childhood illness that indicate the need for treatment* 33% 54% 45% 60% Change is signifiant and on target. *significant change ** The evaluator recommended removing other MNC targets due to need to focus the project efforts during the remaining time [1] 2005 Demographic Health Survey for youth in metropolitan Port-au-Prince 10 The only indicators that show significant impact at this point are the contraceptive use by youth, and the knowledge of danger signs. While the only baseline for condom use is the DHS, it also seems that condom use has significantly increased. These results reflect the strength of the youth volunteer efforts, as well as the current project focus on sharing information. However, it is clear that overall impact is still very limited, probably due to the lack of penetration in the communities at large. There are still a significant number of households that are not covered by the youth volunteers and that have not yet had any contact with the project. It may also be due to a lack of clarity about the key messages and behaviors that are project priorities. Work Plan Status Note: The complete table is attached in Annex 14. Objective and Activities Achieved Status and comments IR 1: Empowered communities with increased knowledge and interest in maternal, child and youth health promotion Establish behavior change strategy Training of staff and partners on behavior change strategy planning using BEHAVE and Barrier Analysis Yes Training completed in May 2007 with consultant. Developed complete frameworks in Dec 2007 Develop BEHAVE strategy for safer sexual practices among youth and pregnancy prevention, MNC danger signs and PP visit, care of sick child, treatment of drinking water, vaccinations, and exclusive and complementary breast feeding. Yes, except MNC still incomplete Done for sexual practices, care of sick child, treatment of drinking water, vaccinations, and exclusive and complementary breast feeding including doer-non￾doer analysis. Not yet completed for MNH. Adapt and develop BCC materials for HIV, sick child care, vaccinations, infant and young child feeding and MNC Yes, except MNC. Materials for ORS developed. Module and facilitators guide completed for dialogues. Draft completed for sick child care. Train 26 community members (CBOs, youth volunteers, health agents) as Facilitators using adult learning, dialogue approach Yes Trained during one day per week for 10 weeks in November(?) 2007 Community Mobilization Work with CBOs and Youth Groups to identify, train and support Youth Volunteers for health promotion within the sub-zones Yes Selected and orient 588 out of the originally planned 1137. 438 youth volunteers active CBOs and youth volunteers organize community meetings on MNC danger signs and post partum visits, to plan emergency support plans for hospitalization No Postponed in year two 11 Objective and Activities Achieved Status and comments CBO & Youth Volunteers organize community dialogue on HIV, C-IMCI practices including vaccination, infant and child feeding, and water and sanitation. Yes 26 Facilitators organizing groups of 25-40 parents on two modules per month (total 9 modules) Orientation of Youth Volunteers and CBOs on CMAM screening Added 1/2 day orientation and follow-up supervision. IR2: Enhanced availability of and access to reproductive and child health services for disadvantaged households in urban areas Identification, selection, and orientation of distributers for condoms targeting youth aged 15-24 years with Youth Clubs Yes 70 YVs as distributors, volumes of condoms distributed in year 3: Cite Okay 17,408; Descayette 4,508; Jalousie 16,438; Bois Moquette 33,018. Over 52,000 condoms were distributed in St. Martin Water and Sanitation Committees conduct situation analysis on opportunities for public hand washing stations and distribution points for ORS and Pur/Chlorine Yes Establish handwashing sites at family latrines at Descayettes, 16 St. Martin and 12 Cite Okay Training on Essential Drugs Management Yes Completed in year one Reorganization of Vaccination register by Sub-zones to facilitate tracking with CBO & Youth Volunteers Yes Changed strategy so JVs checking EPI cards of all children in their coverage area Procure posters, algorithms on IMCI in the health centers Yes HFA 2008 indicator all 3 HCs have posters and algorithms clearly posted Train and initiate associationTBAs. Establish social marketing of clean delivery kits No Postponed in year two IR 3: Increased quality of reproductive and child health services in selected government and private non-profit health centers Training health workers in St. Martin, Cite Okay & Descayettes on IMCI Yes December 2007 Training of Quality Assurance Facilitators Modified Reflection on IMCI services by trainees, problems and solutions identified, and action plans set. Follow up on plans underway. COPE process begun in St. Martin II. IMCI quarterly joint supervision to 3 health centers Yes Project doing regular supervision, MSPP participation limited. HC staff deliver quality information on growth monitoring, acute malnutrition screening and nutrtion counsellng Yes Trained HC and BC staff on CMAM and nutrition counselling. Growth monitoring done during vaccination sessions. Counseling training during IMCI training. 12 Objective and Activities Achieved Status and comments Collaborate with MSPP on refresher package for prestataires on antental care with HIV/AIDS integration and post partum care No Postponed with MNC component. Could still do this in St. Martin with HIV project. Updating standardized HMIS system with MSPP for all health facilities and training staff in its use Added Completed in year one. No review of BC and DSO info systems. Management meetings take place quarterly but data very limited. IR 4: Improved policy environment for the urban populations, putting emphasis on protection for the poorest people. Quarterly Urban Platform meetings No MSPP DSO was not clear on the purpose, nor on who or how to convene such a group. Need to pull together more of the project stakeholders to advance urban health programming conversation. Engage with MSPP in the development of the national C-IMCI strategy with particular accent on malnutrition Modified No national effort underway in this area. Discussions held with CMMB who are piloting the approach. Raised and discussed at USAID CSHGP quarterly meetings. Linking effort with CMAM intervention and food crisis. Host national reflection on management of severe malnutrition and orientation to community based therapeutic care approach Yes National nutrition committee established, CMAM services established for all neighborhoods under two￾year Goldman/UNICEF funding Participate in global urban health forum and bring best practices and lessons learned into national discussion Yes Clinton Global Initiative, ICUH participation and presentations, exchanges with Bangladesh It is notable that the majority of activities for the MNC interventions have consistently been postponed due to the complexity of the project and the corresponding work load. The other lag is in training the projected 1136 youth volunteers, although more feasible alternatives for coverage are also being considered. However, much of the work plan is on schedule, despite the significant constraints the project has faced. Assessment of Progress – Technical Approaches CDD The project developed a comprehensive approach to addressing diarrheal disease including an emphasis on water treatment, hand washing, infant hygiene, diarrhea case management including continued feeding and fluids, and increased access to and use of ORS for diarrhea treatment. It did this through IMCI training at the health center level, and using their community health promotion structures (CBOs, youth volunteers, and community facilitators) to promote these behaviors at the community level and to follow up individual children with diarrhea at the household level. 13 There has been an increase in the recognition of danger signs, as well as in the availability of ORS at the community level. There is also an increased awareness of the importance of clean drinking water. However, there are a couple of significant challenges: • At the beginning of the project, community distributors were able to get an adequate supply of free ORS (funded by UNICEF). This is no longer available so the project has replaced their supplies with socially marketed ORS obtained from PSI. People are resistant to purchasing a product they believe should be free. This is an example of why even maintaining the baseline level of ORS use is an accomplishment. • The indicator for clean water measures the treatment of water. It is confounded by the availability of cheaper, supposedly treated, water from trucks in the neighborhoods. When broken down, just over 50% of people are buying water that is supposedly treated, and 38% of those report also treating it. IF the purchased water is clean, this is an unnecessary expense, and if it isn’t, then more people need to treat their purchased water. The cost of water in general is also a significant inhibitor for hand washing. Recommendation: Test water being sold as treated water in the communities to be sure it is clean. If yes, adjust messages accordingly since it is cheaper than bleach treatment. If not, continue to promote bleach treatment, but letting people know the water in the trucks still needs treatment. Nutrition Note: Refer to Results Highlight at back of report for more detail While nutrition was not part of the original proposal, it was added to the DIP due to the high rates of malnutrition and the felt need found during the project’s baseline research. The project did an additional survey more carefully assessing nutritional status and feeding practices, and applied for additional funding to implement a complementary Community Management of Acute Malnutrition (CMAM) component. The project benefited significantly from Concern’s organizational commitment and experience with the CMAM approach, and the introduction of a nutrition intervention which responds to the ever increasing food crisis has energized the other child survival interventions. The nutrition component uses the youth volunteers to do household screening of all children under five and to refer those who are severely malnourished. Health center providers have been trained in the assessment and treatment of severe acute malnutrition without complications and to refer cases with complications to the hospital. At the moment, numbers in the program are still low but the model for intervention seems to be well received by both staff and community. The next phase of the project will focus on improving coverage and Concern is applying for additional funding to add treatment of moderate acute malnutrition to the existing services. 14 Despite the strength of the CMAM intervention, it is not yet well integrated with the other IMCI / child survival messages and interventions. At the community level, the messages promoting exclusive breast feeding, complementary feeding, and Vitamin A are not well understood and don’t seem to have been integrated with the CMAM screening and follow up. At the health center level, it has been difficult for the IMCI approach to gain traction and consistent integration of the CMAM intervention has suffered as a result. This is likely due to challenges with implementing IMCI consistently in the face expectations for incentives and staff turnover. Recommendations: • Drawing on the BEHAVE framework and the IMCI training, integrate the child survival messages and interventions with the CMAM intervention at the community level. Reinforce IMCI at the health center level, including integration of malnutrition case management. • In collaboration with the other nutrition efforts, work to develop strategies to strengthen livelihood options and support groups for mothers of children both in the CMAM program and those discharged from nutrition treatment. HIV/AIDS The youth volunteers do peer outreach and education for HIV/AIDS prevention. In addition, the project has emphasized access to condoms at the community level through community distributors. Condom use and adolescent use of contraceptives are the two indicators that have apparently improved the most, although the majority of contraceptive use is condoms, indicating there is a gap in the promotion and uptake of more effective contraceptive methods. There are several challenges with this intervention. The project justified its focus on the prevention aspects of the intervention, arguing that other programs and funders were covering other aspects such as counseling and testing and PMTCT, yet they have not effectively coordinated with these other services. In the FOCAS areas there are conflicting messages between another project focusing on abstinence and being faithful and the child survival project that also promotes use of condoms. The project also expected to address stigma and the needs of persons living with AIDS but has not particularly done so. These issues have not been integrated into an overall HIV/AIDS strategy with well developed behavior change messages and approaches. Finally, the project partner structure contributes to this shortfall. For Concern and GRET, their primary focus has been on the more core child survival interventions and they haven’t had the time or ability to extend significant effort to these challenges. FOCAS, given that the HIV/AIDS component is its only intervention might have been the one to more aggressively address these constraints. However, they have been overextended and their implementing partners have lacked the vision and leadership to meet this challenge. This will hopefully change with the shift in responsibility to Food for the Hungry. Recommendations: • FH should take the lead in strengthening the HIV/AIDS package of interventions: 15 o Clarify and harmonize messages between those promoting abstinence and faithfulness with those also promoting condoms and dual contraception. o Integrate an approach to addressing stigma through building on the CBO, facilitator, and youth volunteer structures established by the project. o Integrate the project efforts on prevention and stigma with the other HIV interventions, including support for those living with HIV/AIDS, which are supported from other sources. ARI / EPI The ARI and vaccination interventions are implemented under IMCI. To date, the project has focused more on vaccination coverage, promoting the vaccination calendar and using youth volunteers to track drop outs. With the baseline survey indicating a reasonable level of recognition of danger signs and care seeking for pneumonia this has been less emphasized, although community members expressed interest in more information. The greatest impact on vaccination coverage occurred in St. Martin where health agents (paid by MSH) were also coincidentally introduced through the MSPP with other funding. They extended access through outreach sessions which complemented the youth volunteer efforts. Otherwise, as with the other interventions, lack of penetration and coverage is the greatest challenge for these interventions. Recommendations for coverage are included in the cross-cutting section on community mobilization. MNC The project originally proposed to develop a complete intervention package for MNC including promotion of danger signs, prenatal and post partum care; identification and training of TBAs, and strengthening of peri-natal services at the local health centers. However, the revision of the TBA training curriculum has been entangled in politics, the health services strengthening has had more than it could handle with reinforcing IMCI, and the project has postponed the formative research required for developing the community level promotion package. The constraints of Haiti, overall complexity of the project, and complexity of a comprehensive MNC approach all contributed to this postponement. The fact that none of the project health centers actually does deliveries also means the original strategies would need to be reconsidered. At this point, while project and MSPP staff are reluctant to drop the intervention altogether, it is clear that the project is not in a position to complete the intervention as originally planned. As a result, while not ideal, the team developed a recommendation that would focus on the delays in getting urgent obstetrical care at the community level while not getting involved with strengthening the health facilities. This should allow for the maximum benefit from the synergies with the existing community structures while not significantly increasing the project’s work load or complexity beyond its current level. In addition, at the request of the MSPP, the project will continue to promote pre and post natal care, but also will not get directly involved with improving the quality of those specific services (beyond the general strengthening associated with the IMCI and 16 quality improvement approaches they will be pursuing in two health centers). The proposal for adjusting the MNC objectives and targets takes these shifts into account. Despite the project’s inability to comprehensively address MNH at this time, it might be possible to begin to collect some qualitative stories of near misses (or verbal autopsies) to clarify the key enhancers and barriers to safe deliveries in the urban setting. This could contribute to an understanding of how an urban maternal health package should differ from that in a rural health community, given the differences in the availability of and distances to tertiary services. This could provide the basis for future work in developing an urban health model specific to maternal health needs. Recommendations • Focus maternal health activities on behaviors around safe delivery at the household level (addressing delays in recognizing danger signs, deciding to seek care, and getting to services) rather than on the service strengthening and capacity building interventions. o Continue to promote knowledge of danger signs, and attendance for pre and post natal care. o Add promotion of birth preparation in case of complications including preparing savings for transport, identification of the hospital, and clarification of who will make the decision to seek emergency services. • Develop qualitative documentation of women with complications who did well or who did not in order to clarify the real gaps and most appropriate solutions for improving MNC in an urban setting, to be used in the future for further elaborating the urban health mode.. Assessment of Progress – Cross-Cutting Approaches It is the cross-cutting approaches and their application in the urban setting of Port Au Prince that make this project interesting and challenging. The project takes on the challenge of targeting and measuring its impact on the poorest third of the population in the slums of Port Au Prince, and it does so by using community based organizations and youth volunteers to reach people in communities that are fluid, not clearly defined, and don’t have the traditional leadership structures of rural villages to depend upon. Consortium of Three Partners The decision to implement this project through three partners increased both the challenges and the potential impact for this project. Concern had a model for urban health they wanted to test with other implementers. Leadership in both Concern and GRET are committed to developing the urban health agenda and the partnership enhances this goal, involving more players and bringing more voices to advocacy for implementing health programs reaching the urban poor. Finally, each partner brings different skills to the partnership: Concern had experiences with child survival in urban settings, FOCAS had experience with child survival in Haiti which Concern hoped would contribute to the basic implementation process, and GRET had a track record working with a community based organization in the area where they are implementing the project. 17 The partners acknowledge they have benefitted from being part of the partnership. Participation has involved technical capacity building in formative research and behavior change strategy, monitoring and evaluation, and strategies which actively involve youth in the health of their communities. The partners also benefitted from an exchange visit to Bangladesh so that everyone could share a common vision for their work. For Concern, the biggest disadvantage to the partnership has been the loss of control over assuring the quality and timeliness of project implementation and reporting. The other partners seem to have different levels of commitment to the partnership and different levels of capacity for implementing their commitments. This has made it hard for the Concern Project Manager to develop the three partners into one team working together. It is hard to get consistent participation in meetings and difficult to get reports on time. Both of the partners are stretched in terms of staffing, field staff are pulled off of their job for other tasks, and field staff feel they don’t get the support they need. These problems are exacerbated by FOCAS’s not having the full package of interventions which complicates both the model and the implementation discussions. Up to now, the project has failed to mine the potential synergies and advantages the partnership would offer. However, with the transition from FOCAS to Food for the Hungry, and with the success of GRET’s efforts during the past six months to make up for the lost time as a result of the security situation, the project is in a position to make a renewed effort to enhance benefits of this partnership. Recommendations: Strengthen the vision and coordination of the three partners to contribute as one team: • Work as a team to develop a vision for the next two years of the project, a belief that it is possible, and commitment to the cause of access to health for the urban poor. • Enhance more equal participation and ownership on the part of all three partners through sharing responsibility for the monthly meetings and regular exchange visits to share the experience and learning of different partner approaches. • Develop reporting lines across agencies that encourage direct reporting from project field staff to the Concern Project Manager. In addition, senior staff need to free up and support field staff in the carrying out of their responsibilities. • Use financial and activity reporting systems to hold all partners accountable for their commitments. • Work with partners to more explicitly identify the special contributions each can make to the partnership. Examples include FH working on the HIV interventions as well as assisting with monitoring of behavior change, while GRET might contribute to the model for reinforcing and motivating the CBOs and the facilitators. 18 Community Mobilization This project has drawn upon experience in Bangladesh to further develop a model for community mobilization and reaching the household level in poor urban settings. In the absence of the traditional leadership structures found in rural areas, or functional local municipal government, identification and organization of local community based organizations to work for improving health in the community along with recruitment, training and support for youth volunteers form the core of this model. Out of these groups, local distributors for health products such as ORS and condoms, and facilitators who have additional training to lead community discussions on child survival topics are selected. All these cadres form a series of hopefully interlinking structures that help establish social cohesion around health. There are a total of 27 active CBOs involved with the project, with a range of 2 – 10 per zone. The CBOs provide the prioritization, guidance, and support for the other health activities. They recruit youth volunteers, they have become the people community members consult with when there is a health problem, and they assist with community￾wide health activities. However, they are constrained because they also depend on volunteerism, there is a wide range of commitment and capacity among them, and they tend to have high expectations of the project. In one zone, Concern has invested heavily in creating a CBO federation (KDSM). This has led to significant dependence on Concern and has not been terribly successful. As a result, in the other Concern zone they are trying to allow the linkages between the CBOs to develop with less intervention. In the GRET zone, they are working with one strong CBO which seems to be working very well. Youth Volunteers Even though there was significant skepticism at the beginning, this project has managed to develop a structure which actually reaches the household level, and which uses youth volunteers to do it. Many of the youth expressed a lot of enthusiasm for what they are doing, and the addition of the nutrition screening and referral surprisingly enhanced the momentum because it gave them something concrete where they could make a difference. They also recognize benefits like working with their friends, collaborating together to cover their zone for nutrition screening and suggesting that: “I became more outgoing as a result of being a volunteer”. “One of the most important lessons I learned from the training was the importance of my own self-esteem.” This said, motivating the youth volunteers is probably also the biggest challenge. Many receive pressure from their parents to be earning money for their activities and they are enthusiastic as long as the job continues to be challenging and interesting, requiring constant “updating” of the activities and content. People don’t see or acknowledge the non-monetary benefits of being a volunteer. Finally, the CBOs could potentially be given 19 a more active role with guidance from project staff, including financial resources, for youth volunteer management as part of their annual action plans. Part of the overall motivation effort also needs to be “marketing” the benefits of being a volunteer. The table below gives an idea of some of the strengths and challenges of the volunteer program. The project has actually trained less than half of the 1156 proposed volunteers – a number based upon the number needed if every household were to be covered at a rate of 15 households per volunteer. The active volunteers exceed those trained in Descayette because additional ones have been trained on the job through working with those who were trained. The dropout rate was significantly higher in Jalousie and Bois Moquette, reportedly because the volunteers’ expectations weren’t met and because of outward migration from the areas. In addition, likely due to the implementation by second tier NGO partners, there was less of a purposeful strategy for volunteer management. This said, it is notable that there are still a significant number of enthusiastic youth in these areas. Areas # of Volunteers Trained # Volunteers active % Retained SanMartin 159 142 89% Cite okay 164 137 84% Descayette 86 120* 140% Jalousie 65 35 54% BoisMoquette 80 40 50% * The higher number is due to volunteers who have not yet been trained but who atttend meetings and participate in activities. Lesson Learned After experimenting with both older and younger volunteers, it seems that the 16 – 18 year old high school students work the best. They are old enough to have some authority, they are not from the poorest echelon where earnings are absolutely essential, and they can see how the experience enhances their education. Facilitators and Distributors 26 facilitators were trained in more depth, both for health content and facilitation skills. The idea was that they would do most of the health education through community “reflection circles” (see discussion in next section) but would be supported by the mobilization efforts of the youth volunteers. This was meant to limit the work load and the level of expectations placed on the volunteer youth. Currently, each of the partners has somewhat different expectations of their facilitators, and each of the partners has also developed different approaches to the need for incentives. In (FOCAS) the facilitators are also paid as community health workers by 20 MSH. GRET provides a stipend, recognizing that the facilitators are expected to do quite a bit of work, the project is depending on them for coverage, and there aren’t that many of them. Concern provides a pro-rated amount for snacks which has been inadequately tracked. Whichever the approach, the concern at this point in time is that expectations for incentives have been set without full and transparent consideration of their implications. 70 community distributors were drawn from the CBOs, youth volunteers, and some private business people to increase the access to ORS, condoms, and bleach for water treatment at the community level. By popular demand due to heavy presence of mosquitoes, insecticide-treated mosquito nets were added to the stock, although the malaria epidemiology would not indicate this as a priority. All distributors were provided with an initial stock by the project. However, restocking is inconsistent and systems for assuring resupply are limited. The mosquito nets were sold on credit and have no possibility for replenishment without additional inputs. The project actively seeks free condoms and ORS from the MSPP, which it supplements with PSI social marketing supplies when needed. It also purchases bleach on the local market. It has established depots in the two Concern zones with the hope that the CBOs will increasingly take responsibility for their management and resupply. GRET manages resupply through the health center, which is managed by their partner CBO, and the distributors in the FOCAS zones have not restocked at all since their initial supply in May. Despite the challenges, people at all levels feel this is an important service and should be maintained. Overarching Challenges In addition to the cadre-specific strengths and challenges mentioned above, there are two overarching challenges which need to be addressed. Despite the importance of linkages in the model, there is still a lack of coordination between the different cadres in most of the zones. There is no forum where cadres meet together regularly, CBO support for youth volunteers seems to be somewhat zone-specific and ad hoc, and CBO action plan development has not been effective nor have these action plans yet been successfully integrated with the other community strategies. With decentralization only very recent, and local officials often unclear about their roles, the project has not yet actively involved local elected officials in project activities. While it would imply a significant effort, it might also be an opportunity to involve these officials in facilitating the recognition and cohesion that would make coordination feasible. The experience from Bangladesh in involving local government could be drawn upon, although the starting point for Haiti is likely significantly lower. Finally, while the effort to limit the youth volunteer work load led to limiting their responsibility to 15 households, it means that the project has not yet reached the coverage needed to achieve significant behavior change at the household level. It is also important to note that there is significant variation in actual level of involvement and coverage between sub-zones. In areas where CBOs and youth volunteers are active, there are more volunteers and 60% – 70 % of the households may actually be covered (even at the rate of 15 households per volunteer) where in other areas the coverage is much lower because the different cadres are not active. 21 Recommendations 1. Strengthen the model for community mobilization a. Focus on the strong sub-zones where both CBOs and youth volunteers are working well to reinforce and refine the model. b. Improve the coordination between the different community workers (youth volunteers, distributors, facilitators, and CBO members) through having them work together to prioritize, develop, implement, and monitor their activities. Involve local elected leaders in this process. c. Clarify the availability and criteria for providing funds to support the community action plans. Criteria should include proposed activities contributing to well analyzed problems that are also feasible, and that draw upon local resources and solutions (not necessarily money) in addition to those from Concern. d. As a three partner team, review options for motivating CBOs (providing funding and community credibility should help) and facilitators, maintaining a realistic long term perspective and drawing on GRET’s experience with both. 2. Strengthen the management and reach of the youth volunteers. a. Experiment with increasing the number of assigned households up to 30 per volunteer in the strong sub-zones, but reinforce their efforts through encouraging them to work together and to focus their home visits on the households with children under two and pregnant women. b. Transfer skills of organizing health dialogues to youth volunteers. c. Organize the youth volunteers in sub-groups to increase their social cohesion, to encourage them to work together to reach their households, and to provide a structure for encouraging their representation in coordinating the community activities. d. Work with the CBOs and the facilitators to include a strategy for consistently supporting and motivating the youth volunteers as part of their action plans. Part of this strategy should include ways to “market” the benefit of volunteering with the youth, their parents, and the community in general, and promoting the volunteers when opportunities for short-term incentives (e.g. health campaigns) from other projects arise. It needs to be clear that these action plans can be funded by the project once the criteria are met. e. Consider linking the youth volunteers and/or other components of the model more closely with existing community structures such as Scouts, churches, or schools. 3. Strengthen access to essential products through reinforcement of distributor activities. a. Develop a reliable system for restocking supplies. In the zones where multiple products are provided, this may depend on CBOs bringing supplies from the Bureau Communal / Health center (for free supplies), PSI for (ORS and socially marketed condoms, and purchasing bleach. In the zones where condoms are the primary product the health center may be the primary 22 supplier. The project should not invest any more resources in mosquito net supplies. b. Identify the distribution points with signs and by providing lists of the points to the health centers for referrals. Behavior Change The project did extensive formative research and preparation for their behavior change strategy. This included staff training in the BEHAVE framework, qualitative focus groups and a doer non-doer survey focusing on exclusive breast feeding, complementary feeding and feeding during diarrhea. In addition, they invested in a significant amount of training for the 26 facilitators in the “Reflection Circle” approach to dialogues where community members are encouraged to explore the underlying reasons for their attitudes and actions. The dialogue approach, while falling short of the in-depth reflection approach it was modeled after, seems to have been very successful in shifting people’s approach to health promotion away from one of preaching messages towards one of participatory review with discussion on the designated topic. Participants, facilitators, youth, and staff all appreciated the new approach to health education, and one youth even suggested: “they should use this kind of approach in schools as well”. Unfortunately, the complexity of the project led to an inability to prioritize and focus the essential behavior change messages out of the many messages, interventions and behaviors they were trying to cover. The BEHAVE model, while it would potentially have been helpful with such prioritization, seems to have been more than people could absorb. It was also completed after people had started doing health promotion in the community. Finally, a few of the enthusiastic mothers expressed an interest in helping to spread messages with their friends. One staff member suggested identifying “model mothers” as a strategy for promoting the essential behaviors. Unfortunately, feeling pressure to deliver a significant number of “parents reached” with a structure that wasn’t reaching many people, staff succumbed to the community pressure to offer people snacks for attending health education sessions. This is an unfortunate approach to getting people interested in learning about their health. Recommendations: Strengthen the behavior change approach to focus messages and increase participation. 1. Reinforce the prioritization and clarification of messages according to the relative impact on child mortality, project objectives and the results of the formative research 2. Strengthen the dialogue approach to awareness raising including orienting the youth volunteers to the approach and incorporating case studies with questions and answers as the primary structure for all awareness raising activities. Develop 23 a process of identifying themes and developing new case studies based on issues that come up during these discussions. 3. Identify model mothers who are interested, enthusiastic, and can help promote the ideal behaviors, particularly for complete vaccination, use of ORS, and treatment of the sick child. 4. Experiment with strategies which make it possible (e.g. limited duration, convenient times, and engaging content) which make it possible to avoid offering snacks or other motivators for people to come to discussion sessions. Recognize that attendance may drop, but there will be interested people and this is not the only way the project is raising awareness. Capacity Building This project has done a lot of capacity building with CBOs, youth volunteers, facilitators, and health service providers. The initial training for the youth volunteers was short (3 days) with an emphasis on follow up “training” during the monthly meetings in order to be feasible given the large number of volunteers proposed. Concern did a lot of materials development for training both the youth volunteers and the CBOs, and assisted GRET with training the CBOs. The facilitator training took place over a period of weeks and was much more in depth given the expectation that they are the primary health promoters and recognizing that the dialogue approach was new and needed reinforcement. This was done by a local consulting group that was instrumental in the original development of the approach. Concern worked with staff from the MSPP in developing and implementing IMCI and CMAM training with the health providers. Given the importance of the CBOs in the overall model, Concern emphasized assessing and reinforcing their capacity. The baseline CBO capacity assessment contracted to Genesis, a local consulting firm, came up with excellent information that should be extremely useful for prioritizing training and other project interventions for CBO reinforcement. Concern’s experience and lessons learned with the formation of the CBO federation KDSM should also contribute to strategies for reinforcing the CBOs. While recognizing the benefits of affiliation for advocacy, leveraging resources, and sustainability, there has been a tendency for KDSM to be too dependent on Concern and not to seek resources elsewhere as hoped. Despite the significant effort in training, the overall training approach seemed to lack coherence and consistency. The different training components did not seem to complement each other, and collaboration in training strategies and content between the partners seemed to be the exception when such collaboration might have promoted significant synergy. Field monitoring was underutilized as a tool to assure the consistent understanding of content and priorities. Given that training will not be a priority during the second half of this project, no specific recommendations were formulated. However, these challenges should be considered if/when additional training is undertaken. 24 Health Systems Strengthening Health Facilities This project selected one health facility per zone (4 operated by NGOs and 1 government) for health service strengthening. 14 providers from all of these centers received IMCI training. Of those who were trained, there has already been significant attrition. Stock management systems were introduced, and project staff provided follow up supervision, making an effort to involve the MSPP in joint supervision by providing transport. Other funding also assisted with health center remodeling, particularly for the health center in St. Martin. As part of the CMAM interventions, providers in four of the centers were trained in management of severely malnourished children. A strengths and threats approach to quality improvement was begun in all five health centers, while a more comprehensive COPE-type approach involving all the health center staff was begun in St. martin II with assistance from the Concern HIV/AIDS program. The health center run by SNELAK in Descayette, and the health center in St. Martin II have benefitted from the synergy of both IMCI and CMAM interventions, and St. Martin II has recently had a director appointed, likely as a result of Concern’s advocacy efforts. At higher levels, the project has involved the communal level with both training and supervision, and was recognized at both the communal and departmental levels as communicating well and being truly collaborative with the MSPP in their efforts. Despite significant effort, it has been difficult for the project to feel like it was making progress with the health centers. The level of MSPP participation has been sporadic, one of the health centers (HaitiMed in Cite Okay) was (and still is) not accredited in spite of significant project efforts, and the support for the two FOCAS health centers has been weak. In many cases, it is difficult to see how the services have improved. Recommendations: Focus health service strengthening on the two health centers where both IMCI and CMAM are being implemented: 1. Using a COPE-type approach (such as was begun with St. Martin II), identify the most significant challenges to quality, reasonable and feasible action plans, and a committee for monitoring progress. Like with the community, these action plans need to be feasible, mobilize people to solve their own problems, and focus on problems that CAN be resolved (low-hanging fruit). The approach also needs to start with 1 – 2 key problems in order to establish a track record of success. 2. Consider involving representatives from the community in the quality improvement committee in order to strengthen the link between the health center and the community, and to increase mutual accountability for quality. 3. Use the framework established in the quality improvement approach to consider criteria (if needed) for performance incentives. These could be administered (with support) by the quality committee. 4. Encourage someone from the communal level MSPP to be involved and provide the necessary “weight” behind changes that may be difficult to make. 25 5. Coordinate training and supervision to reinforce the integration of IMCI with CMAM – assuring nutrition screening for all children under five and assuring the IMCI protocol is applied to children seen for nutrition rehabilitation. Policy and Advocacy One of the strengths of this project is Concern’s organizational commitment to urban health and reaching the most poor. They have brought both experience and vision to the effort, clearly considering policy and advocacy as key to enhancing their interventions in the communities. The original plan was to establish a Platform that would involve decision makers from the zone and district levels to share experiences and ideas, and to advocate for development of an urban strategy. The project also supported the participation of the Director of the West Health Department in the global Urban Health Conference. Unfortunately, the Platform had difficulty gaining traction. The Director wasn’t clear who should be participating nor what the purpose actually was, so was slow to support the effort. However, he indicated he is still interested in the cause and would be willing to work on it. Concern’s positive track record in communicating and collaborating with the District also helps with this effort. As a result of discussions during the evaluation, it seems there may actually be two goals for the advocacy effort with a need for different approaches. The goal of encouraging development of a national strategy for urban health is probably better met by developing informal networks of interested people rather than through a formal structure. Meanwhile, the goal of providing an opportunity to exchange ideas and a structure to support the community level efforts of the CBOs might be better met through a lower level type of coordinating committee. Recommendations: Develop structures and advocacy strategies to enhance broader support for the community structures as well as to promote the urban health agenda. 1. Develop opportunities for exchange across zones for learning and support. This might evolve towards an informal network of CBO leaders and MSPP, but should not « belong » to Concern. 2. Develop connections with other people in Haiti who are also interested in promoting the urban health agenda in order to advocate for policies that support an urban health mode. Scaling Up This project targeted nearly a quarter million people drawing on experience from Bangladesh to develop a model that was new to Haiti. It purposely decided to work with two partners to help assure the replicability of the model as well as to add scale by building on existing efforts and structures developed by those partners in the areas they work. The project also took on the challenge of developing a model for reaching the 26 household level while working at scale, an important strategy in a context where many people are marginalized for different reasons. This said, the project is still in the learning phase of the model development. While the experience and lessons learned will contribute to the vision for an urban health strategy, the model is not yet ready to be replicated more broadly without further work. Equity This project, and Concern as an organization, has made a significant commitment to assuring their interventions actually reach the poorest segments of the population. The selection of the urban slums of Port Au Prince as a target is the first evidence of this. In addition, Concern has developed a methodology for analyzing LQAS data segmented by the lowest third of the population as defined by the variety of food they consume. This analysis allows the project to target the poorest segment of the population according to the most risky behaviors they practice. In addition, it allows the project to assess whether its interventions are also reaching this segment of the population. While the data and program understanding allow for significant segmenting and targeting of the poorest population, this capacity is not yet reflected in the program approach. The project could potentially use this information to advantage as it works towards increasing the focus and effectiveness of the youth volunteers. Sustainability The project carefully planned for sustainability using the Sustainability Framework and identifying sustainability indicators during the planning process. At every point along the way, it has made design decisions that would not threaten sustainability in the future. The model is structured to maximize potential community cohesion and CBO support for the youth volunteers, to maximize links between the community, health services, and the MSPP, to reinforce independent action planning and action, and to minimize dependence on Concern or other short term resources. In addition, by targeting secondary students as youth volunteers and CBOs with members who have other livelihoods, they explicitly tried to identify a population that might have more time, energy and interest in being volunteers rather than those who are worried about food for their next meal. This said, conditions in Haiti are deteriorating, health service support is decreasing, the global food crisis is challenging everyone’s ability to eat in these neighborhoods, and it is unlikely that this project will achieve sustainability by its end. As a result, it will be important for management to begin planning for follow-on funding early as they continue to refine and reinforce their intervention package. Mission Collaboration Collaboration with USAID has been excellent in both Port Au Prince and Washington. The evaluator was encouraged to contact the Washington DC CSHGP Technical Advisor who was unfortunately too busy to actively provide input to prior to the evaluation; however, she participated in a briefing in DC on October 1st. In Haiti, the person responsible for child survival was recently on maternity leave. However, she is 27 enthusiastic about child survival, organizes a regular coordination meeting for PVOs with child survival grants, and appreciates the leadership Concern has offered. In addition, the Mission has played a role in coordinating target areas and interventions between the PVOs and the local bilateral managed by Management Sciences (MSH) for Health. Unfortunately, there are still some discrepancies in approach between MSH and the PVOs in community level interventions and incentives. The Mission might be able to take a larger role in facilitating discussion around these differences – particularly their implications for an urban health strategy. Contextual Factors As mentioned throughout this report, the contextual factors have offered significant challenges to this project. • The lack of security has intermittently prevented project staff from even entering target areas, and it significantly limits staff’s ability to move freely within the communities. • Decreasing funding for routine health services means there are less free condoms and ORS available, requiring people who are used to having these products for free to now pay for them. In addition, donor support for health services has also decreased in most of the zones. • The food crisis has significantly increased the pressure people in the target communities feel, thus also increasing their pressure to devote volunteer time to earning money. The midterm survey confirmed this by showing that people’s consumption of food varieties had significantly decreased. • The four hurricanes that passed through Haiti prior to the midterm evaluation drew resources, both human and financial, off the development efforts addressing chronic poverty towards the emergency. Hopefully, this might assist with leveraging additional resources for development in the long run. Concern was submitting two additional community nutrition management proposals which will allow them to expand their screening as well as to add interventions for the moderately malnourished children as well as the severe ones. • Finally, the lack of senior program staff was the biggest management challenge for this project. It is difficult to recruit skilled international staff in the face of deteriorating security and living conditions. Conclusions and Recommendations In conclusion, this project was ambitious in its effort to develop and implement a model for urban health which reaches the household level and targets the poorest households. The effort was further complicated by the decision to involve the two other partners, with different levels of capacity and commitment. 28 Despite the many constraints, the project has successfully moved forward with developing both the community mobilization structures and behavior change priorities for most of the interventions. To varying degrees, it has accomplished this among all three partners. At this point, the linking of the different components and “moving parts” of this project into a coherent and “well-oiled machine” is what is lacking. This is the case among the different cadres at the community level as well as among the implementation partners. Key recommendations focus around: 1. Strengthen the model for community mobilization by focusing on the strong zones where both CBOs and youth volunteers are working well and by improving the coordination between the different community workers. 2. Focus the behavior change strategies by prioritizing and limiting the health promotion messages and by focusing on the key child survival interventions. 3. Focus health service strengthening efforts on the two health centers where both IMCI and CMAM are being implemented. 4. Reinforce the teamwork of the three implementing partners in developing a vision for the next two years of the project, a belief that it is possible, and commitment to the cause of access to health for the urban poor. Specific recommendations have been integrated in the preceding text. A compiled list of recommendations is also included in Annex 11. The recommendation to focus efforts on sub-zones where the model is working relatively well, and the recommendation to limit the interventions in maternal health to the community level delays (danger signs and birth planning) lead to the need to adjust the project objectives and targets. The MNC objectives with the exception of knowledge of danger signs should be dropped, while adding a new objective on birth planning. Otherwise, the target levels for the child health behaviors should mostly be lowered in order to be realistic given the challenges in achieving coverage that were outlined previously. A table with proposed revisions in the indicators and targets is attached in Annex 12. Action Plan Field and senior staff from all three implementing partners as well as from the zone and district MSPP participated in a one day session debriefing the evaluation and working on developing an action plan for the next few months. Everyone agreed on the need for the partners to strengthen their efforts to work together, on the need to increase the links and coordination between the different cadres working in the communities, and on the need to focus the activities, interventions and coverage to achieve the desired impact on behavior change. The first step will be to work with the partners to internalize the evaluation recommendations and to develop a common vision that they can believe is possible. This will also provide an opportunity to begin focusing the behavior change priorities, and to 29 30 clarify a common approach to coordination and action planning at the community level. All of the partners are already planning for a community level meeting to begin clarifying the roles and responsibilities of each cadre, and strengthening the links between them. The new Health Coordinator is starting with Concern in mid-October, and the technical backstop is planning a trip to Haiti shortly after that to help with a partner meeting. Other “next steps” suggested by some of the zones included working on testing the quality of the water in the water trucks, strengthening an approach to impact monitoring including identification of the strong sub-zones and designation of target levels specific to those zones, reinforcing the role of the youth volunteers in health promotion and facilitating dialogues, and linking distributors with CBOs and/or health centers to assure their supplies. In the meantime, Concern is proceeding with seeking additional funding for expanded nutrition interventions. This will allow for further response to felt need by offering interventions for the moderately malnourished children in addition to begin working to prevent recurrence of malnutrition among children who are treated. This will also offer further opportunities to strengthen the links between CMAM and child survival.