Final Evaluation of RICHES 2000 July 11-25, 2000 Franklin Baer, Team Leader James Setzer, CARE/Atlanta Guy Fred Celestin, AOPS Dr. Arsène Ferrus, FOCAS Dr. Jean Frantz Lemoine, MSSP Pascale Crussard, CARE/Haiti Mirelle Sylvain, CARE/Haiti Paula S. Brunache, RICHES 2000 Rose-Marie Chérilus, RICHES 2000 Adelette Pierre, RICHES 2000 Executive Summary The objective of the RICHES 2000 project (October 1996-September 2000) was to improve child survival by emphasizing maternal health services in eight communes of Western Grand’Anse. The project area included a total population of 212,352 and direct target population of 53,088 women. Children and men also benefited indirectly from project interventions. The project approach was based on the institutional strengthening of health centers to improve the delivery and management of the quality services for maternal health, family planning and HIV/AIDS and MST. The project also trained health center personnel in crosscutting strategies for community mobilization and behavior change communications related to the technical interventions. Project indicators included objectives to increase: $ the percentage of recent pregnancies with at least 3 prenatal visits from 56 to 75%; $ modern contraceptive prevalence among females from 14 to 20%; $ the percentage of women (and partners) using condoms from 1 to 10%; and $ the institutional capacity of partner institutions by 50% above their baseline score. The final evaluation of RICHES 2000 took place July 11-25, 2000. The evaluation team included four external evaluators, three CARE headquarters and national staff, and three project field staff. The team reviewed project reports and results from the baseline and final KPC and institutional capacity surveys. The team made field trips to four communes to interview health center staff and to conduct discussions with a number of community-based youth clubs, mothers’ clubs and fathers’ clubs. The evaluation team found that RICHES 2000 had made very good to excellent progress in the achievements of its objectives. The project surpassed all of its objectives for the family planning and institutional strengthening components, achieved or nearly achieved all of its objectives for maternal health, and made impressive progress for the component of HIV/AIDS and STDs. The evaluation team found that a major strength of RICHES 2000 was in balancing health center-based activities to improve quality and services with community-based activities to increase knowledge and demand. This was particularly impressive in the area of family planning where contraceptive prevalence rose from 14% to 33%. The team also found that the emphasis on direct maternal health services also contributed indirectly (or did not limit) to improvements in health services for children. The project also found that institutional strengthening is enhanced if it includes or builds upon existing community mobilization and behavior change communications strategies. The project was weak in the monitoring of project objectives and the information system. Inconsistencies between the DIP and KPC methodologies made it difficult to assess project achievement of objectives. The evaluation team identified a number lessons learned during RICHES 2000. These lessons are listed on the following page. Lessons Learned - RICHES 2000 Project 1) The management of an institutional strengthening project is quite different, more difficult and slower than a project to develop direct service delivery at the community level, but offers greater possibilities for the continuation of services after the end of the project. 2) A project based on institutional partnerships requires a careful, flexible implementation process, a continuous two-way dialogue and clear roles that are respected by each partner. 3) In an institutional strengthening project, the decentralization of planning plays an important role in creating local ownership. 4) Every sustainability strategy should systematically include a structured phase-out plan that is known by all the partners. 5) The effectiveness of an institutional strengthening project is enhanced when it builds on existing community mobilization and behavior change communications strategies. 6) In an institutional strengthening project it is important to balance health center-based activities to improve quality and services with community-based activities to increase knowledge and demand. 7) A project that focuses on maternal health services can, at the same time, indirectly maintains or improves health services for children. 8) Behavior change communications through youth clubs, mother’s clubs and father’s clubs is effective because of the multiplication effect in the dissemination of information and the reduced cost in adding new messages. 9) The level of knowledge required to achieve a significant change in behavior varies by type of intervention. 10) By responding to client demand for clinic-based methods it is possible to increase the utilization of family planning services without depending on a well-developed community-based distribution network. 11) A good system to care for STDs and AIDS requires a functioning referral system. 12) Cost recovery objectives and activities should be included in every project in order to encourage sustainability. Acronyms and Abbreviations AEADMA Association d’Entreaide de Dame Mariens AFSC American Friends Service Committee AIDS Acquired Immuno-deficiency Syndrome AOPS Association d’Oeuvres Sanitaires Privés ARI Acute Respiratory Infection BCC Behavior Change Communications CBD Community-Based Distribution CDD Control of Diarrheal Disease ColVol Collaborators Volontaires (volunteer community workers) CS Child Survival DIP Detailed Implementation Plan EPI Expanded Program of Immunizations HHF Haitian Health Foundation HIV Human Immunodeficiency Virus IEC Information, Education and Communications IGA Income Generating Activity KPC Knowledge, Practices and Coverage Survey MCH Maternal and Child Health MSPP Ministry of Health & Population MTE Midterm Evaluation NA Not Applicable or Not Available NGO Non-Governmental Organization ORT Oral Rehydration Therapy QA Quality assurance Profamil Association for the Promotion of the Haitian Family PVO Private Voluntary Organization RH Reproductive Health SBP Soeurs de Bon Pasteur SOE Service Oeucumenique d’Entraide STDs Sexually Transmitted Diseases TBA Traditional Birth Attendant TT2 Tetanus Toxoid Vaccination (2 doses) UCS Communal Health Unit UNFPA United Nations Fund for Population Activities USAID United States Agency for International Development Table of Contents Executive Summary and Lessons Learned Map of Project Area Acronyms and Abbreviations I. Assessment of Results and Impact of the Program A. Data Summary Chart- baseline and final data for all objectives/indicators B. Overview of the program approach: C. Results by Technical Intervention 1. Maternal Health 2. Family Planning 3. HIV/AIDS and STDs D. Cross Cutting Approaches 1. Institutional Strengthening 2. Community mobilization and Behavior Change Communications E. Capacity Building Approach 1. Strengthening the PVO Organization 2. Strengthening Local Partner Organizations 3. Health Facilities Strengthening (see Institutional Strengthening) 4. Strengthening Health worker performance and Training F. Sustainability Strategy II. Program Management A. Planning B. Staff Training C. Supervision of Program Staff D. Human Resources and Staff Management E. Financial Management F. Logistics G. Information Management H. Technical and Administrative Support I. Management Lessons Learned J. Other Issues Identified by the Team III. Conclusions and Recommendations A. Overall achievement of Objectives and Project Success B. Important achievements and constraints C. Lessons learned. D. Recommendations for USAID/BHR/PVC E. Plan for dissemination of information by PVO HQ F. Results highlight ATTACHMENTS List of Figures and Tables Figure 1: Map of Project Area Figure 2: Maternal Health Objectives Figure 3: Vaccination Coverage Figure 4: Family Planning Objectives Figure 5: HIV/AIDS and STD Objective Figure 6: Institutional Assessment by Management Component Figure 7: Institutional Assessment by Health Center Figure 8: RICHES 2000 Organizational Chart Table 1: Project Progress in Achievement of Objectives Table 2: Comparison of RICHES I, II and 2000 Table 3: Institutional Profile of Partners of RICHES 2000 Table 4: Modern Contraceptive Preference Table 5: Institutional Capacity by Management Component Table 6: Institutional Capacity by Health Center Table 7: Comparison of Management Capacity by Development Stage Table 8: Summary of RICHES 2000 Training Table 9: Sustainability Plan Table 10: NGO Development Strategies Final Evaluation of RICHES 2000 Page 1 I. Assessment of Results and Impact of the Program A. Data Summary Chart Table 1: Project Progress in Achievement of Objectives Objective Baseline Final Target 1. Increase use of appropriate maternal health services as measured by. . . Increase in % of most recent pregnancies benefiting from at least 3 prenatal visits from 56% to 75% 56 73.3 75 Increase in TT2+ coverage from 43% to 85% 43 65.3 85 Increase in % of births attended by trained providers from 65% to 85% 65 78.9 85 Increase in % of women who can spontaneously cite at least two danger signs of pregnancy that require immediate seeking of medical care from 21% to 65% 21 57.6 65 Increase in % of women who prepare at least three items to prepare for home childbirth from 20% to 65% 19.3 29.8 65 Increase in % of women who can spontaneously cite at least three food groups that should be eaten during pregnancy from 35% to 65%. 35 92.6 65 Increase in % of births followed by postnatal care [for the baby] within 8 weeks from 69.8% to 40% 69.8 72.1 40 2. Increased Family Planning as measured by. . . Increase modern contraceptive prevalence among females 15-49 years from 14% to 20% 14 33 20 Increase % of women who can cite at least two methods of modern contraception from 47 to 65% 47 84.8 65 Increase % of women who can cite at least two sources of modern contraception from 59% to 65% 59 75.9 65 Increase % of women who can cite at least two advantages for utilizing family planning methods from 24% to 65% 24 67.9 65 3. Increased use of appropriate STD and HIV preventive behaviors and services as measured by. . . Increase by 50% the number of suspected cases of sero-positive HIV that are referred for testing. NA NA NA Increase in % of women (and their partners) using condoms sometimes or always from 7.5% to 10%. 7.5 32.5 10 Increase in % of women who can spontaneously cite two modes of HIV transmission from 22% to 65% 22 36.4 65 Increase in % of women who can spontaneously cite two STD other than HIV/AIDS from 3% to 65% 3 34.2 65 Increase in % of women who know that STDs are transmitted sexually from 75% to 86% 75 92.8 86 Increase in % of women who can spontaneously cite two methods to prevent the transmission of STDs16% to 65% 16 41.6 65 Increase in % of women who are able to talk about STDs/AIDS with their partners from 57% to 65% 57 71.3 65 4. Increase institutional capacity of partner institutions by 50% above their baseline score of on the Institutional Capacity Assessment Instrument. 19 45 29 Final Evaluation of RICHES 2000 Page 2 B. Overview of the program approach The long-term goal of the RICHES 2000 project is to improve child survival by decreasing maternal morbidity and mortality in the eight communes of Western Grand (see map). RICHES 2000 builds on the success of RICHES I and II, while moving from community-based direct delivery of child survival services to capacity-building of local health institutions of services. Table 2 compares the objectives and time frames of the three RICHES projects. The project area included a total population of 212,352 and a direct target population of 53,088 women. Children and men also benefited indirectly from project interventions. Table 2: Comparison of RICHES I, II and 2000 RICHES I RICHES II RICHES 2000 Time Frame Oct 88- Sep93 Oct 93- Sep96 Oct 96- Sep00 Total Population 42,349 34,500 212,352 Target Population 10,588 women 6,352 children 8,625 women 5,575 children 53,088 women Geographic 19 outreach postes in numerous communes 19 outreach in two communes 8 communes Intervention Strategy Community-based delivery of CS and MCH services Concentration of community-based delivery of CS and MCH services Capacity-building of health centers for the delivery and management of maternal health interventions Emphasis Interventions EPI CDD pre/post-natal monitoring micro-nutrients de-worming family planning pilot EPI CDD ARI Maternal Health Family Planning HIV/AIDS and STDs Institutional Strengthening Maternal Health Family Planning HIV/AIDS and STDs The Institutional Strengthening component of RICHES 2000 worked in partnership with the Ministry of Health and Population (MSPP), local NGOs, and health centers institutions in eight communes. A complete list of these partners and health centers is shown in Table 3. RICHES 2000 worked in partnership with the MSPP and other local partner agencies to organize and provide training, material, and financial support for the health centers institutions. Each health Final Evaluation of RICHES 2000 Page 3 center was trained to develop an annual action plan (micro-plan) for their respective catchment area to improve service delivery and management capacity. Table 3: Institutional Profile of Partners of RICHES 2000 Health Center Partners Type of Center Moron MSPP: Ministry of Health & Pop. Public; Center without beds Chambellan SOE: Service Oeucumenique d’Entraide MSPP Ministry of Health & Pop. Mixed; Center without beds AEADMA: Association d’Entreaide de Dame Mariens AOPS: Association d’Oeuvres Sanitaires Privés Dame Marie MSPP: Ministry of Health & Pop. Mixed; Center with beds Irois AFSC: American Friends Service Committee Center without beds Anse d’Hainault MSPP: Ministry of Health & Pop. Public; Center with beds Bonbon MSPP: Ministry of Health & Pop. Public; Dispensary Abricots MSPP: Ministry of Health & Pop. Public; Dispensary Roseaux SBP: Soeurs Bon Samaritain Private; Dispensary The Maternal Health component was designed to improve community participation, IEC and links to health centers for maternal care. This included increasing knowledge, access and utilization of quality maternal and newborn care services, e.g., pre-natal and post-natal care, deliveries by trained TBAs and medical staff, and improved response to obstetrical emergencies. The objective of the Family Planning component was to increase access and use of family planning services especially short-term methods at the health center. This component also included a behavior change communications (BCC) strategy to improve community knowledge of family planning methods and sources to obtain contraceptives. The HIV/AIDS and STDs component of RICHES 2000 developed community-based structures (youth clubs and mothers clubs) to promote recognition of STDs, and to encourage care-seeking behaviors for treatment and testing of HIV/AIDS. Community Mobilization and Behavior Change Communications were major crosscutting approaches to establish community structures, e.g., Mothers’ Clubs, Father’s and Youth Clubs. This strategy used culturally appropriate non-formal and participatory adult education techniques to improve knowledge and demand for maternal and newborn care, family planning and HIV/AIDS and STDs services. Final Evaluation of RICHES 2000 Page 4 C. Results by Technical Intervention 1. Maternal Health and New Born Care The objective of the project for this component was: “To improve access to and use of quality maternal and newborn care services. To improve appropriate health seeking behaviors of women in the pre-, peri- and post-natal period.” The RICHES 2000 Maternal and Newborn Care component was developed to improve community participation, health seeking behavior, IEC and links to health centers in order to increase knowledge, access and utilization of quality maternal and newborn care services, e.g., pre-natal and post-natal care by trained providers, deliveries by trained TBAs and medical staff, and improved response to (and outcomes of) obstetrical emergencies. The actual specific technical interventions and activities in this component included: ÿ TBA training: support to partner institutions to organize and conduct training of TBAs; ÿ TBA supervision and linkages to formal health care system: provide adequate supplies (initial stocks) of safe birth kits and promotion of TBA supervision by partner institutions; ÿ Training and standardization of pre-natal care services: organize and conduct training for partner institution staff; ÿ Material and equipment support: provide micro-project grants to partner institutions for equipment and other material; ÿ Pre-natal care promotion and education: develop messages and materials to support IEC activities to be conducted in communities and partner institutions; ÿ Improved access to and references for pre-natal visits: promote organization of pre-natal services at rally posts and improved references of patients to appropriate level fixed facilities; ÿ Training in delivery and newborn care: organize and conduct training of partner institution personnel; Final Evaluation of RICHES 2000 Page 5 ÿ Promotion and education related to delivery and newborn care: organize and conduct training of partner institution trainers and community educators; ÿ Training and standardization of post-partum visits: organize and conduct re￾training of partner institution staff; and ÿ Post-partum care promotion and education: promote increased utilization of services through training of and promotion by health agents, community educators and TBAs. Table 1 and Figure 2 summarizes the evaluation team findings of the RICHES 2000 results against the specific objectives for the maternal and newborn care component as documented in the approved Detailed Implementation Plan (DIP). The table indicates that, in general, the project made significant, positive progress towards meeting RICHES 2000 objectives. The evaluation team identified several methodological issues/questions in its efforts to assess project results against maternal and new-born care component objectives. In several instances, there were differences found between the wording of Knowledge, Practices and Coverage (KPC) survey questions and the wording of the objectives as contained in the DIP. The KPC is the primary tool used in the evaluation of child survival projects (including RICHES 2000). The wording changes limited the team’s ability to use the KPC to directly measure progress towards the project’s defined objectives. None the less, the project appears to have made significant improvements towards meeting its objectives in the areas of: promotion of women’s utilization of pre-natal services (attendance at three or more pre-natal consultations rose from 56% to 73.3% compared to the DIP objective of 0 10 20 30 40 50 60 70 80 90 100 Percent Achievement 3+ Prenatal visits TT2+ coverage Births attended Knows pregnancy dangers Prepares 3 childbirth items Knowledge of food groups Postnatal care for baby Target Progress Baseline Figure 2: Maternal Health Objectives Final Evaluation of RICHES 2000 Page 6 75%). The project also increased the use of trained birth attendants for their most recent delivery (65% percent at baseline versus 78.9% at final, compared to an objective of 85% of deliveries). The difference between 73.3% and 75% is not significant given KPC methodologies. An increase in the number of caesarian sections performed and obstetric emergencies referred to appropriate facilities over the life of project (16 in 1998, 84 in 1999 and 149 in 2000) while not RICHES 2000 objectives seem to confirm and support these findings and the notion of improved use and availability of maternal health care services. These findings were confirmed by discussions with women in a number of communities (members of project organized mothers’ clubs) who reported that their knowledge of when to seek care for maternal health had improved as had their knowledge related to nutrition during pregnancy. Women interviewed indicated that “Now we are better informed about preventive care for mothers and children.” And “Now there are no longer women who die.” The DIP objective of 85% for TT2 coverage for pregnant women was set based upon an erroneous calculation of baseline coverage levels for TT2 as 81% rather than 43%. The final KPC indicated that coverage rates for TT2 vaccination were 65.3%. This indicates significant improvements in coverage while falling short of the established target of 85%. It should be noted that this objective was, perhaps, set unrealistically high based upon the error in estimation of the baseline coverage level. The change form 43% to 65% represents an important, positive contribution to reducing a recognized cause of neonatal mortality. There were differences between the wording of the DIP objective and baseline and final KPC question related to women’s ability to cite danger signs associated with delivery. The KPC questions measured women’s actual experience rather than their knowledge of (ability to spontaneously cite) specific danger signs. Women’s knowledge of danger signs of pregnancy rose significantly from 21% to 57.3% but did not meet the set objective of 65%. It should be noted that objectives for all indicators of women’s knowledge for all project components were set (perhaps arbitrarily) at 65%. It is unclear that this level is related to levels necessary to measure important changes in behavior within the community. There were differences in the wording of DIP objectives and KPC questions related to women’s knowledge of pregnancy and nutrition making it impossible to assess progress towards those objectives. Significant change was measured with respect to a proxy/similar question (ability to cite at least three food groups that should be eaten while pregnant) contained in both baseline and final KPC questionnaires (from 35% to 92.6%). The DIP contains an objective related to utilization of post-natal services. It is unclear whether these refer to post-natal services targeting mothers, their infants or both. The KPC questionnaire only asks about utilization of post-natal services for infants. The baseline measure for this indicator was 69.8%. The DIP baseline was reported as 11% (the evaluation team was not able to Final Evaluation of RICHES 2000 Page 7 determine the origin of this value from the baseline KPC report) and the objective was set at 40% (suggesting that the indicator was indeed intended to refer to services for women which are almost always far lower than levels of utilization of services for infants). The final KPC reports that 72.1% of women took their infants for a post-natal visit within the first eight weeks of life. The RICHES 2000 approach of community mobilization to promote awareness of and demand for services and improved health seeking behaviors combined with institutional strengthening to improve the availability and quality of services appears to have been particularly successful in achieving its family planning component objectives. Partner institution personnel recognized the importance of the service quality improvements supported by the project (especially for pre-natal and family planning services) in combination with community mobilization to increase demand for services as particularly important and perhaps unique to RICHES 2000. The contribution and/or effectiveness of the community health agents and TBAs trained and supported by the project appears to have been particularly important in achieving this success (and perhaps relatively more so than for other project components). This led the evaluation team to propose the following lesson learned: Lesson Learned: In an institutional strengthening project it is important to balance health center-based activities to improve quality and services with community-based activities to increase knowledge and demand. The evaluation team was pleased to learn that the project emphasis on maternal health with women as the direct beneficiaries of project interventions did not result in a decrease of services for children. Figure 3 shows a comparison of final and baseline KPC results indicating that vaccination coverage in the project area increased significantly. For example, Polio 3 and DTP 3 coverage were10% and 6.6% respectively in 1997 increased to 60% and 58% respectively. Similarly, measles vaccination coverage doubled from 31% to 62% and BCG increased from 38% to 67%. 0% 10% 20% 30% 40% 50% 60% 70% % Coverage BCG Polio 3 DTP 3 Measles Type of Vaccination 1997 2000 Figure 3: Vaccination Coverage Final Evaluation of RICHES 2000 Page 8 The final KPC found that the prevalence of diarrhea had slightly increased from 22% to 29%. The difference, however, may well be due to seasonal fluctuations. The treatment of diarrhea with oral rehydration increased from 42% to 44%. The percentage of mothers who said that they would give more fluids to a child with diarrhea increased from 52% to 54%. More significantly, the percentage of mothers who said that they would decrease fluids to a child with diarrhea decreased from 15% to 4%. In addition, the percentage of infants that were principally breastfed between 0-3 months increased from in 61% 1997 to 72% in 2000. It is not possible to directly attribute these positive changes in children’s health services to activities by RICHES 2000. Activities and inputs from other partners working the project area also played an important role, e.g., MSPP, SOE, AOPS and HS2004. The evaluation team was pleased to find that the emphasis on maternal health by RICHES 2000 had not detracted health personnel from their duty in the provision of child health services. Institutional strengthening activities of RICHES 2000 probably also contributed to part of the increase, e.g., improved planning, management and supervision of the vaccination services. Finally, since “as the mother goes, so goes the child,” it is also possible to argue that a strategy to target services to mothers can also contribute to an increase of services to children. This is an area where further analysis of KPC results or a separate study might be useful to better document this relationship. With respect to these results, the evaluation team formulated the following lesson learned: Lesson Learned: A project that focuses on maternal health services can, at the same time, indirectly maintain or improve health services for children. 2. Family Planning The objective of the project for this component was: “To address unmet need for high quality information related to family planning. To improve and assure access and utilization of long and short term family planning methods” The strategic approach adopted by RICHES 2000 for the Family Planning component was developed to improve community participation, IEC and links to health centers in order to increase access and use of modern family planning services, e.g. short-term methods delivered at the community and partner institution clinic level and long-term methods (voluntary surgical contraception, implants) delivered via a mobile clinic. Specific activities were defined as: ÿ Promotion of family planning (identification of potential users): mapping of communities and support to community agents to implement IEC strategy to promote family planning; Final Evaluation of RICHES 2000 Page 9 ÿ Improved access to family planning methods at community and clinic level: support to logistics systems to improve access to family planning commodities at clinic and community levels; ÿ Quality assurance (QA) for family planning services: support to clinic staff to improve and maintain quality of clinic based family planning services; and ÿ Supervision: support to institution and MSPP staff to improve supervision of family panning services and activities. Table 1 and Figure 3 summarize the evaluation findings with respect to RICHES 2000 project objectives as contained in the DIP. The results show that the RICHES 2000 project met or exceeded all of the objectives set by the DIP for the family planning component confirming this as the project’s most successful component. The most impressive of project results (see Table 4) is the increase in the level of contraceptive prevalence as measured by the KPC from 14% to 33% (as compared to the objective of 20%). This far exceeds national levels of contraceptive prevalence. It was also noted that the increases are due to increased preference for DepoProvera and decrease in the use of pills and voluntary surgical contraception. The decrease in VSC was due to the discontinuation of a mobile clinic strategy by ProFamil. Since these methods are only available at health centers, the relative importance of community-based distribution of contraceptives actually decreased during RICHES 2000. 0 10 20 30 40 50 60 70 80 90 100 Percent Achievement Contraceptive prevalence Knows 2 contraceptive methods Knows 2 sources for FP Knows 2 FP methods Target Progress Baseline Figure 4: Family Planning Objectives Final Evaluation of RICHES 2000 Page 10 Table 4: Modern Contraceptive Preference Type of Contraceptive Baseline KPC n=167 of 1560 KPC Final n=195 of 596 Difference Pill 24.6 10.6 -14 Injection 34.1 59 24.9 Condom 10.6 9.7 -0.9 Implants 13.8 16.3 2.5 Surgical Contraception 13.2 3.6 -9.6 Vaginal Cream/Tablet 1.2 0 -1.2 IUD/Diaphragm 2.4 .6 -1.8 It also suggests that the project went well beyond satisfying an existing/latent demand for family planning services and actually created new demand via its strategy of community mobilization and communication. Community mobilization and increasing client satisfaction were clearly important elements in the success of increasing contraceptive prevalence even with a low CBD system and geographic difficulties of access. This result suggests the effectiveness of the RICHES 2000 strategy of combining community mobilization with institutional strengthening in bringing about key behavior change in a largely rural and under-served population. As a result, the evaluation team formulated the following lesson: Lesson Learned: By responding to client demand for clinic-based methods it is possible to increase the utilization of family planning services without depending on a well-developed community-based distribution network. The project strategy was highly successful in increasing women’s knowledge about family planning. Knowledge levels exceeded project objectives for all chosen indicators. Logistic problems limiting the availability of contraceptives to partner institutions (and therefore to communities) will limit the long term success and sustainability of the high levels of demand and utilization for family planning services demonstrated by the project. The project made efforts to improve MSPP and partner capacity in order to improve supplies. Inclusion of youth clubs as a means for community communication for family planning played an important and innovative role in reaching a typically ignored population. The augmentation of knowledge for youth 15-19 years increased dramatically for family planning as a result of this strategy to include youth in family planning education efforts. Final Evaluation of RICHES 2000 Page 11 The lack of a cost recovery strategy to improve the sustainability of contraceptive access is of concern. There appears to be a willingness to pay for family planning services and contraceptives. However, MSPP and donor organizations have been reluctant to implement cost￾recovery for family planning (see section D.1 for further discussion of cost recovery). 3. HIV/AIDS and STDs The objective of the project for this component was: “To increase knowledge of HIV/AIDS and other STDs. To increase prevalence of safe sex practices. To increase access to and utilization of adequate STD and AIDS services.” The STD and HIV/AIDS component focuses in community-based education to promote recognition of STDs and improvements in care seeking behaviors for both STDs and HIV infections. The strategy also is designed to improve the quality of counseling, diagnosis and treatments for STDs and HIV/AIDS infections. Project activities were: ÿ Education of communities: support to partner institutions to improve STD/HIV/AIDS communities education activities; ÿ Condom promotion: development and diffusion of key educational messages to improve/promote condom use; ÿ Training of trainers for community education: training of partner institution staff to become effective educators/trainers; ÿ Clinical training: support for training of partner institution clinical staff by Cornell￾Gheskhio; ÿ Improved access to condoms: support to logistics systems to improve condom access and collaboration with PSI to promote condom availability through social marketing strategies; ÿ Supervision: support to partner institution and MSPP staff to improve supervisory skills; and ÿ Improved diagnosis and treatment: support to clinical skills of partner institution staff via training and support to improved referral of suspected HIV cases. Table 1 and Figure 5 summarize the evaluation findings with respect to RICHES 2000 project Final Evaluation of RICHES 2000 Page 12 objectives as contained in the DIP. While indicators of practice measured by condom use promoted as part of the RICHES 2000 strategy showed significant, positive change, indicators of knowledge change did not meet the objectives set by the project. The only exception was the 92.8% of the women who knew that STDs were transmitted sexually (versus an objective of 65%). There was no way to measure (either through KPC or project/institution health and management information systems) progress towards meeting the objective of increased referral and testing of suspected HIV cases. In accordance with MSPP policy at the time, the project did not develop testing capacities at the health center level. The lack of institutional strengthening and a functioning referral system constrained the ability of the project to introduce syndromic diagnostic and treatment strategies for STDs at partner institutions. As a result of this project experience, the evaluation team formulated the following lesson learned: Lesson Learned: A good case management system for HIV/AIDS and STDs requires a functioning referral system. Significant improvements were observed in use of condoms among women The KPC survey showed significant increases in the number of women who report using condoms sometimes or always (7.5% to 32.5%). This result is confirmed by an even more impressive number (13%) of women who reported using a condom during their most recent/last sexual act. Half of women (50.5%) reported knowing where condoms could be purchased. This indicates that RICHES 2000 messages had been transmitted widely throughout the community and not merely limited to those women who directly participate in the clubs and other structures supported by the project. 0 20 40 60 80 100 Percent Achievement Referral for HIV testing Use of condoms Knows 2 HIV transmission modes Knows 2 types of STDs Knows sexual transmission of STDs Knows 2 ways to prevent STDs Talks with partner about STDs/AIDS Target Progress Baseline Figure 5: HIV/AIDS & STDs Objectives Final Evaluation of RICHES 2000 Page 13 The levels of change in indicators of women’s knowledge were far below the set objectives of 65%. It should be noted that objectives for all indicators of women’s knowledge for all project components were set (perhaps arbitrarily) at 65%. It is unclear that this level is related to levels necessary to measure important changes in behavior within the community. The percentage of women who reported being able to talk about STDs and HIV/AIDS with their partners (71.3%) rose significantly and exceeded the objective of 65%. Women indicated that their participation in the project supported women’s/mother’s clubs was important in giving them the confidence to discuss these issues with their partners. Project supported youth clubs appeared to have been an effective mechanism to transmit important messages to youths. Participants appear eager for club activities to continue despite the end of RICHES 2000 project support. The evaluation team was particularly impressed by the increase in reported use of condoms “sometimes or always” and “during the last sexual act.” These important changes in practice were achieved despite the fact that the project did not achieve the degree of increase in knowledge of HIV/AIDS and STDs that it had intended. It appears, therefore, that a high level of knowledge is not always a prerequisite to making changes in practices. The evaluation team formulated the lesson learned that: Lesson Learned: The level of knowledge required to achieve a significant change in behavior varies by type of intervention. D. Cross Cutting Approaches 1. Institutional Strengthening The objective of institutional strengthening within RICHES 2000 was to reinforce support systems for health services, especially for maternal health, family planning and MST and HIV/AIDS. In March 1997, RICHES 2000 contracted with AOPS to conduct a management capacity assessment. AOPS used a management capacity assessment tool developed by the HS-2004 project to establish a baseline management capacity for the eight participating institutions. The assessment examined institutional capacity in the areas of organization, population/health indicators, planning, human resources, information systems, stock management, financial management, and IEC. The assessment established a baseline capacity score for each institution and for each management component. Final Evaluation of RICHES 2000 Page 14 Based on the assessment findings, AOPS assisted the project in the designing and conducting a variety of management training sessions (see Table 8). These included management training and equipment assistance to improve capacity, while simultaneously stimulating the demand for appropriate health services, and the ability of health institutions to supply the required services on a lasting basis. This training led to the development of micro-projects in the form of annual action plans with key priorities and activities to be carried out by the institutions. In July 2000, the project repeated the management assessment survey. It was after the completion of data collection for that survey, however, that it was discovered that the questionnaires for the baseline and final were not identical, and that the method for scoring some variables had not been well documented during the baseline survey. The two survey results are not, therefore, entirely comparable. A recalculation and readjustment of results was done during the final evaluation to try to make the results more comparable while sacrificing some precision in indicators. These adjustments, however, do not change the overall conclusion that the project met its objective to increase management capacity by at least 50%. Table 5 and Figure 6 show the increase in capacity for each of the major management components. The overall baseline capacity of 19% increased to 45%. This represents an increase of 131%. The largest gains in capacity were in the areas of planning, human resources, financial management and stock management. For example, within the planning component most of the capacity gains came from a significant increase in the setting and monitoring of objectives. Table 5: Institutional Capacity by Management Component Management Component Baseline 1997 Final 2000 Maximum Points Organizational Structure 7 7 15 Population and Health Indicators 5 7 35 Planning 8 30 55 Human Resources 2 10 38 Information Systems 5 6 10 Stock Management 8 21 45 Financial Management 6 27 44 IEC 14 19 40 Total 54 126 282 19% 45% Final Evaluation of RICHES 2000 Page 15 Table 6 and Figure 7 show the increase in overall management capacity for each of the eight health institutions The baseline capacities ranged from 16% at Bon-bon to 25% at Dame-Marie with an average of 19%. The final survey capacities ranged from 38% at Anse d"Hainault to 51% at Dame-Marie and Abricots with an average of 45%. The overall increase in capacity was 131%. Table 6: Institutional Capacity by Health Center Moron Cham￾bellan Dame￾Marie Irois Anse d"Hainault Bon￾bon Abri￾cots Roseaux Average Total 1999 47 56 70 53 50 45 50 64 54 Total 2000 115 121 143 141 107 110 143 127 126 % 1997 17% 20% 25% 19% 18% 16% 18% 23% 19% % 2000 41% 43% 51% 50% 38% 39% 51% 45% 45% Improvement 145% 116% 104% 166% 114% 144% 186% 98% 131% 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Average Score Organization Structure Pop/Health Indicators Planning Human Resources Information Systems Stock Management Financial Management IEC 2000 1997 Figure 6: Assessment by Component Final Evaluation of RICHES 2000 Page 16 Table 7 compares the management capacity of the eight health centers by development stage. According to the scaling system of the original assessment methodology all eight health centers were classed in the start-up development stage. The second assessment July 2000 found that six health centers had progressed to the second stage of development while two health centers, Dame-Marie and Abricots, had increased their capacity into the three “growth” phase. Table 7: Comparison of Management Capacity by Development Stage Démarrage Début/ Startup I Phase Démarrage Moyen/ Startup II Phase Croissance / Growth Phase Consolidation Phase 0 - .25 .26 - .50 .51 - .75 .76 - 1.0 1997 All 8 Institutions 2000 6 Institutions Moron, Chambellan, Irois, Anse d"Hainault, Bon-bon, Roseaux 2 Institutions Dame-Marie, Abricots 0% 10% 20% 30% 40% 50% % Capacity Moron Chambellan Dame-Marie Irois Anse d"Hainault Bonbon Abricots Roseaux Average 1997 2000 Figure 7: Assessment by Health Center Final Evaluation of RICHES 2000 Page 17 In addition to training, the project also provided material assistance as part of institutional strengthening, e.g., health education materials, maternal health equipment, and laboratory equipment. The impact of equipment furnished to partner institutions by RICHES 2000 was, in some cases, limited by the availability of personnel at those institutions to use or maintain effectively, e.g., some laboratory equipment. It should be noted, however, that most of the equipment provided is considered low cost and low maintenance. The revolving drug funds established by partner institutions (using initial stock and supplies provided by the project) appear to be functioning adequately and have improved drug availability. Drug supply problems and occasional stock-outs continue within partner institutions due to logistics and supply problems at the departmental level. Drug fund revenues have also been used to provide limited financial support to community health agents. The project has not identified a funding source for TBA kits and no cost recovery mechanisms were put into place for these important kits. It is unsure if partner institutions will continue to provide TBAs with clean birthing kits. These questions give rise to questions of the appropriateness of providing equipment and materials without implementing cost recovery strategies to address long-term recurrent cost for resupply or maintenance. This led the evaluation team to formulate the following lesson learned: Lesson Learned: Cost recovery objectives and activities should be included in every project in order to encourage sustainability. 2. Community mobilization and Behavior Change Communications The project used the same community mobilization and behavior change communications (BCC) strategies and materials that were developed during RICHES I and II. These projects were recognized nationally for their culturally-appropriate, participatory non-formal adult education approach to preventive health service delivery. RICHES I and II developed more than thirty child survival, maternal health and family planning “lessons.” RICHES 2000 added a series of five lessons for HIV/AIDS and MST prevention. In the case of RICHES 2000, the community structures are primarily youth clubs and mothers’ clubs. The twenty-seven clubs formed by the project appear to have been successful mechanisms for the diffusion of IEC messages. They allowed for the sharing of messages with larger numbers of individuals than alternative strategies such as one-on-one counseling and education. They also have encouraged club members to act as health education “multipliers,” i.e., to pass messages to Final Evaluation of RICHES 2000 Page 18 other community members. This has resulted in higher overall levels of knowledge within the entire community. For example, the inclusion of youth clubs for BCC in family planning played an important and innovative role is sharing information with their peers throughout the community. The augmentation of knowledge for youth 15-19 years increased 54.7 to 96.6% dramatically for family planning. It was also reported that the sharing of experiences between women’s groups was an important motivation to encourage increases in family planning knowledge and practice. These positive results were formulated into the following lesson learned: Lesson Learned: Behavior change communications through youth clubs, mother’s clubs and father’s clubs is effective because of the multiplication effect in the dissemination of information and the reduced cost in adding new messages. Clubs also took the initiative to compose songs and plays to promote the transmission of health messages. Certain clubs, especially combined mothers-fathers clubs, also developed a plan for the evacuation of obstetrical emergencies. The organization of community-based income generating activities (IGAs) for economic and re-creative reasons was begun during the last year of the project through mothers-fathers clubs. For example, one club, ALAZI is currently producing and marketing chocolate from cocoa nuts. This strategy shows considerable promise. The impact and effectiveness of the RICHESS 2000 strategy for behavior change communication benefited from the experience in community mobilization by RICHE II. The evaluation team agrees that the strategies employed by RICHES 2000 appear effective. It would be of interest to know if a similar institutional strengthening project in an area without a prior community mobilization foundation could attain similar levels of success/effectiveness. This led the evaluation team to formulate the following lesson learned: Lesson Learned: The effectiveness of an institutional strengthening project is enhanced when it builds on existing community mobilization and behavior change communications strategies. E. Capacity Building Approach 1. Strengthening the PVO Organization RICHES 2000 marked a major strategic shift in health programming for CARE/Haiti. Prior to RICHES 2000 CARE/Haiti health programs (as exemplified by RICHES 2) were based upon CARE’s proven ability to work directly with communities (often in extremely remote areas) to Final Evaluation of RICHES 2000 Page 19 provide key health services. This strategy was effective in bringing about important knowledge, behavior and health status improvements in those communities, which CARE was able to reach directly. Under this strategy the number of communities was limited by resource availability and questions arose as to the sustainability of the results. Through RICHES 2000, however, CARE/Haiti has shifted its strategic approach to one that involves community development and mobilization through partnership. These partnerships have been developed with permanent health institutions, which deliver health services (including curative) in the project area. The shift to partnership in RICHES 2000 is consistent with a major strategic approach to embrace partnership in CARE programs worldwide. RICHES 2000, therefore, provided CARE Haiti with an important opportunity to develop the tools and approaches necessary to operationalize this strategic shift to partnership. CARE Haiti now has a successful model for projects which includes: ÿ guidelines for the development of partner relationships and agreements with both private and private/public mixed institutions; ÿ tools, approaches and materials for building the institutional capacity of partners to increase their ability to deliver quality services and mobilize communities to increase the demand for and utilization of those services; and ÿ a project management model for CARE to effectively manage resources to support partners. CARE/Haiti continues to assess the effectiveness of its partnership strategy and the nuts and bolts aspects for its effective implementation. RICHES 2000 evaluation results will contribute directly to this dialogue. An ongoing process to evaluate program and strategy effectiveness is already in place. CARE/Haiti’s ability to design and implement effective health projects has been enhanced as a result. RICHES 2000’s partnership strategy is consistent with a shift away from direct service delivery programs in the health sector for CARE worldwide. RICHES 2000 evaluation results will provide important feedback to CARE headquarters as it evaluates the effectiveness of this approach. It will allow CARE to design and implement more effective and (presumably) sustainable health programs worldwide. 2. Strengthening Local Partner Organizations Collaboration and institutional strengthening activities carried out with the MSPP posed a special challenge. The continued support by the MSP of the partner institutions is a key element of the Final Evaluation of RICHES 2000 Page 20 RICHES 2000 sustainability strategy. The project made considerable efforts to build MSPP capacity and to assist and support the MSPP in supervising and supporting the partner institutions. High turnover and lack of resources (vehicles, per diems, etc.) at the MSPP departmental level hampered these efforts. For example, the absence of perdiems at the MSPP level meant that supervision was not done together in every instance. 3. Health Facilities Strengthening (see Institutional Strengthening) 4. Strengthening Health Worker Performance and Training The health personnel interviewed generally recognized the importance and impact training and supervisory activities provided through RICHES 2000. Table 8 shows a summary of the numbers of personnel trained by category of personnel and topic. This information is incomplete, however, because the reporting system for training has not been well maintained by the project. In addition to technical training in the areas of Maternal Health, Family Planning and HIV/AIDS and MST, the project also provided considerable training for important components of management, e.g., planning, financial management, and stock management. Health Center personnel also recognized that in addition to being motivated by training that they were also motivated by: $ feedback that they received during supervision visits; $ “contests” in which results from institutions are compared at quarterly meetings; $ the provision of audio-visual equipment; $ the existence of norms, procedures, job descriptions, management tools; and $ the process of planning and their own program together with the community. Final Evaluation of RICHES 2000 Page 21 Table 8: Summary of RICHES 2000 Training Category of Personnel Maternal Health Family Planning HIV/AIDS and STDs Management Doctors 1 0 11 Nurses 4 12 2 Nurses Aids 1 28 13 Health Agents 73 55 107 ColVols 38 109 606 Matrons 339 9 131 Leadership and Management (25) Participative Approach (19) Stock Management (21) Information System (16) Financial Management (12) Training of Trainers (22) A few staff reported a number of instances when training opportunities were missed due to the inability of the institutions to free up personnel to participate. In addition, the turnover of key MSPP personnel at the departmental level (especially the departmental director) limited the effectiveness of training to strengthen the MSPP planning and supervision. The project also strengthened the work of traditional birth attendants (TBAs) at the community level. The project used community-based focus group discussion and individual feedback from pregnant women help to identify active TBAs at the community level. An assessment of TBAs knowledge and skills was carried out as part of the micro-planning process for each health center catchment area. TBAs then received training appropriate to their current skill level. Older TBAs helped to train novice TBAs. The supervision of TBAs was carried out jointly by the partner institution with staff from RICHES 2000. This included supervision of some home deliveries. During interviews with the evaluation team, health center personnel recognized the important role that matrons and health agents are playing at the community level. They attributed the important increase in utilization of services, especially for prenatal care and family planning, to improvements in the quality of services and the increased demand within the community. F. Sustainability Strategy A sustainability plan was developed as part of the DIP. The midterm evaluation found that the plan was not as detailed as it should be and recommended that it be revised. The revised plan is Final Evaluation of RICHES 2000 Page 22 shown in Table 9. The project has, for the most part, implemented the revised strategy as planned. However, the absence of a structured phase out plan is noted below. Table 9: Sustainability Plan Goals Objectives Results Increase partner’s capacities to provider reproductive health services by strengthening technical, administrative and financial knowledge and skills. Generally accomplished as planned Develop health community agent capacity to promote good practices and pass key health messages to target population. Good for TBAs and Clubs; not as good for health agents. 1) Institutionalizing Reproductive Health Services Increase knowledge base and health seeking behaviors of communities Achieved especially for maternal health and family planning. 2) Reproductive Health Services will continue to be provided at the community￾level in the communes of Abricot and Monon Note: The original objective to “Create a successful UCS” was dropped at MTE. Transfer RICHES II service areas to local NGOs (HHF and Medicins du Monde) Develop a community-based fall back plan in event transfer does not work Completed as planned with transfer to the health centers institutions rather than to local NGOs. 3) Support Grand’Anse Departmental meeting Note: The original objective to “Support Grand’Anse health consortium” was modified during the MTE Develop quarterly meeting plan with MOH Coordination was limited to discussions with MSPP and local partners managing health centers. It did not include other projects working in the area. The process was also handicapped by frequent changes in departmental leadership. Prior to the final evaluation, CARE/Haiti had already concluded that the absence of a structured phase out plan as part of the sustainability strategy was a problem. Certain project staff positions for RICHES 2000 were only planned through December 1999. This was part of a budgeted, but unwritten, plan to phase out these positions nine months before the end of the project. However, the fact that the phase out plan had not been well documented and known by all the partners, had a negative impact on staff moral. As a result of this experience, the final evaluation team formulated the following lesson learned: Lesson Learned: Every sustainability strategy should systematically include a structured phase-out plan that is known by all the partners. Final Evaluation of RICHES 2000 Page 23 II. Program Management A. Planning RICHES 2000 was built on the results and orientation produced from RICHES II with a strategy to continue and expand upon existing community mobilization and BCC activities, but working through an institutional strengthening approach. The principal partners in the project design process were, therefore, representatives of the institutions and the MSPP (see Table 3). For example, the fact that the project paper calls for RICHES 2000 to assist in the creation of a UCS in collaboration with local NGOs was a direct reflection of the MSPP policy and implementation plan at the time of the project design. The project team reported that the DIP process was useful, and that it served as a reference point for developing annual workplans. However, several inaccuracies in the formulation of DIP objectives and indicators which carried through the project undetected emphasize the importance of making sure that the DIP is an accurate reference document. Some of the mistakes should have been detected during the DIP review process, e.g., establishing an objective to increase TT2 vaccination coverage from 81% to 85%. The institutional strengthening process was based on micro-planning by each health center. Based on the findings of the baseline KPC and institutional assessments, the project provided management training to each health center with the objective of developing an institutional￾specific plan to improve the management capacity and health services for that institution. This micro-plan was also the basis for calculating the amount of assistance to be provided to that institution and the basis for preparing a contractual partnership agreement. The project staff felt that the micro-planning process was very important in transferring ownership of the program to the local level. This is an important aspect of institutional strengthening and program sustainability. Based on this result, the evaluation team formulated the following lesson learned: Lesson Learned: In an institutional strengthening project, the decentralization of planning plays an important role in creating local ownership. C. Staff Training and Supervision Most of the project staff had previously worked for the RICHES II project. They were already appropriately trained for work at the community level. The main area of training for project team was, therefore, in the area of management and institutional strengthening. The project opted to hire a person with management background, rather than a health background, to head up the Final Evaluation of RICHES 2000 Page 24 institutional strengthening component of the project. They found that it was easier to train a management person in health, rather than to train a health person in management. This option also had the advantage of bringing non-health personnel into the project with a slightly different development perspective. During the project, CARE/Haiti restructured the direct supervision of the project from a national health coordinator to a regional coordinator. This changed occurred without any major problems. D. Human Resources and Staff Management The organizational chart for the RICHES 2000 project is shown in Figure 8. The configuration and number of personnel was appropriate to the type and size of project. Figure 8: Riches 2000 Organizational Chart Institutional Development Officer Training/IEC Assistant Training/IEC 3 Drivers Administrative Assistant 3 Community Animators Community Mobilization Officer Project Manager E. Financial Management There were no major budget changes during the life of the project. CARE completed their financial match for this project in collaboration with funding from UNFPA. This funding was originally budgeted to end in December 1999, but actually continued through January 2000. The fact that UNFPA funding was not continued to the end of the project does not have any bearing of the completion of the match funding. F. Logistics There were not major logistical problems reported by the project. This, in itself, is an important and positive finding, since the project area in the Grand’Anse is considered one of the most isolated and rugged areas in Haiti. The fact that the project in collaboration with the MSPP and Final Evaluation of RICHES 2000 Page 25 local partners was able to ensure a relatively secure supply line of essential medicines and contraceptives for the project is commendable. G. Information Management Information management was one of the weakest areas of the project. As previously noted, the formulation of project objectives and indicators did not always match the monitoring system. For example, an objective defined in terms of “increasing knowledge” was actually measured by the KPC in terms of “increasing practice.” The fact that most of these inconsistencies were not identified prior to the final evaluation is further evidence that the project did not give adequate attention to the definition and monitoring of its objectives. There was no midterm KPC conducted, even though this was planned for in the DIP. In addition, the project could have monitored several objectives using the MSPP information system, e.g., vaccination coverage for TT2. However, there was no indication that this was done. The institutional strengthening of health centers in the area of information systems did not establish a good monitoring capacity at the health center level. While health centers had clearly defined target groups and monthly objectives, there was no indication that they were monitoring these objectives on a monthly basis. For example, none of the health centers had a “tableau du bord” to chart and monitor their activities. There were several circumstances that made information system monitoring difficult for the project. First, the MSPP has been in the process of revising its information system for a number of years. RICHES 2000 decided, therefore, not to create a parallel and duplicative information system at the health center level. As a result, however, the project kept waiting for the new system to be implemented rather than working to improve monitoring within the existing system. Second, a quarterly monitoring system was used by the project to monitor certain process indicators, e.g., training. However, this system was designed by CARE/Atlanta to standardize their monitoring of projects. As a result, it was not tailored to the specific objectives and indicators of RICHES 2000. Third, following the midterm evaluation, there was an effort to seek outside technical assistance to reduce the number of indicators and to implement a project monitoring system. Unfortunately, the departure of a key technical person from the CARE/Haiti staff occurred before this system could be implemented. H. Technical and Administrative Support Final Evaluation of RICHES 2000 Page 26 The project was able to build on its existing capacity and resources in the areas of community mobilization and behavior change communications. It did not require, therefore, further outside technical assistance in these areas. AOPS provided technical assistance to RICHES 2000 was in the areas of institutional strengthening and management training. This included conducting the baseline institutional capacity survey and developing/conducting training sessions for most of the management components, e.g., financial management. The level of assistance was appropriate. The project manager reported that administrative support from CARE/Haiti and CARE/Atlanta was provided on a regular basis and adequate to the needs of the project. I. Management Lessons Learned In comparing RICHES I and II with RICHES 2000, there was a general consensus among CARE personnel that the management of an institutional strengthening project is quite different and more difficult than a project to develop services directly at the community level. However, they also noted that they felt that the approach used by RICHES 2000 would be more sustainable. In particular, both project personnel and partners at the health center and community level said that the institutional strengthening approach used by RICHES 2000 required a slower development process than CARE had used during RICHES I and II. Project personnel who had worked with RICHES II tended to try to work more quickly with project implementation. This led at times to some frustration expressed by both project and health center personnel. Eventually, however, a negotiated compromise pace for project activities was developed. Based on these observations, the final evaluation team formulated the following lessons learned: Lesson Learned: The management of an institutional strengthening project is quite different, more difficult and slower than a project to develop direct service delivery at the community level, but offers greater possibilities for the continuation of services after the end of the project. Lesson Learned: A project based on institutional partnerships requires a careful, flexible implementation process, a continuous two-way dialogue and clear roles that are respected by each partner. J. Other Issues/Discussion: 1. NGO Development Strategies Final Evaluation of RICHES 2000 Page 27 A conceptual framework David Korten has proposed a useful framework of “Four Generations of NGO Development Strategies” (see Table 10). These strategies include 1) Relief & Welfare; 2) Community Development; 3) Sustainable Systems Development; and 4) People's Movements. This conceptual framework was discussed as part of the final evaluation of RICHES 2000. It was noted that CARE is currently using at least three of the four strategies in Haiti. In particular a comparison of RICHES II and RICHES 2000 is an excellent example of an NGO making the transition from Community Development to Sustainable Systems Development. Table 10: Four Generations of NGO Development Strategies FIRST Relief & Welfare SECOND Community Development THIRD Sustainable Systems Development FOURTH People's Movements Scope Individual & Family Neighborhood & Village District, Region & Nation National or Global Chief Actors NGO NGO plus community Public & Private Institutions Loosely Defined Networks NGO Role Doer Mobilizer Catalyst Facilitator Activist and Educator Management Orientation Logistics Management Project Management Strategic [systems] Management Self-Emerging Networks Examples Feeding centers Hospital care Mobile teams Community￾based development Integrated Health Systems Health Districts Self-Selecting Groups Volunteer Networks Adapted from David Korten, Getting to the 21st Century, 1990. CARE’s development strategies in Haiti Most PVOs and international NGOs, including CARE, began as a response to war or natural disasters with strategies based primarily on providing urgently needed relief and welfare. In this strategy the NGO is the chief actor and “doer”. The primary management orientation and problems encountered usually revolve around logistics, e.g., getting food air from a port to refugee camps. With regards to CARE’s current work in Haiti, its food aid programs in the Northwest province could still be classified as a form of this strategy. Final Evaluation of RICHES 2000 Page 28 After successfully responding to crisis situations, some PVOs moved to the second-generation strategy of community development. In fact, integrated community development is generally recognized as the strength of most NGOs. Child Survival projects like RICHES I and II that work directly with a limited population in a few select communities are good examples of this strategy. It requires that the NGO work in partnership with the community as a “mobilizer” rather than as a “doer.” The management orientation involves the spectrum of management from planning and implementation to monitoring and evaluation. While NGOs are justifiably proud of their community development work, they recognize the limited geographic scope and not-always-sustainable criticisms to this approach. As a consequence, some NGOs have moved to working through partnerships with the government and/or local NGOs for sustainable systems development. The institutional strengthening strategy of RICHES 2000 is a good example of this approach. It requires the NGO to work as a catalyst or facilitator in assisting the Ministry of Health to implement its development policies on a systems wide basis. It also requires that the project assistance be provided not only for health interventions, but also for developing or strengthening the support systems for those interventions, e.g., planning and information systems. Finally, some NGOs have adopted a strategy of creating loosely defined networks of people and organizations to develop self-managing development initiatives. In the case of CARE, a future strategy to encourage a self-expansion of youth clubs, outside of a project framework, could illustrate this type of approach. The role of the NGO, in this case, is as an activist and educator, rather than as a project manager. Discussion Child Survival projects have traditionally been defined and designed as community development projects. Over the last several years, however, the guidelines for development of Child Survival projects have increased the emphasis on capacity building and partnerships with local partners, e.g., the Ministry of Health and local NGOs. RICHES 2000 probably represents one of the few Child Survival projects that has made institutional strengthening a principle objective and key project strategy. This change of approach also represents an important change for the work of CARE in Haiti. There has been considerable discussion within CARE about the pros and cons of this new approach. It is important to recognize that none of the four strategies as proposed by Korten is “better” than the other. Each has its appropriate time and place. III. Conclusions and Recommendations Final Evaluation of RICHES 2000 Page 29 A. Overall achievement of Objectives and Project Success The evaluation team found that RICHES 2000 had made very good to excellent progress in the achievements of its objectives. The project surpassed all of its objectives for the family planning and institutional strengthening components, achieved or nearly achieved all of its objectives for maternal health, and made impressive progress for the component of HIV/AIDS and STDs. The project is to be commended for achieving these results in the project area of Grand’Anse which is one of the most isolated and rugged areas of Haiti. B. Important achievements and constraints The evaluation team found that a major strength of RICHES 2000 was in balancing health center-based activities to improve quality and services with community-based activities to increase knowledge and demand. The project also found that the effectiveness of an institutional strengthening project is enhanced if it includes or builds on existing community mobilization and behavior change communications strategies. Finally, the team found that the emphasis on direct maternal health services also contributed indirectly to improvements in curative and preventive services for children. The evaluation team identified several weaknesses in the monitoring of project objectives and in the project’s sustainability plan. The existence of inconsistencies between the DIP and survey methodologies made it difficult for the project to assess whether it was achieving its stated objectives. The absence of a structured phase-out strategy as part of the sustainability plan made it difficult to assess whether the level at which some project activities would be continued. C. Lessons Learned. Lessons learned from RICHES 2000 are discussed within their appropriate section, e.g., community mobilization. All the lessons learned are summarized as part of the executive summary. Final Evaluation of RICHES 2000 Page 30 D. Results highlight CARE’s RICHES 2000 project in rural Haiti was instrumental in high levels of contraceptive prevalence despite a deteriorating community based contraceptive distribution network. Comparison of baseline and final Knowledge, Practices and Coverage (KPC) surveys conducted in the remote and rural area of the Grand Anse region of Haiti covered by the RICHES 2000 project indicate that use of contraceptives rose from 20% to 33% of all women aged 15-49 years of age. This is especially impressive when compared to national estimates of contraceptive prevalence of approximately 15%. Clearly the RICHES 2000 strategy of combining community mobilization and education with efforts to strengthen the delivery of key women’s health services delivered by fixed health facilities in the region has contributed to this important result. The results are all the more impressive considering that the Ministry of Public Health and Population (MSPP) had, for largely financial reasons, withdrawn support to its network of rural, community based agents de sante (community health agents). These agents had been the foundation for MSPP efforts to increase access to modern contraceptives as distribution points for contraceptives (mainly condoms and pills). At its inception the RICHES 2000 project sought to increase contraceptive use in its eight communes in rural Grand Anse region by supporting the community health agents and improving the capacity of the fixed health facilities in the area to deliver family planning services. These efforts to improve the availability of services were coupled with aggressive and innovative community mobilization and communication activities intended to increase the demand for those services. At the same time as RICHES 2000 was improving the availability of all methods of modern contraception (including clinic-based methods) a shift in client preference was occurring. More and more women were choosing injectable contraceptives (depo-provera) over other methods. These methods were available only at clinics. This shift in preference then coincided with PICHES 2000’s efforts to improve their availability at the fixed facilities in the area. This shift in preference is seen in RICHES 2000 KPC results which show an increase in overall contraceptive prevalence and a shift towards the use of injectable contraceptives (36% of women practicing family planning at baseline used injectables as compared to 51% four years later). Final Evaluation of RICHES 2000 Page 31 List of Participants Franklin Baer, Team Leader James Setzer, CARE/Atlanta Guy Fred Celestin, AOPS Dr. Arsène Ferrus, FOCAS Dr. Jean Frantz Lemoine, MSSP Pascale Crussard, CARE/Haiti Mirelle Sylvain, CARE/Haiti Paula S. Brunache, RICHES 2000 Rose-Marie Ch érilus, RICHES 2000 Adelette Pierre, RICHES 2000 Sandra Laumark, CARE/Haiti Isaac Michel, CARE/Haiti Imran Zafar, PSI M. Bonavente, HS-2004 Dr. Ludzen Silvestre, CARE/Haiti Dr. Jean Hugues Henrys, SOE Dr. Yves Marie Bernard, USAID Dr. Antoine Augustin, CITYMED Edline Joseph Toussaint, ADRAS PROJET RICHES 2000 STUDY OF KNOWLEDGE, ATTITUDES AND PRACTICES IN MATERNAL AND CHILD HEALTH Colette Vilgrain (INTELL CONSULT) Linde Rachel Isaac Michel Pascale Crussard Mireille Sylvain July 2000 Analytic Summary CARE-Haiti has intervened in Gande-Anse for more than 13 years and has implemented the RICHES I and II projects. RICHES 2000 started in September 1997 and should finish in September 2000. The final goal of RICHES 2000 is to improve child survival by reducing child and maternal mortality and morbidity in the remote regions of Grand￾Anse, which are served by 10 health facilities. The present study evaluates the attainment of project objectives in four areas: maternal health, family planning, AIDS and STDs. The evaluation consisted of pre and posttests. The sample for the first study consisted of approximately 595 women aged 15-49 who have children less than 5 years of age and who reside in the project intervention area. The following summary shows the level of achievement for each project indicator. Maternal Health a) 73.3% of pregnant women had at least 3 prenatal visits during their last pregnancy (vs. 56% in 1997 according to the baseline study). The study also shows that the women were well educated about timing of prenatal consultations: 86.2% knew that it was important to attend a prenatal consultation during the first trimester. b) 65.3% of women received at least two tetanus vaccinations (vs. 43% in 1997). c) 78.9% of deliveries were in the assisted by qualified personnel (vs. 65% in 1997), the majority of women (57.3%) were assisted by a certified midwife and less than a quarter (21.6%) of women were assisted by a doctor or nurse. d) 57.6% of women could spontaneously list at least two pregnancy danger signs (21% in 1997). In addition, the study revealed that 11% of women reported having problems during delivery; the most common problem indicated was hemorrhaging (in more than half of the women with bleeding persisting three days after delivery); the second most common problem reported was infection during the postpartum period. e) 29.8% of women were able to prepare three materials for delivery (vs. 19.3% in 1997). f) 92.6% of women were able to list, unprompted, at least three types of nutritious foods which should be eaten during pregnancy. It is important to note as well that each of the food groups were consumed daily in 75% of cases. g) 72.1% of women brought their newborns for medical consultation during the first eight weeks after birth (vs. 69.8% in 1997). The mothers living in towns brought their children for medical consultation sooner after birth than those living outside towns (70% of children living in towns, and 44% of children living outside of town were brought in for consultation in less than a month). Family Planning a) 33% of women use modern contraceptive methods (compared to 14% in 1997). The most popular method was the same as in 1997: injections. The pill had been the second most common choice but this last study found Norplant to be the second most popular method. There has also been a decrease in condom use for family planning. b) The percentage of women who could list at least two modern contraceptive methods went from 30.8% in 1997 to 84.8% in 2000. c) According to the most recent study almost all women (99%) have heard about family planning while only 66% reported having heard of family planning in 1997. The health center is the principle source of information for 86.7% of mothers; health agents are the second most common source at 41.4% and in last place is the radio or friends. Most women could list at least one place to obtain family planning services; 90.3% of women mentioned the health center. d) 67.9% could list at least two advantages to family planning use. Concerning AIDS and STI prevention a) 32.5% of women claimed to have used a condom during sexual relations. But only 13.8% reported condom use during their last sexual encounter. Nonetheless, this constitutes an improvement compared to 1997 when only 7.5% of women reported condom use. b) Only 36.4% of women could list at least two modes of transmission for HIV/AIDS (22% in 1997). The mode of transmission known by the majority of women (81.3%) is sexual transmission. The other modes of transmission are less frequently known: for example only 19.2% of women mentioned using dirty syringes, and 6.9% of women cited blood transfusion. When the question was asked more directly, such as for mother to child transmission, the percentage reached 88.2% (otherwise only 3.2% of women spontaneously mention this mode of transmission). c) 34.2% of women could spontaneously name two STIs other than HIV/AIDS. The most frequently cited were blennorragia (56.5%) and syphilis (23.5%). d) 92.8% know that STIs are transmitted sexually (versus 75% in 1997). e) 41.6% of women could spontaneously list two ways to prevent STIs (versus 16% in 1997). The two most cited in order of frequency were condoms and fidelity. f) 71.3% of women claimed they could communicate with their partners about STIs and AIDS. This study has shown that progress has been made in knowledge, especially family planning. On the other hand, there is still some work to be done especially in relation to STI knowledge. INTRODUCTION Haiti occupies the western third of the island of Hispanola in the Caribbean. Political agitation and social injustice have contributed to the poor health status of the majority of the population. The principle causes of death among children are preventable disease such as diarrhea and acute respiratory infections. The principle causes of death among adults include cardio-vascular illness and AIDS. The total fertility rate at the national level is estimated at 4.8% per women, but this number rises to 5.8 per women in rural areas. The maternal mortality rate of 460 to 600 deaths per 100,000 live births reflects the difficulties in access to information and to reproductive health services. PROJECT DESCRIPTION The health situation is worse in the departments of Grand-Anse d’Haiti, the project’s intervention area. This is one of the most isolated and rutted regions in the country, so it is essentially cut off because of bad roads and lack of infrastructure. CARE has been working in Grand-Anse for more than 13 years and has executed three health projects in this region: RICHES I and RICHES II. These community child survival projects focused on dispensing health education directly at the community level and providing a wide range of select prevention services. The STD/AIDS prevention program works to reinforce the capacity of health institutions to reduce the transmission of STDs and AIDS. Currently, two other projects are also in progress: a) The care of persons living with AIDS project of which, the principle objective is to: improve the adaptation mechanisms among individuals living with HIV/AIDS, their families and their communities through a system of community curative and preventive care. b) The RICHES 2000 project started in September 1997 and will end in September 2000. The final goal of project RICHES 2000 is to improve child survival and reduce the rate of maternal mortality and morbidity in the regions around Grand-Anse, which are served by 10 health facilities. The RICHES 2000 project will concentrate on the following interventions: maternal health, prevention and treatment of STD/AIDS, family planning and the institutional reinforcement and partnering with 10 institutions of MSPP. Maternal health is the key element to child survival. The women’s age, parity, birth interval, health and nutritional status are among the many factors, which influence pregnancies. If we consider the prevalence of AIDS in the rural areas of Haiti and the subsistence economy in which most of the households live, we understand that the illness or loss of a family member can be disastrous. The prevention and treatment of STDs, family planning, adequate prenatal care and the utilization of institutions designated for at risk cases and clean delivery are among the interventions, which can reduce maternal mortality. The RICHES project responds, in part, to the current incapacity of rural health systems to dispense such services in an effective and consistent manner. Project Objectives: The long-term goal of the project will be to attain objectives in four areas: Maternal Health: a) Improve the percentage of women who have at least three prenatal visits at prenatal clinics during their last pregnancy from 56% to 75% b) Improve the complete tetanus vaccination coverage (TT2+) from 43% to 85% among all women c) Increase the number of women who can list at least two danger signs during pregnancy which require seeking immediate medical assistance from 21% to 65% d) Increase the number of women who understand the importance of the three food groups during pregnancy from 34.6% to 65% e) Increase the number of women whose last birth was attended by a trained person to 65% f) Increase the percentage of women who bring their child for a consultation during the first 8 weeks after birth (last child) Family Planning: a) Increase prevalence of contraceptive use among women 15-49 from 14% to 20% b) Increase from 47% to 65% the percentage of women who can list at least two modern family planning methods c) Increase from 59% to 65% the percentage of women who can list at least two sources of modern methods d) Increase from 24% to 65% the women capable of listing at least two advantages of using family planning methods STDs/AIDS a) Increase by 50% the percentage of suspected HIV+ cases which were referred for testing b) Increase the percentage of women who can spontaneously list at least two modes of AIDS transmission from 22% to 65% c) Increase from 3% to 65% the percentage of women who can list at least two STDs (in addition to AIDS) d) Increase from 75% to 86% the percentage of women who know that STDs are sexually transmitted e) Increase from 16% to 65% the percentage of the women who can list at least two methods of STD prevention f) Increase the percentage of women whose partners use condoms at least sometimes from 1% to 10% g) Increase the percentage of women who are capable of talking about STDs/AIDS with their partners from 57% to 65% Institutional Capacity: a) Each institution will increase their institutional capacity score by 50% EVALUATION OBJECTIVES The principle objective of this evaluation is to provide viable data on the current project evaluation indicators. The results should permit a final and definitive evaluation in relation to the attainment of project objectives comparing the current with the baseline data. METHODOLOGY The impact evaluation is a pretest-posttest design in which the same indicators are measured two times: during the baseline and again during the second study at the end of the project. The methodological propositions that follow are for the final studies. According to the project evaluation indicators and the baseline data, the study targets a population, which fits the following criteria: a) women of reproductive age (15-49 years) and b) mothers of children 0-5 years of age. Like the baseline study, the impact study uses two methodological approaches: a study of women at the household level and an institutional diagnostic study at the partners’ structural levels. The second study results will be reported in a separate report. Sample Following the baseline study, the cluster sample methodology was used in order to guarantee the comparability of the two studies. The key indicators are based on the proportion of women who benefited from the project through the improvement of knowledge, practices and access to health services. The following formula was used as recommended for the estimation of proportions (populations for which the exact population number is not known): N = z2 *c(p(100-p))/a2 N = 14.76*1.5(50*50)/102=553 women This number was increased to 595 to palliate the problem of non-response. z = confidence interval (1.96 for a 95% degree of confidence) p = proportion a= desired precision (10) Choice of clusters Once the sample size was determined, a systematic random sample was taken to choose the locations constituting the sample cluster. Thus, from a list of all the localities in the project area submitted by the head of the project, 35 clusters were chosen from the 58 on the list. These 35 clusters were distributed among the 8 parish communities included in the study. Selection of women within clusters The household was considered a basic unit. The criteria guiding the choice of households were the existence in the household of a woman between 15 and 49 years having a child less than 5 years old. Using a geographic method, the supervisors, with the help of guides, determined the center of the locality and from the center; each interviewer went in a direction and selected households meeting the criteria. One woman was interviewed per household until the number of women to be interviewed was reached (17 women). Data collection instrument The basic data collection instrument used for this study was the questionnaire created for the baseline study. The key questions were asked in the same manner during the baseline study. The team revision of the instrument permitted the removal of certain unused questions, the addition of certain supplementary questions and the improvement of the format to facilitate immediate entry of the data (parallel to collection). Data collection team training In order to assure the mastery of the data collection instrument, a 5-day training was conducted with the staff from May 29th – June 1st. This training was assured by the coordination team and supported by the CARE training office. During the training period, the following themes were covered: - Interview techniques - Study objective - Methodology - Sample size - Questionnaire contents - Technical details in filling out the questionnaire Also, to ensure comprehension and in order to standardize the interviewing techniques, practice exercises were conducted by interviewers such as role plays and simulations. The data collection phase lasted twelve days, from June 5th – 17th, 2000. Three teams of four (4) interviewers and one (1) supervisor worked under the supervision of a coordinator who served as the team leader in the field. Quality control of data To ensure the reliability and validity of the collected data, two levels of supervision were established: - the first verification was made by the supervisors who, at the end of each work day, would meet with the interviewers to review the completed questionnaires and make necessary corrections - the second level of verification was made by the verifier assigned to this task. This person went back to the field to systematically verify 105 randomly selected complete questionnaires each day. Based on the errors found, recommendations were made in order to minimize the entry of inaccurate data. Data entry and cleaning The data collection phase stared two days after the first data were collected. This phase was executed in the field parallel to the collection activities into ISSA (Integrated System for Survey Analysis). This procedure allowed for the systematic correction of possible errors during completion of questionnaires. The verification of the data entered followed double entry and data file comparisons. CARE International Haiti Reproductive Health Project (PN-56) PROJET RICHES 2000 Results from an institutional survey on the management capacity of eight partner institutes in the eight towns of la Grande Anse By MTIE/CARE July 2000 CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale i Acknowledgements The achievement of this Institutional Diagnostic study , led by project partners of RICHES of CARE-Haïti, is due to a number of people both within and outside of CARE who worked beyond their usual responsibilities for the project's success. First, it is necessary to thank the head of project RICHES, Mrs. Paula S. Brunache, who supported the data collection process from beginning to end. We are equally grateful to the advisor and assistant technical advisor of CARE-Haiti, to the advisor of follow up assesment of CARE-Haiti, and to the staff at MTIE who facilitated the data collection,analysis of data and the production of the final report. We would also particularly like to express our thanks to those individuals at the health institutions who acted as partners of project Riches 2000 and who agreed to sacrifice their time to work for our cause . Finally, to all of those who contributed in one way or another to the realization of this study, we hope you find in this document a profound expression of our gratitude. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale ii PROJECT STAFF Co-ordination Team : Linde Rachel, Advisor of follow up assesment, CARE Pascale Crussard, Health Technical Advisor, CARE Paula Brunache, Head of Project RICHES 2000, CARE Mireille Sylvain, Assistant Health Technical Advisor , MTIE-CARE Isaac Michel, Co-ordinator MTIE-CARE Collection of data : Patrick Vieux, Officer of Collection, MTIE-CARE Woudy Thomas, Officer of Collection, MTIE-CARE Data entry : Dieunaille F. Exil, Administrative Assistant, MTIE-CARE Data analysis : Woudy Thomas, Officer of Collection, MTIE-CARE Myrtho Joseph, Data Management Coordinator, MTIE-CARE Drafting the Report : Woudy Thomas Patrick vieux Mireille Sylvain Isaac Michel Linde Rachel CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale iii TABLE OF CONTENTS Page Acknowledgments i Program Staff ii Table of Contents iii 1. Introduction 1 2. General Framework of the Project 1 3. General Framework of the Survey 1 3.1 Survey Objectives 1 3.2 Methodology of the Survey 1 3.3 Hypothesis and Defined Indicators 2 4. Evaluation Report 4 4.1 General Presentation 4.2 Staff and Available Services 5 4.2.1 Technical Staff 5 4.2.2 Administrative Staff 6 4.2.3 Available Services 6 4.3 Health Facilities 7 4.4 Institution Profiles 8 4.5 Community Understanding 9 4.5.1 Population / Health Indicators 9 CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale iv 4.5.2 Community 10 4.5.3 Communty Agencies 11 4.6 Economic Planning 12 4.7 Management of Human Resources 13 4.8 Management of Information Systems 13 4.9 Management of Materials and Logistics 15 4.10 Financial Management 15 4.11 Maternal Care 17 4.12 Information, Education, Communication 17 4.13 Conclusion 19 ANNEXES A . Level of Institutional Development 21 B . Summary Table 21 C. GRAPHS Figure 1 : Level of Institutional Development 22 Figure 2 : Institution Profiles 23 Figure 3 : Economic Planning 24 Figure 4 : Management of Human Resources 25 Figure 5 : Management of Information Systems 26 Figure 6 : Management of Stock 27 Figure 7 : Financial Management 28 Figure 8 : Information, Education, Communication 29 D. Survey Questionnaire 30 CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 1 1 INTRODUCTION In an effort to evaluate the integrated project called Reproductive Health (PN 56), CARE International Haiti and the regional health staff conducted an interview about the intstitutional development capacity of their health partners in Grande Anse region. The partners interviewed in this study included the following : le Dispensaire des Abricots, le Dispensaire Bon Samaritain des Roseaux, le Dispensaire Chemin La Vie de Bonbon, the AFSC Helath Center des Irois, Chambellan Health Center, the Anse d’Hainault Health Center, the Moron Health Center and the Dame Marie Health Center. 2 GENERAL FRAMEWORK OF THE PROJECT The reproductive health sector is included in the national health policy and is one of the key elements of minimal services packages. One of the risk indicators projected by the National Health Plan forecasts a drop in fertility rates from 4.8 children per woman to 3.5. Other aspects of the National Health Plan also deserve to be mentioned. Within the domain of reproductive health there is an objective to lower the rate of maternal mortlaity from 4.6/1000 to 1/1000. The fundamental elements of CARE’s Reproductive Health Project are built on our commitment to the following: 1. Family Planning / Reduction in fertility rates 2. Identification and care for pregnant women 3. Vaccination of women of child bearing age 4. Identification of emergency gynecological surgeries 5. Prevention and control of STDs and AIDS 3 GENERAL FRAMEWORK OF THE SURVEY 3.1 Survey Objectives This study was designed to evaluate the capacity of partner institutions to provide minimum service packages in Reproductive Health. 3.2 Survey Methodology The questionnaire used in the baseline survey served as a tool for data collection for the institutional diagnostic. The following activities show the range of management interventions that were analyzed as part of the interview : - Institutional profile - Level of community understanding CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 2 - Planning Capacity - Management of human resources - Information systems - Management of material and logistics - Financial Management - Health Facilities - Magagement of maternal care - IEC Each section of the module is composed of a certain number of questions that are each assigned a particular score: one point for each acceptable answer and zero for each less than acceptable answer. Thus, an institutional stage of development is established for each section and each module. However, it is necessary to understand that the tools used for this study were not exactly the same as those used for the baseline study. Since the baseline study was completed modifications have been applied to the current tool, since a comparison of scores was originally difficult to ascertain. 3.3 Hypothesis and defined indicators To track the levels of relevant progress in each institution, ideal qualities with corresponding indicators were defined from the beginning. These qualities describe the management of an ideally run institution : · Favorable initial management indicators, demonstrating the potential of the institution to accept the project: management experience with other projects, staff stability, presence of an advisor, successful integration with other projects. · A sufficient level of community understanding including understanding the geography, socio-economics and political point of view of the target community. · An efficient planning system which is part of a dynamic process, presents strategic and operational plans and integrates all levels of execution, thus beingflexible. · A management of human resources which follows pre-established norms, periodically evaluates personnel, uses evaluation for planning, and takes into consideration staff reccommendations. · An institutional information system that gives every level of management timely access to information, permitting timely decision making and improvements in management practices. · A supplies management system based on needs and with access to warehouse facilities CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 3 · A financial management system that plans necessary resources, protects its assets, gives pertinent reliable financial information that allow managers to make directed decisions and produces a fixed price scale of costs and services based on anticipated future need. · Adequate health facilities - human and material resources necessary to offer quality services. · Good management of material goods including an information system, a well￾maintained physical location, and well trained personnel. · The existence of an IEC service capable of planning the education of concerned communities, including requisite equipment and trained personnel. EVALUATION REPORT The survey was conduted at the institutional level by M. Woudy Thomas and M. Patric Vieux of MTIE. 4.1 General Presentation a) Institutional Category Those institutions that had been a part of the baseline study constituted our group of interview sites. They were as follows: the Moron, Chambellan, and Irois health centers (without beds); the Dame Marie and l’Anse d’Hainault health centers (with beds); and the Roseaux, Bonbon and Abricots clinics. Table 1 : Category of institutions Institution HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinic Roseaux Clinic Catategory Health Center with out beds Health Center with out beds Health Center with beds Health Center with beds Health Center with beds Clinic Clinic Clinic b) Type of institution The health centers of Moron and l’Anse d’Hainault and the Bonbon and Abricots Clinics are public institutions under the management of the health department of Grande Anse. The Chambellan, Dame-Marie and Irois health centers as well as the Roseaux clinic are privately managed but have members of their personnel that are paid by the Ministry of Public Health and Population (MSPP). These health facilities are listed as “mixed“ in the following table. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 4 Table 2 : Type of institution Institutions HCWOB Moron HCWOB Chambellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainau lt Bonbon Clinic Abricots Clinic Roseaux Clinic Type Public Mixed Mixed Private Public Public Public Private 4.2 Personnel and services offered 4.2.1 Technical staff As we can see from the following table, the number of qualified technical staff is insufficient in all the communities, although certain improvements were noted following support provided by Cuban medical staff. At the community level, the staff (AGS et colvols ) seemed to be available – in fact 7 of 8 institutions maintain community workers; however in order to fully analyze the table, it is important consider the ratio of AGS to colvols available to the client population. Table 3 : Technical staff Personnel HCWOB Moron HCWOB Chambellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinic Roseaux Clinic General Practitioner 2 1 3 1 1 0 0 0 Specialist 0 0 0 0 0 0 0 0 Médecin SS 0 2 1 0 0 0 0 0 Sanitary Nurse 0 0 0 0 0 0 0 0 Nurse 2 0 2 1 3 0 1 1 Infirm. SS 0 0 1 0 0 0 0 0 Nurse’s Aid 0 4 4 3 3 1 1 1 Nurse’s Aid’s Aid 0 1 0 0 0 0 0 3 Health Agent 2 6 3 14 2 0 1 1 Bookkeeper 1 0 0 1 2 0 1 0 Lab Tech 0 1 2 1 0 1 0 0 TB Agent 0 0 1 1 0 0 0 0 CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 5 Personnel HCWOB Moron HCWOB Chambellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinic Roseaux Clinic CHW 32 54 25 60 25 15 10 0 Dentist 0 1 1 0 0 0 0 0 CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 6 4.2.2 Administrative Personnel At this time only those institutions with beds have administrators and bookkeepers on their administrative staff. Their administrative sections were completely closed at the time of the data collection, which did not allow us a response to certain administrative questions. Holding to the the administrative training strategy of the project, we have to evaluate the functional level of this structure and analyse the polyvalence of technical staff appointed to other institutions. In the institutions like Roseaux, certain positions are polyvalents; they are named employees and fill the need for filing and other related tasks. Table 4 : Administrative Personnel Personnel HCWOB Moron HCWOB Chambellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Administrator 0 0 1 0 1 0 0 0 Bookkeeper 0 0 1 0 1 0 0 0 Drug Clerk 1 1 1 1 1 0 1 - Secretary 1 1 - - - 0 0 - Driver 0 1 - 1 - 0 0 0 Gardener - 1 - - 1 0 1 - Cleaner - 1 - - 1 2 1 - Logistician 0 1 - - 0 0 0 - Cashier 0 0 - - 1 0 0 - 4.2.3 Services Offered In general all the institutions surveyed offer packages that are somewhat comprehensive in providing reproductive health services with the exception of the Roseaux clinic, which does not offer family planning services because of its religious affiliation. The Bonbon clinic does not offer pre- or post-natal consultations because a maternity clinic that offers these services is located less that 300 metres from the clinic. The Bonbon clinic does, however, allow patients to fill their prescriptions at their location. We found that Anse d’Hainault does not do vaccinations and does not have a refrigeration system. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 7 Table 5 : Services Offered Services Offered HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCW OB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinc Roseaux Clinic General Consultations yes (1)1 yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Prenatal Consultations yes (1) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) Postnatal Consultations yes (1) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) Family Planning yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) no (0) Vaccination yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) yes (1) IEC yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) 4.3 Health Facilities In general all the institutions, according to classifications, have health facilities available and adequate staff to deliver services. We were not able to determine if the clinics work in appropriate areas in all cases. However, we are not in a postition to speak of their vulnerability, because we have not investigated the details concerning upkeep, building maintainance, insurance of equipment or the fire extinguisher. Table 6 : Health Facilities Health facilities HCWOB Moron HCWOB Chambellan HCWB Dame Marie HCW OB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinic Roseaux Clinic Consultation Room (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Warehouse (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) no (0) yes (1) First Aid Room (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) 1 The number in the parentheses gives the score obtained by each institution marking the existance or lack of service considered. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 8 Health facilities HCWOB Moron HCWOB Chambellan HCWB Dame Marie HCW OB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinic Roseaux Clinic Waiting Room (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Toilet (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Lab room (1) yes (1) yes (1) yes (1) yes (1) yes (1) no (0) no (0) yes (1) Dental Room (1) no (0) yes (1) yes (1) no (0) no (0) no (0) no (0) no (0) Pharmacy (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Hospital Room (1) no (0) no (0) yes (1) no (0) yes (1) no (0) no (0) no (0) Space for IEC (1) no (0) no (0) yes (1) no (0) yes (1) yes (1) yes (1) no (0) Upkeep (1) - - - - - - - - Building Maintenance (2) - - - - - - - - Equipment Insurance(5) - - - - - - - - Fire Extinguisher - - - - - - - - Responsible staff - - - - - - - - 4.4 Institution Profile Among the institutions we found two that have an administrative board. At the Dame Marie health center there is a local board and at the Irois HCWOB certain staff members make up a local administrative supervision board (AFSC). Generally, they have an organization chart with fairly stable staff. The legal statute is not available at the majority of institutions, particularly in institutions that are part of the public sector and most likely function with legal provisions from MSPP. As for the documentation of budget, we have not found, at the time of the study, a single document for a current budget. In contrast, the data shows that there are guidelines used for the use of generated revenue. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 9 Table 7 : Institution Profile PROFILE HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWB Irois HCWB Anse d’Hainault Bonbon Clinic Abricots Clinic Roseaux Clinic Administrative Board (3) no (0) no (0) yes (1) yes (1) no (0) no (0) no (0) no (0) Organizational Chart (3) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) no (0) Legal Statute (1) no (0) no (0) yes (1) no (0) yes (1) no (0) no (0) no (0) Stable Staff (4) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Documented Budget (4) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) 4.5 Community Understanding 4.5.1 Population / Health Indicators Those responsible at each institution know the population that they serve at each level of their institutions. These numbers are registered in the report on the baseline institutional survey and the micro-projects, and have not been updated since the last census which dates from 1982. However, those responsible feel confident that they have accurate statistics about their clientele’s population, but feel that they have a very weak knowledge of other health indicators. They feel certainthat the population numbers are held by the departmental heads at MSPP. Worth noting is that those responsible at HCWOB Irois and at HCWB d’Anse d’Hainault know the prevalence of family planning, and at Abricots clinic administrators have access to data about access to potable drinking water. Table 8 : Population / Health Indicators Indicators and Population HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Population served and registered (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Family Planning prevalence (5) no (0) no (0) no (0) yes (1) yes (1) no (0) no (0) no (0) Maternal no no no no no no no no CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 10 Indicators and Population HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Mortality (5) (0) (0) (0) (0) (0) (0) (0) (0) Infant Mortality (5) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) Vaccine Coverage (5) no (0) no (0) no (0) yes (1) no (0) no (0) no (0) no (0) Potable Water Coverage (5) no (0) no (0) no (0) no (0) no (0) no (0) yes (1) no (0) Rate of Malnutrition (5) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) 4.5.2 Community Virtually all the institutions work with professionals in the community such as leaders, local elected officials and religious authorities who actively participate in the selection of agents and colvols and in the realization of community activities. Nonetheless, the institutions have not conducted a single study with thorough understanding in this area, which explains why these institutions have not been able to quantify their objectives or to have operational objectives. Table 9 : Community Community HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Understanding of Traditional Entities (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Community Meetings (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Home Visits (2) no (0) yes (1) no (0) yes (1) no (0) no (0) yes (1) yes (1) Meeting Posts (5) yes (1) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) Health Committee (10) no (0) yes (1) yes (1) yes (1) yes (1) no (0) no (0) no (0) Clubs of Satisfied Clients (10) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) Meeting w / the associations (10) no (0) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) no (0) Meeting with the leaders (10) yes (1) yes (1) yes (1) yes (1) no (0) no (0) yes (1) no (0) Course in Pediatric Nursing (5) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 11 4.5.3 Community Agents All of the institutions have CHWs and Community Agents. The majority of community agents have documents and descriptions of their duties. These Community Agents more or less know the population they serve and their own coverage objectives. They receive basic health care training in reproductive helath and distribute contraceptives. CHW activities are evaluated monthly in a majority of institutions. It is important to note that the Bonbon clinic has neither a CHW nor a Community Agent. Table 10 : Community Agents HA / CHW HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Have CHW / Health Agent yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Description of job for HA (5) yes (1) no (0) yes (1) yes (1) no (0) n/a - yes (1) yes (1) HA are trained in SR (10) yes (1) yes (1) yes (1) yes (1) yes (1) n/a - yes (1) yes (1) Population served know by HA(5) no (0) no (0) yes (1) no (0) yes (1) n/a - yes (1) yes (1) Objectives known by HA (5) no (0) no (0) yes (0) no (0) no (0) n/a - no (0) no (0) HA distributing contraceptives (5) yes (1) yes (1) yes (1) yes (1) yes (1) n/a - yes (1) yes (1) Availabliity of reference sheets for HA(2) no (0) yes (1) yes (1) no (0) no (0) n/a - no (0) no (0) Monthly evaluation of HA (5) yes (1) yes (1) yes (1) yes (1) no (1) n/a - yes (1) no (0) 4.6 Planning Sector Of the eight institutuions evaluated, four were found to have written mission statements and two (HCWOB Moron and HCWOB Irois) to have a strategic plan. It must be said that these missions statements, when they do exist, are not specific to the individual institution, but are based on the general statement given by MSPP. Within the contraints of available human resources, each institution works to ensure that planning CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 12 allows for institutional efficacy. Each institution has a calendar of activities which it follows and by which activities are evaluated. Table 11 : Identified strenths and weaknesses Planning Capacity HCWO B Moron HCWO B Cham￾bellan HCWB Dame Marie HCWO B Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Available written Mission Statment (10) no (0) yes (1) no (0) yes (1) no (0) no (0) yes (1) yes (1) Available Strategic Plan (10) yes (1) no (0) no (0) yes (1) no (0) no (0) no (0) no (0) Planning by sector (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Objectives (10) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Calender of activities (10) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Evaluation and follow through on activities (10) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 13 4.7 Management of Human Resources Certain management tools can help institutions manage human resources responsibilities. Those institutions running most efficiently maintain : a letter of nomination (6/8), a job description (4/8), degrees or certifications (5/8), and attendance sheets (6/8). In contrast, certain files, such as personnel manuals, evaluation and performance documents, personnel hiring sheets, and vacation/time-off sheets (6/8) were not often found during this review. This shows the weakness of these institutions in their capacity to follow and evaluate the performance of their personnel. Table 12 : Strengths and weaknesses at the human resource level Human Resources HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Job description (10) no (0) no (0) yes (1) yes (1) no (0) no (0) yes (1) yes (1) Letter of recommenda￾tion(2) no (0) yes (1) yes (1) no (0) yes (1) yes (1) yes (1) yes (1) Degrees/ certificates (2) no (0) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) no (0) Time off sheets (2) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) Attendance Sheets (2) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) no (0) Personnel Manuel(5) no (0) no (0) yes (1) yes (1) no (0) no (0) no (0) no (0) Performance Evaluation (10) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) Formation Plan (5) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) 4.8 Management of Information Services Each of the institutions has an information system that, at minimum, keeps statistics on services and periodic reports. Certain of these reports are often transported to the central office of the Ministery of Public Health. Above all, these reports allow institutions to make better decisions regarding the allocation of resources that are available to prevent loss of stock. These information systems have been further developed since the periodic collection of goods. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 14 With very little means, the institiutions manage their information system in a fairly effective manner. The sanitary goods are regularly registered by personnel staff, the health agents, and teams of volunteers. There is also a good system tied to the utilization of goods considered in making a decision, including consideration of administrative acitivities and programming. However the indicators of program impact are not measured. Table 13 : Strengths and weaknesses of the management of information services Information System HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hain ault Bonbon Abricots Roseaux Registry of Clients PF(1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) no (0) PF Client Identification Cards(1) yes (1) no (0) no (0) no (0) yes (1) yes (1) no (0) n/a - Indicators of the volume and quantity of services offered (1) yes (1) no (0) no (0) yes (1) no (0) no (0) no (0) n/a - Indicators of the impact of services (2) no (0) no (0) no (0) no (0) no (0) no (0) no (0) n/a - Collection of data (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Preparation and mailing of regular and monthly reports (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Utilization of information for decision making (2) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Follow up system yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 15 4.9 Management of Materials and Logistics In almost all of the institutions, inventory is satisfactorily managed. The institutions put the majority of their needs in inventory, and the management tools most utilized are the stock inventory forms and the report cards. Certain others also have warehouses for their goods, however they lack space and do not have designated space for supplies or medicines. For example, at the Abricots health center the same space that serves as a warehouse also serves as a room where wounds are dressed. Nonetheless, only the Roseaux clinic has a document on the inventory management procedures and only three institutions regularly make inventory reports. Table 14: Strenths and weaknesses of the management of materials and logistics Management of goods HCWOB Moron HCWOB Cham￾Bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Haina ult Bonbon Abricots Roseaux Warehouse(10) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) no (0) yes (1) Management procedures for goods and inventory (10) no (0) no (0) no (0) no (0) no (0) no (0) no (0) yes (1) Regular inventory of goods (10) no (0) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) no (0) Inventory forms (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Stock reports (5) yes (1) yes (1) no (0) no (0) no (0) no (0) yes (1) no (0) Projection of Needs (5) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) yes (1) 4.10 Financial Management The majority of institutions do not have a proper budget. According to some, a global budget that covers all health centers is made annually by the central office of the Ministry of Public Health. In any case, the management of financal resources is accounted for in every institution. There is a regular inspection of daily transactions that is completed from within the institutions, with particular emphasis on the receipts coming from drug sales, consultations, and laboratory testing. The reception registry and dispersal of funds and the minimal budget for management salaries are managed externally. Institutions have to permit the analyses of financial information and produce an inspection report. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 16 Table 15: Identified Strengths and Weaknesses of Financial Management Accounting and Finances HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Registry for the reception and spending of funds (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Inventory of goods (2) - - - - - - - - Minimum budget for salaries yes (1) yes (1) yes (1) no (0) yes (1) yes (1) yes (1) yes (1) Analysis of financial information (10) yes (1) no (0) yes (1) yes (1) no (0) yes (1) yes (1) yes (1) Report of budget inspection (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Classification of pieces by project and period (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Assessment (5) - - - - - - - - External Audit (5) no (0) yes (1) yes (1) no no yes (1) yes (1) No Bookkeeping procedures (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Availability of a basic system for follow up on financial management (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 17 4.11 Maternal Care In maternal health, huge efforts were made by the institutions to provide good coverage of women of reporductive age. The general services of maternal care are provided at all institutions. The referral system for obstetric emergencies is not working in the majority of them (4/8) although the communication network with mothers is present in all institutions. Table 16 : Identified strengths and weaknesses concerning maternal care Maternal Care HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Med OBGYN (5) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) Pre- and post-natal clinic (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Essential Drugs (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Information Systems (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Refrigerator (2) yes (1) yes (1) yes (1) yes (1) no (0) yes (1) yes (1) yes (1) Delivery Room (2) no (0) no (0) yes (1) no (0) yes (1) no (0) no (0) no (0) System of emergency referrals (5) yes (1) no (0) yes (1) yes (1) no (0) yes (1) no (0) no (0) Collaboratio n with the mothers (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) 4.12 Information, Education, and Communication In the majority of these institutions there is a staff member trained in counseling and one trained in IEC. About half of the institutions have IEC materials and an appropriate space for counseling. CARE has worked hard to partner with these institutions to reinforce the strength of their personnel in the area of Information, Education, and Communication. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 18 Table 17 : Identified Strengths and Weaknesses of IEC COUNSELLING HCWOB Moron HCWOB Cham￾bellan HCWB Dame Marie HCWOB Irois HCWB Anse d’Hainault Bonbon Abricots Roseaux Space for IEC (10) no (0) no (0) yes (1) no (0) yes (1) yes (1) yes (1) no (0) Personnel trained in counseling (10) yes (1) yes (1) yes (1) yes (1) no (0) no (0) yes (1) yes (1) IEC Calender (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) Availability of a trainer (10) no (0) no (0) no (0) no (0) no (0) no (0) no (0) no (0) Training material for IEC (5) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) yes (1) CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 19 4.13 CONCLUSION The baseline study on the organizational capacity strengthening of the partner institutions of Project RICHES 2000, conducted in 1996 revealed the weakness of these institutions to manage support systems for health services.Taking into account the baseline survey results, a curriculum was put in place with the hope of reinforcing and strengthening the capacity of these institutions. The objective of this recent study was to analyze the efforts carried out by measuring the updated level of indicators used in the initial study conducted. In effect, the baseline data was fixed at 50% of the effort that should have been accomplished by the institution in terms of increase in performance score. Based on results abtained recently, the project largely surpassed this objective. From 19% in 1996, the level passed to 45%, which indicated an increase of 131%. These results have allowed each of the eight institutions in the study to pass from an organization at a newly developing level to a stage of subsequent development (6 institutions) or to a stage of growth (2 institutions). There is no doubt that the investment in the project for strengthening and reinforcing the capacities of these institutions has triggered results that go far beyond that which was expected. However, the absence of a phase-out structure makes it difficult to say much about the sustainability of these efforts. It is certain that the members of related institutions benefitted from these training sessions. Also the material resources provided by the project were of great benefit to the institutions and were a major output of the program. It is our hope that these results are sustained over time. CARE would be very interested in conducting a similar study in another year to verify improvements resulting from our activities. CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 20 ANNEXES CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 21 Resume of scores obtained by each institution by category INSTITUTIONA L PROFILE POPULATION & SANITARY INDICATORS PLANNING CAPACITY MANAGEMENT OF HUMAN RESOURCES MANAGEMENT OF INFORMATION SERVICES INVENTORY MANAGEMENT FINANCIAL MANAGEMENT IEC TOTAL SCORE % SCORE 15 35 55 38 10 45 44 40 282 Moron 6 5 32 2 7 20 27 16 115 40.8 Chambellan 6 5 32 6 5 28 23 16 121 42.9 Dame-Marie 10 5 24 18 5 24 31 26 143 50.7 Irois 9 13 40 16 6 16 25 16 141 50.0 Anse - d’Hainault 7 9 24 4 6 20 19 18 107 37.9 Bonbon 6 5 24 4 6 16 31 18 110 39.0 Abricots 6 9 32 14 5 20 31 26 143 50.7 Roseaux 6 5 32 12 5 24 27 16 127 45.0 Comparison of the organizational capacity of institutions by stage Level Phase of Initial Development Phase of Subsequent Development Phase of Growth Phase of Consolidation Score 0 - .25 .26 - .50 .51 - .75 .76 - 1.0 1997 All eight Institutions 2000 6 Institutions Moron, Chambellan, Irois, Anse d"Hainault, Bon-bon, Roseaux 2 Institutions Dame-Marie, Abricots CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 22 22 40.8 42.9 50.7 50.0 37.9 39.0 50.7 45.0 0.0 10.0 20.0 30.0 40.0 50.0 60.0 Percentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 1: INSTITUTIONAL LEVEL OF DEVELOPMENT CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 23 23 40.0 40.0 66.7 60.0 46.7 40.0 40.0 40.0 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 Pourcentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 2: INSTITUTIONAL PROFILE CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 24 24 58.2 58.2 43.6 72.7 43.6 43.6 58.2 58.2 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 Percentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 3: PLANNING CAPACITY CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 25 25 5.3 15.8 47.4 42.1 10.5 10.5 36.8 31.6 0.0 5.0 10.0 15.0 20.0 25.0 30.0 35.0 40.0 45.0 50.0 Percentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 4: HUMAN RESOURCE MANAGEMENT CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 26 26 70.0 50.0 50.0 60.0 60.0 60.0 50.0 50.0 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 Percentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 5: MANAGEMENT OF INFORMATION SYSTEMS CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 27 27 44.4 62.2 53.3 35.6 44.4 35.6 44.4 53.3 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 Percentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 6: MANAGEMENT OF INVENTORY CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 28 28 61.4 52.3 70.5 56.8 43.2 70.5 70.5 61.4 Moron Chambellan Dame-Marie Irois Anse -d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 7: FINANCIAL MANAGEMENT CARE-Haïti, Projet RICHES 2000: Diagnostic Institutionnel, Evaluation Finale 29 29 40.0 40.0 65.0 40.0 45.0 45.0 65.0 40.0 0.0 10.0 20.0 30.0 40.0 50.0 60.0 70.0 Percentage Moron Chambellan Dame-Marie Irois Anse - d’Hainault Bonbon Abricots Roseaux INSTITUTION FIGURE 8: INFORMATION, EDUCATION, AND COMMUNICATION