Project HOPE in Peru CSXII Final Evaluation 1 DRC 7979 Project HOPE Improving Maternal-Child Health in the Huallaga Valley of Peru CHILD SURVIVAL XII: 1996-2000 Cooperative Agreement No. FAO-0500-A-00-6056-00 FINAL EVALUATION September 2000 San Martin, Peru Renee Charleston, External Consultant Contact persons at Headquarters: Contact person in the Field: Juan Carlos Alegre, M.S. Sandra Contreras, R.N., M.P.H. Program Manager National Director - Perú Luis Benavente, M.D., M.Sc. Associate Director, MCH Programs Project HOPE in Peru CSXII Final Evaluation 2 Table Of Contents A. Summary ……………………………………………………………………...…. 1 B. Assessment of Results and Impact of the Program ……………... 3 1. Summary Chart ……………………………………………………….... 3 2. Technical Approach ……………………………………………….….… 4 a. Overview …………………………………………………………. 4 b. Interventions …………………………………………..………. 5 a. Nutrition ………………………………………….…………. 5 b. Breastfeeding …………………………….……………….. 7 c. Control of Diarrheal Diseases ……….………………. 10 d. Family Planning ………………………..…………………. 12 c. New Tools and Approaches ……………..………………… 12 3. Cross-cutting approaches ……………………………………….…… 13 e. a. Community Mobilization …………………..…………… 13 f. b. Communication for Behavior Change …….………. 15 g. c. Capacity Building Approach …………………….……. 16 h. d. Sustainability Strategy ……….………………………… 22 i. C. Program Management ………………….…………………………………. 24 j. 1. Planning ………………………………..……………………………… 24 k. 2. Staff Training …………….…………………………………………. 25 l. 3. Supervision of Program Staff ………….………………………. 25 m. 4. Human Resources and Staff Management ……….………. 26 n. 5. Financial Management …………………………………………….. 26 o. 6. Logistics ………………………….……………………………………. 27 p. 7. Information Management ………………………………………. 27 q. 8. Technical and Administrative Support ……………………… 30 9. Management Lessons Learned ……………………………….. 31 r. D. Conclusions and Recommendations ……….……………………………. 31 s. E. Results Highlight-Fortified Food ……………………………………………. 37 ATTACHMENTS A. Team members and their titles B. Assessment methodology C. List of persons interviewed and contacted D. Results of the Final Evaluation Interviews Project HOPE in Peru CSXII Final Evaluation 3 E. Results of the Final Evaluation Workshop F. Special reports-Final Survey Results-KPC, Food Consumption and Biochemical Surveys ACRONYMS AND ABBREVIATIONS AIN Atención Integral del Niño (Integrated Care of the Child) ARI Acute Respiratory Infection BF Breastfeeding CDD Control of Diarrheal Disease CEPCO Eastern Center of Studies and Community Promotion CHV Community Health Volunteer CS Child Survival Project DIP Detailed Implementation Plan DIRES-SM Regional Ministry of Health - San Martín EBF Exclusive Breastfeeding FE Final Evaluation FP Family Planning GALME Grupo de Apoyo a la Lactancia Materna Exclusiva (EBF Support Group) GM/P Growth Monitoring and Promotion HIS Health Information System IMCI Integrated Management of Childhood Illness KPC Knowledge, Practices and Coverage Survey LHC Local Health Committee MINSA Ministry of Health MTE Mid Term Evaluation NGO Non-Governmental Organization ORS Oral Rehydration Solution ORT Oral Rehydration Therapy PANFAR Program of Food & Nutrition for High Risk Families PCM Pneumonia Case Management PRISMA Projects in Information, Health, Medicine, and Agriculture PVO Private Voluntary Organization UPCH University Peruana Cayetano Heredia UROC Community level Oral Rehydration Unit USAID United States Agency for International Development Project HOPE in Peru CSXII Final Evaluation 1 A. Summary The Project HOPE Child Survival XII project funded by USAID (10-96 to 9-00) focuses on community organization, training and supervision of CHVs, training and motivation of Ministry of Health (MINSA) field staff, and community education. Four intervention areas were included; Nutrition and Micronutrients, Breastfeeding, Control of Diarrheal Disease, and Family Planning. The project is located in Región San Martín of central Perú on the eastern slope of the Andes and includes three Health Networks: El Dorado, Lamas, and San Martin. The principal partners in the project are MINSA and the University Peruana Cayetano Here dia. The original four-year project has been awarded a three-year extension to continue working in the same 162 communities with the introduction of malaria and respiratory infections, and improve collaboration with MINSA, by transfer of responsibilities, training and supervision, and institutional strengthening. It is difficult to appreciate the impact of the project due in part to the use of indicators that do not reflect what the project has accomplished in the last four years. Only three of the 13 objectives were actually met during the four-year period. This can be partially explained by an overly ambition proposal and not taking into consideration factors which were outside of the scope of the project. The project has shown positive impact in two important areas; women who exclusively breastfeed until 6 months and stunting in children. Other improvements include frequency of growth monitoring, initiation of breastfeeding and use of family planning methods. Some of the major achievements of the project include: · the development of a very effective educational methodology · encouraging the valuation of local foods-not donated or processed foods, · improved quality of nutritional surveillance, · development of formula for fortified foods, · studies which have allowed: focused messages, better understanding of complex disease states, and improved information for use by MINSA officials, · development of leadership skills and greater solidarity in women, Major Problems that still need to be addressed during follow-on project: · Information System is not fully functioning · Supervision and monitoring is weak at all levels · Collaboration and communication with MINSA needs to be further improved · Defining the needs for capacity building and institutional strengthening · Further strengthening of community structures Priority Lessons Learned: · The development of activities which respect and value the local culture have the greatest impact Project HOPE in Peru CSXII Final Evaluation 2 · For adequate implementation, the project should involve all partners (MINSA HOPE Community) from the beginning · Continuous follow-up and support are needed for the development of new abilities · The greatest impact in changing practices comes with clear, simple messages, based on previous study · Take advantage of the felt need of women to meet in their own group, according to their own vision · To decrease the prevalence of diarrheal disease it is necessary to improve sanitation conditions through coordination with other organizations · A project can not advance without the support of local authorities. · The use of checklists clarifies expectations and defines what elements are essential · Other members of the community need to be trained to create an environment of change Priority Recommendations: · The project should select one measure for determining nutritional status and focus training on counseling the family · Alliances should be formed with organizations that have financing and/or offer technical assistance in sanitation and agriculture. · Strengthen the UROC as a community structure with an integrated preventative health role in the community. · Implement family planning activities by involving men · Training for committee members and leaders in community organization should be prioritized during the next 3 years. · MINSA and HOPE should strengthen follow-up to insure the quality of replication of training and the education in the communities. · HOPE should seek technical assistance on institutional assessments from other NGOs who have experience in this area. · Both a formal and informal system of communication should be developed between HOPE and MINSA. · The strategy of using CHV couples should be re-evaluated during the second phase of the project. · Establish a system of indirect supervision using regular monthly meetings between health post staff and all CHVs. · Create Trainers within MINSA in health, gender and educational methodologies. · Create Trainers within the municipalities and teachers in community development, leadership and participative methodologies. · The DIP for the 3 year extension should be developed in a participatory manner, involving community, and MINSA at all levels. · A comprehensive study of needs and an information use plan for the development of an inclusive HIS is urgently needed. Project HOPE in Peru CSXII Final Evaluation 3 B. Assessment of Results and Impact of the Program Results: Summary Chart (Shaded areas indicate the objective was reached) Objectives Baseline Midterm Final 1. Increase from 0% to 20% exclusive breastfeeding for the first 6 months of age of children O% # 65.9% 68.5% 2. Increase from 3 to 5 the average number of meals offered daily to children less than 2 years of age 3 # 4 3.7 3. Decrease prevalence of growth faltering from 55% to 40% 55% 44% 40% 4. Increase number of children with a diet that includes adequate proteins (5/week or 1/day) and calories Not collected at baseline % of children consuming at least 90% of recommended dietary intake: For protein: 67.8% For calories: 39% % of children consuming at least 90% of the recommended dietary intake: For protein: 55.7% For calories: 25.3% 5. Increase dietary intake of micronutrients, including Vit.A, and iron sources to 3 times per week Not collected at baseline % of children consuming at least 90% of recommended dietary intake: For iron: 15.3% For Vit. A: 43.1% % of children consuming at least 90% of the recommended dietary intake: For iron: 5.1% For Vit. A: 26.9% 6. Decrease the prevalence of anemia in children under 3 yrs from 44%; and 3-6 yrs from 52% to 35% 51.2% 54.3% 58% 7. Decrease the prevalence of serum retinol deficiency from 7.6% to 4.0% for <10µg/dl, from 68% to 50% for <20µg/dl <10 µg/dl = 8% <20µg/dl=68% Not collected at Mid term <10 µg/dl = 0.5% <20µg/dl= 12% 8. Increase number of mothers using ORS from 19% to 30% 19% 21% 15% 8a. Increase mothers using some form of ORT 54.2% 63.9% 50% 9. Increase number of mothers who give more liquids during a diarrhea episode from 25% to 60% 25% 37% 38% 10. Increase number of mothers who give equal or more food during a diarrhea episode from 63% to 80% 62% 47% 45% 11. Increase number of mothers who give food more often and in smaller feedings after a diarrhea episode from 34% to 50% * * * 12. Decrease prevalence of diarrhea in children between 6-36 months from 67% to 50% 67% 66% 61% 13. Increase child spacing to 2 years in families with children less than 3 years * * * * Information not available # These values were obtained using a different methodology at baseline, making them not comparable with midterm and final values. Project HOPE in Peru CSXII Final Evaluation 4 2. Results: Technical Approach a. Overview This XII Child Survival (CS) project was funded by USAID from October 1996 to September 2000. The project focus is community organization, training and supervision of CHVs, training and motivation of MINSA field staff, and community education. The project does not supply any direct services. The project also provides technical assistance to the DIRES-SM in planning, evaluation, administrative reform, epidemiology, and professional improvement and in acquisition of basic intervention-specific pharmaceuticals. Four intervention areas were included; Nutrition and Micronutrients, Breastfeeding, Control of Diarrheal Disease (CDD), and Family Planning (FP). The project is located in Región San Martín of central Perú on the eastern slope of the Andes along one of the main tributaries of the Amazon, the Huallaga River. For administrative purposes the Ministry of Health (MINSA) divides Región San Martín into units called Health Networks. The project includes three Health Networks: El Dorado, and Lamas, which include nearly all of the two mountainous provinces of the same name, and the lower part of the San Martin Health Network, which is a very inaccessible region along the lower Huallaga River and its tributaries. These three Health Networks are among the four in the region with the poorest health indicators. The target population of 24,763 women of reproductive age and 13,202 children under five in 190 communities was planned. At the time of the first annual report, the number of communities was reduced from 200 to 162 communities (20 in Lamas, 63 in El Dorado, 79 in San Martin) with the same total target population. Actual figures show 17,460 direct beneficiaries according to project census or 24,284 according to MINSA estimates. Indirect services to a greater number of women and children are provided by MINSA staff to communities outside of the project area. Principal partners in the project are MINSA and the School of Public Health/University Peruana Cayetano Heridia (FASPA/UPCH). Collaboration with two local NGOs CEPCO and PRISMA also serves to increase knowledge of local conditions. The original four-year project has been awarded a three-year extension, to continue working in the same 162 communities. During the extension, major changes will be to introduce malaria and ARI (Acute Respiratory Infections) and to improve collaboration with MINSA, through a transfer of responsibilities, training and supervision on the use of MINSA protocols, and institutional strengthening. A qualitative Final Evaluation (FE) was conducted during August￾September of 2000 with complementary information from a Knowledge, Practices Project HOPE in Peru CSXII Final Evaluation 5 and Coverage (KPC) survey, a biochemical analysis of retinol and hemoglobin, and a food consumption study (included in Attachment F). This report serves to summarize the findings and recommendations of a participatory process involving various stakeholders. A detailed explanation of the methodology used is included in Attachment B. Results from the FE are attached, divided into results from interviews with community members and MINSA staff (Attachment D), and results from team analysis during two workshops (Attachment E). The purpose of the evaluation is dual; to learn from the experience of implementation during the first four years (Lessons learned are printed in Italics) and recommendations for improving strategies during the next three years (Recommendations are printed in Bold). b. Interventions Nutrition 2. Increase from 3 to 5 the average number of meals offered daily to children less than 2 years of age 3 # 4 3.7 3. Decrease prevalence of growth faltering from 55% to 40% 55% 44% 40% 4. Increase number of children with a diet that includes adequate proteins (5/week or 1/day) and calories Not collected at baseline % of children consuming at least 90% of recommended dietary intake: For protein: 67.8% For calories: 39% % of children consuming at least 90% of the recommended dietary intake: For protein: 55.7% For calories: 25.3% 5. Increase dietary intake of micronutrients, including Vit.A, and iron sources to 3 times per week Not collected at baseline % of children consuming at least 90% of recommended dietary intake: For iron: 15.3% For Vit. A: 43.1% % of children consuming at least 90% of the recommended dietary intake: For iron: 5.1% For Vit. A: 26.9% 6. Decrease the prevalence of anemia in children under 3 yrs from 44%; and 3-6 yrs from 52% to 35% 51.2% 54.3% 58% 7. Decrease the prevalence of serum retinol deficiency from 7.6% to 4.0% for <10µg/dl, from 68% to 50% for <20µg/dl <10 µg/dl = 8% <20µg/dl=68% Not collected at Mid term <10 µg/dl = 0.5% <20µg/dl= 12% The Nutrition component increased from 40% of project effort to 45% after the MTE. A major effort of the project has been in nutritional surveillance, the impact is easily seen through the KPC with the percent of children being weighed in the last 4 months increasing from 36.5% at baseline to 94% at final. Sixty￾nine percent of the 162 communities weighed and measured children during the last surveillance in May. Project HOPE in Peru CSXII Final Evaluation 6 The project has shown a rather amazing 15% decrease (from 55% BL to 40% FE) in stunting (Height/Age<-2z) over the four years of the project, but a similar decline has not been seen in wasting (Wt/Ht<-1z) which did not show significant variation from 29%BL to 30%FE. Anemia increased by 7% in children under three. In addition, the project has revealed a significant reduction of serum retinol deficiency from 7.6% to 0.5% for <10µg/dl and from 68% to 12% for <20 µg/dl. These results shown that the project has contributed to the eradication of vitamin A deficiency in the project target areas. Whereas the food consumption study did not show an improvement in dietary intake of Vitamin A, it can be assumed that the retinol increase is due to Vitamin A supplementation. Other results from the food consumption study are difficult to interpret mainly due to myriad factors which effect food consumption, which are outside the control of the project. For example, it was found that during the MTE 49% of the families surveyed received donated food, during the FE this was reduced to 22% of the families. The appropriateness of the nutrition indicators selected for this project need to be further studied by Project HOPE, so that indicators which are more indicative of project results are utilized during the extension period of this project. A report of the KPC, biochemical and food consumption surveys is included in Attachment F. An interesting finding was the presence of salt with adequate levels of iodine in 55% of households at FE, compared to 14% at BL. This can be attributed to improved supervision of the iodizing process by DIRES and the availability of low cost iodized salt at health centers, in part motivated by project findings during the baseline survey. Through the use of a participatory adult education methodology, CHVs have been trained in nutrition concepts and nutritional surveillance. A supervision checklist on quality of anthropometric measures and on-site supervision supported the training in nutritional surveillance. Of the 15 observations made of CHVs weighing and measuring a child during the FE, only one had problems. The training has been very effective and the use of the checklist has reinforced training. The problems identified were with the registration of the numbers (weight, height, age) on a growth card and graph for diagnosis. The FE team did not evaluate counseling skills during the field visits. The system currently being used for diagnosis of malnourished children at the community level is too complex and time consuming for the health promoter and misses the point that weighing and measuring children is not the project goal￾intervening in family level feeding habits is where the project wants to have impact. A great deal of effort has gone into the mechanics of anthropometric Project HOPE in Peru CSXII Final Evaluation 7 measures and little on counseling mothers on what to do when a child is malnourished. The project should select one measure for determining nutritional status and focus training on counseling the family on what concrete actions can be taken. It is strongly suggested that the MINSA child growth card be used for the diagnosis, and that HOPE not introduce new measurement instruments which have been shown in other projects to not be sustainable. A balanced diet cooking contest generated a lot of enthusiasm and support for the use of local foods. During the height of coca production, people were moving towards the use of processed foods, and under-valuing the importance of locally produced foods. Focusing people once again on the value of local products has increased acceptance and impact of the project. A unique aspect of the CS project is the provision of research and technical development of a fortified food to treat anemia. After a number of studies of various options, it was decided to develop the idea of cocoa fortified with heme iron. Cocoa is a locally produced product and heme iron from cow’s blood can be purchased from Chile. The project has worked with a number of local NGOs and the university to develop a formula for the fortified cocoa and is working with local entrepreneurs by providing technical assistance to market the product. The project is encouraging a local NGO, who distributes Title II foods, to purchase the cocoa as an addition to donated foods. A study carried out using the cocoa plus an anti-parasite drug, showed a 50% decline in anemia. A brief explanation of the steps taken in developing this product is included in Section E. A prototype solar dryer, for preserving fruits and vegetables, was developed and presented at a local fair, using informational pamphlets and demonstrations. The project is evaluating the possibility of linking the dryer with another HOPE project in the area-village health banks. The cost of the dryer (over US$ 100 in materials and an additional US$100 in labor) will place it out of the reach of most individual families. The effect of efforts made during this project in the development of the solar/gas food dryer and fortified food products are not yet visible. As part of the final evaluation of the follow-on project, the impact of the food dryer and fortified food should be included. Project HOPE in Peru CSXII Final Evaluation 8 Lessons Learned · The development of activities which respect and value the local culture have the greatest impact · Involving MINSA in training activities aides in reaching the communities with health messages. · Unifying the language used in messages promotes better learning · For an adequate implementation, the project should involve all partners (MINSA HOPE Community) from the beginning · Continuous follow-up and support are needed for the development of new abilities · The supervision checklist on anthropometric measurement was very effective Breastfeeding 1. Increase from 0% to 20% exclusive breastfeeding for the first 6 months of age of children Information not reliable 65.9% 68.5% The project showed an increase in exclusive breastfeeding (EBF) from 0% at baseline to 68.5% at final. The data from the baseline is unreliable because the question on EBF asked recall of the mothers of children 6-36 months old, rather than measuring actual practice in children under 6 months of age. During the midterm the question was re-worded and mothers were interviewed with children 0-6 months of age showing a rate of EBF of 65.9%. The increase in EBF was also reported anecdotally during interviews as part of the FE. The KPC also showed an increase in the percent of women with children 0-23m, who were beginning breastfeeding within the first eight hours after birth, from 65% at BL to 82% at final. A model example of how to change behavior was carried out by the project to modify breastfeeding practices. Students from UPCH studied cultural practices of breastfeeding and introduction of foods using focus groups and developed a social marketing strategy to motivate people to put off the introduction of bananas until 6 months. The project used the study to modify the messages and then one simple message was the focus of education-don’t give ripe banana (maduro) to the child until 6 months of age. Maduro is the traditional first food, and is many times introduced during the first month. This simple message has been extremely effective, everyone gave the same clear message, through various sources and the effect has been extremely positive. One of the principal strategies for encouraging breastfeeding was through the formation of breastfeeding support groups (GALME). GALMEs are reportedly present in 94% of the communities, although according to the groups of women interviewed during the FE, half of the GALMEs are not meeting (8/16). Project HOPE in Peru CSXII Final Evaluation 9 The GALMEs do not follow the model of what a support group is but have developed into more of a mothers group. A support group would work well in a community that already has a women’s group and the GALME would be a sub￾group, but when no mothers group exists, the GALME is seen as only focusing on a few women, and excluding the others. The GALME is used by MINSA and the CHV as a way of channeling information to the communities. The GALMEs look at other issues, not just BF or health and want to be involved in a number of activities, sports, sewing classes, gardens, etc. The concept of a support group is not an indigenous idea and was perhaps introduced prematurely by the project. Women are hungry for their own group, which they control. Almost all CHVs are men, and they would be in charge of any group they were involved in. By having women chose a female coordinator, the GALME had the unexpected result of providing women with a safe environment for developing leadership skills and self-esteem. All mothers groups interviewed during the FE reported that their husbands were supportive of their attending GALME. The project needs to focus efforts on organizing women in a group which they are comfortable with, not excluding older women, midwifes, etc. or having “rules” as to how the group should be formed. The suggestion was even made during the FE to include men in the group. The concept of support groups is not really understood in the communities, and perhaps in the future when women are more accustomed to leading a group and sharing ideas (instead of “receiving” talks), it would be more appropriate to introduce support groups. One of the ideas discussed informally during the FE was to link new mothers or women with BF problems to a “yunta” or partner on a one-to-one basis for support and guidance. This strategy might be better understood and accepted than the support group. Other activities to support EBF were the participation with MINSA during National Breastfeeding Week which included community marches to increase awareness of the importance of EBF. The formation of a network of GALMEs was planned but was not possible during this period. An informal network around sporting events has grown up to link a few groups. Several alternatives were brought up during the FE to support the exchange of ideas between GALMEs. The formation of a link between GALMEs should be established, using one (or more) of the following mechanisms: 1). Formation of a network of GALMEs 2). Integration of the Coordinators of GALMEs into the CHV network Project HOPE in Peru CSXII Final Evaluation 10 3). Visits between groups for sports events and to exchange ideas about health issues 4). Annual conference of all coordinators to reinforce their work and to define future directions Lessons Learned: · The GALME should not solely be a receptor of messages, but an opportunity to develop leadership skills. · The GALME should be for all women, include other topics of interest to the women and include activities with their husbands. · The greatest impact in changing practices comes with clear, simple messages, based on previous study · Take advantage of the felt need of women to meet in their own group, according to their own vision Project HOPE in Peru CSXII Final Evaluation 11 Control of Diarrheal Diseases (CDD) 8. Increase number of mothers using ORS from 19% to 30% 19% 21% 15% 8a. Increase mothers using some form of ORT 54.2% 63.9% 50% 9. Increase number of mothers who give more liquids during a diarrhea episode from 25% to 60% 25% 37% 38% 10. Increase number of mothers who give equal or more food during a diarrhea episode from 63% to 80% 62% 47% 45% 11. Increase number of mothers who give food more often and in smaller feedings after a diarrhea episode from 34% to 50% ** ** ** 12. Decrease prevalence of diarrhea in children between 6-36 months from 67% to 50% 67% 66% 61% The project objective of increasing the percent of mothers using Oral Rehydration Solution (ORS) during an episode of diarrhea from 19% to 30% was not met. Final KPC results show a decreased use of ORS to 15%. The focus of this objective was modified during implementation, to bring it more in line with MINSA policy. ORS is recommended to be used only in cases of dehydration, and home-based fluids are recommended for uncomplicated cases. The indicator was also measure as ORT (Oral Rehydration Therapy), which includes all fluids, in addition to ORS. The use of ORT was also unchanged according to results of the KPC. Baseline results showed a 54% use rate and FE showed 50%. There was an increase in mother giving more liquids (from 25% to 38%) during diarrhea, but a decrease from 62% to 45% of mothers giving the same or more foods. Neither of the indicators reached the stated goal of 60% and 80% respectively. Project staff felt that not enough emphasis had been given to the home based nutritional management of disease conditions. During the extension project a more integrated approach using IMCI principles will be implemented. Elaborate and broadcast basic integrated messages aimed at improving knowledge and practices of mothers with ill children (Diarrhea, respiratory infections, malaria) with a focus on feeding during and after the illness. Mothers most commonly identified weakness/no energy as a sign of when to seek medical attention in cases of diarrhea, instead of a more specific sign such as dry mucous membrane, decreased urine/tears, etc. More focus is needed on recognition of signs of danger for both mothers and CHVs. An important point related to CDD that was not addressed during this CS project is the high use of anti-diarrhetics and antibiotics when child has diarrhea. According to the baseline KPC 55% of mothers thought that in cases of diarrhea the child should be given anti-diarrheal drugs or antibiotics. This element was Project HOPE in Peru CSXII Final Evaluation 12 not reported on in the Midterm and Final KPC reports. The widespread use of these medications was also reported during the FE. During the next phase of the CS project, one focus of education should be aimed at decreasing the use of inappropriate medications for diarrhea. A small decrease in the prevalence of diarrhea was seen from 67% to 61% in children 6-36 months of age, according to results of the KPC. Decrease in cases of diarrhea may in part be attributed to the distribution of anti-parasite drugs and through improved hygiene practices. A greater focus on sanitation is needed to have an impact on the prevalence of diarrhea. The technical needs for jungle areas are more complicated for latrines and water and require specialized technical assistance. The ability of the department of sanitation of MINSA to effectively work with communities is limited by a minimal budget and personnel. They do provide an important function of motivating and supervising the use of facilities. The project needs to help communities to develop proposals to present to municipal authorities and other organizations for sanitation projects. Alliances should be formed with organizations who have financing and/or offer technical assistance in construction or latrines, water systems, etc. Community oral rehydration units (UROCs) are present and functioning in 131 communities. The UROCs need to be modified to provide a more integrated preventative health approach. The centers-possibly to be renamed “House of Health” or “House of the Promoter”-would take on the role of integrating care within the community-namely CDD, ARI and malaria. In many communities the UROC already houses a community pharmacy and sometimes a family planning center. Strengthen the UROC as a community structure with an integrated preventative health role in the community through increased community participation involving the CHV, LHC, leaders, GALME group, and health staff. Lessons Learned: · To decrease the prevalence of diarrheal disease it is necessary to improve sanitation conditions through coordination with other organizations that work in this area. · Educational messages which focus on improving feeding during and after a disease episode will have the greatest impact on changing knowledge and practices Project HOPE in Peru CSXII Final Evaluation 13 Family Planning 13. Increase child spacing to 2 years in families with children less than 3 years * * * * Information not available MINSA has a major family planning promotional effort in the region, resulting in a fairly high acceptance of modern methods. This project supports MINSA's educational efforts and assists with stabilizing the supply of methods. The scope of this intervention is limited, it accounts for only 5% of the project’s effort. The information on child spacing is not yet available, but the percent of mothers who desire no more children in the next two years or are not sure, who are using a modern contraceptive method increased from 62% at BL to 77% at final. While this change can not be attributed solely to the project, training efforts to 188 CHVs and participation by CHVs in family planning campaigns of MINSA have helped to encourage the use of modern FP methods. It was found in the BL KPC that the contraceptive pill used to be most commonly used method, but now the most common used method is anovulatory injections. The injections have long been the preferred method, but there were problems in the past with supplying the method. Those problems have been resolved and adequate supply has been greatly improved. Lactational Amenorrhea is not specifically encouraged as a FP method but is mentioned as one of the benefits of breastfeeding. According to women’s groups interviewed during the FE, one way to improve the use of FP methods is to provide information to the couple and to improve communication with the husband. The new approach for the next 3 years will be to involve men in child spacing discussions and decision making, as a partner with women. Implement family planning activities by involving men and husbands in educational activities. c. New tools or approaches A principal partner of the project is the School of Public Health/UPCH. This relationship has opened a unique opportunity for operations research, with positive results for the project in obtaining new information to help guide program decisions, and for the university by providing a opportunity for students to complete field studies in a needed area. The project has been involved in the decision of what studies were to be conducted, and the locale for studies. A total of 28 students participated in project field activities: 20 from UPCH (school of medicine, nursing, and public health), 5 from UNIFE (school of nutrition), 2 from Universidad Antenor Orrego (school of communications), and 1 from Pontificia Universidad Católica del Perú (anthropology). Project HOPE in Peru CSXII Final Evaluation 14 Project staff felt the relationship had been positive with UPCH and other universities that cooperated with the project and encouraged continuing the relationship, and including nursing students in the future. The university supported the staff in their own research, helping to develop protocols and publishing their results. A number of studies were conducted, many focusing on the use of fortified food to treat anemia, cultural patterns in geophagia as a source of iron, and breastfeeding. One of the more interesting studies on breastfeeding beliefs and practices was previously discussed in the Breastfeeding section. The project activity on food fortification was previously discussed in the Nutrition section. In addition to research activities, the project expanded the traditional use of the KPC survey to include a food consumption study and biochemical indicators (hemoglobin, parasites, iodine content of salt, and retinol). These studies were utilized as part of the baseline, midterm and final evaluations and have added to the national and international body of knowledge. Results from these surveys have been instrumental in working with MINSA to prioritize action, i.e. study of the actual iodine content of iodized salt has lead to actions to monitor the iodizing process. 3. Results: Cross-cutting Approaches a. Community Mobilization Community organization continues to be a challenge for the project. According to the DIP, communities will be given assistance in learning to make decisions based on information, prioritize issues, conduct democratic meetings, access other sources of technical assistance and funding, motivate citizen involvement, and to evaluate efforts. The project reports committees in 96 communities. According to committee members interviewed during the FE, 6 of the 8 committees meet monthly, but only 1 of the 8 had developed a work plan. Communities need to be involved more in planning and evaluating, and in the use of information for decision making. A great deal of additional work needs to be done to strengthen community structures, particularly the Local Health Committee (LHC) and community leaders. During the FE authorities were open about their lack of knowledge about health problems and project activities and eager to receive training. Training for committee members and leaders in project development and community organization should be prioritized during the next 3 years. Project HOPE in Peru CSXII Final Evaluation 15 A model structure presented in the DIP was to include the coordinator of GALME and the CHV as members/technical assistants to the health committee. This has not been accomplished, but will continue to be a strategy during the next three years. Women’s participation in the committees is negligible. According to interviews during the FE, only 2/16 committees have participation of a woman. Women traditionally do not attend community meetings. The formation of a community support system where CHV, GALME coordinator, committee and leaders all work together to provide mutual support is imperative. This support system should be based first on improved communication to insure that everyone has a clear understanding what the others are doing. According to interviews during the FE, 11 of the 16 GALME coordinators, 3 out of 8 of committees, and 10 out of 12 leaders did not know how many malnourished children there were in the community. Real change within a community will come when everyone moves together with the same messages. Sustainability must come from within the community, through encouragement and prioritization of health as an action issue. A network of CHVs has been organized in the district of San Jose Sisa, El Dorado Health Network. Twenty-one CHVs organized in July of 1999 and wrote bylaws for the organization. The bylaws were being used in another Health Network to replicate the activity. A network of CHVs already exists in San Martin, with the help of a local Catholic priest. A network of GALMEs was also planned for the first four years, but was not accomplished. Suggestions for continuing with this activity during the next period were already discussed in the Breastfeeding section. Good relations have been developed by the project with municipal authorities, who have been sharing costs for CHV training, based on their fiscal possibilities. Agreements have been signed with municipal authorities for specific short term needs. The municipal authorities are excellent contacts for the communities for the funding of future small projects, such as latrines. Another challenge for the project in the next 3 years is how to involve more community members in the educational process, so an environment of change can be established, rather than just some women trying to change without community support. During interviews as part of the FE, most groups mentioned the importance of involving men as well as other community members such as midwives and older women. Additional work needs to be completed in the next 3 years to consolidate efforts made in community organization. During discussions with community leaders, committee members, volunteers and women, there seems to be a strong positive feeling about the project, and a desire to continue working together in the Project HOPE in Peru CSXII Final Evaluation 16 future. Listening more to communities to encourage their involvement is needed to cement relations and move in directions defined by mutual agreement. Community diagnosis should not just be seen as a one time event; evaluation, analysis and planning need to be ongoing. Community diagnosis and ongoing meetings to evaluate, analyze and plan should be held together; community (including women), health personnel and HOPE. Lessons Learned: · A project can not advance without the support of local authorities. · Communities must be involved as full partners in the development process, including being involved more in planning and evaluating b. Communication for Behavior Change Educational modules in community participation/leadership, nutrition and micro￾nutrients, nutritional surveillance, CDD, breastfeeding and adult education have been developed by HOPE staff. The modules are of very high quality and include a number of inter-active participatory activities which enhance learning. Good evaluation tools to insure understanding of the concepts have also been developed. The project has an excellent focus on message delivery for CHVs and MINSA. The courses have been well received, and positive progress has been made in providing information at the community level but the process needs to be taken to the next step. CHVs and GALME coordinators need improved skills in more than message delivery but also in counseling, identified barriers to change and negotiating behavior change. During the FE, in questions asked to the CHVs, the main messages given to mother of malnourished children were to increase the frequency of feeding, give thick foods instead of thin soups, and give a balanced diet, but the addition of oil to the child’s food was not mentioned. This is just an example of why basic messages need to be strengthened to enhance the impact of actions on the part of the mother. Teaching a concept to CHVs does not mean it will be correctly transmitted to mothers. MINSA and HOPE should strengthen follow-up to insure the quality of replication of training courses and the education in the communities. A checklist should be developed for insuring quality of educational activities of CHVs, GALMEs and MINSA staff. Educational materials are an effective strategy for 1.) enhancing learning by community members, and 2.) motivating CHVs and MINSA staff. The current project had a very limited budget for materials, and this was recognized as a limiting factor in implementing the project (Group work during the Analysis Workshop identified the lack of materials as a limiting factor and even the mothers groups interviewed during the FE said educational sessions would be better with materials). The follow-on funding also has an extremely limited budget for educational materials. HOPE has received a grant to develop materials Project HOPE in Peru CSXII Final Evaluation 17 with BASICS and CHANGE to help women recognize danger signs and risk factors which require medical care. In addition to this, HOPE should determine a mechanism for acquiring educational materials for all communities and MINSA health posts to support educational messages. This funding should come from a modification of the budget of the 3-yr follow-on project, or through complementary funding. c. Capacity Building Approach (i) Strengthening the PVO Organization The CS grant has improved the institutional capacity of Project HOPE through a number of mechanisms; · Through the design and evaluation process of the CS grant, HOPE has increased their expertise in project development and monitoring, and the identification of high quality technical assistance to strengthen both technical and managerial aspects. · Project HOPE has been able to strengthen their management of human resources; by clearly defining roles and responsibilities of local staff, development of concrete job descriptions, and the elaboration of a personnel evaluation protocol. · The monitoring of financial aspects has assisted HOPE to identify weaknesses in their system, and motivated the development of a complementary tool for financial control, which has been used in other HOPE programs. Project HOPE is continuing to improve the financial system to better serve the needs of CS and other programs. · Project HOPE has improved inter-and intra-agency communication through the development of a Internet web site used to disseminate lessons learned from Child Survival projects within the People-to-People Health Foundation and to other organizations and the general public. (ii) Strengthening Local Partner Organizations An agreement was signed between the CS project and DIRES-San Martin in October of 1996. There were a number of problems during the first two years of this grant period when HOPE was working very independently from MINSA. The MTE was a turning point in evaluating project strategies and coordination has markedly improved in the last two years, but is still not 100%. The first step in improving work with MINSA is to identify together what areas need to be prioritized. An institutional diagnosis of MINSA should be carried out as part of the development of the DIP for the second phase to identify individual training needs and points for institutional strengthening. The diagnosis could be repeated at the end of 3 years to measure progress. HOPE should seek technical assistance on institutional assessments from other NGOs who have experience in this area. Project HOPE in Peru CSXII Final Evaluation 18 Capacity building in the future should focus on strengthening systems, for example; referral system, supervision system, a simple system at the health post level to track who the CHVs are and what training they have received to enhance supervision and planning for training. MINSA sees the educational methodology developed by HOPE as one of the most valuable contributions. MINSA has applied the training they received from HOPE to their own work by replicating courses for their staff in other Health Networks where the project does not work, thus increasing the impact of the project in other geographical areas. The Basic Language of Nutrition was a particularly successful course which was originally given to nurses, but later physicians were so impressed with the course, that an additional course was provided for them. Other MINSA projects such as the cervical cancer awareness project and ELITES are also using the methodologies. There are good personal relations and good coordination with the MINSA staff, particularly at the DIRES level. MINSA has seconded one nurse full time to the project, and HOPE pays an additional stipend to equalize her salary with other HOPE staff. Many former HOPE staff now work with MINSA and many former MINSA staff now work with HOPE. The greatest change seen in the working relation with MINSA is greater enthusiasm for the project and a better understanding of the role of HOPE. Continued effort is needed to achieve a good understanding at all levels and a sense of ownership on the part of MINSA of the CS project. Supervision of activities at the community level by MINSA staff has improved. The supervision checklist for anthropometric measurement has been very effective in clearly defining expectations of what is quality. The anthropometric checklist should be expanded to include diagnosis of malnutrition and counseling of the mother on what actions she can take. The checklist concept should be expanded to include other areas for supervision. Meetings are frequently held at the DIRES level, but communication is still a problem at the other levels. The project can not just assume that information will filter through channels, contact is required at all levels. Both a formal and informal system of communication should be developed between HOPE and MINSA: · HOPE should participate in the planning and evaluation meetings of the AIN 2001(Integrated Care of the Child) program. · Regular meetings with the DIRES, Health Networks, health center and health post levels Project HOPE in Peru CSXII Final Evaluation 19 · An annual planning and evaluation workshop should be implemented with participation from all levels, to review project achievements and plan together for needed readjustments. Lessons Learned · Don’t divide the work with MINSA, go with them to support the implementation and quality of the work. · The use of checklists clarifies expectations and defines what elements are essential · CHVs should not have an institutional affiliation, but should clearly be of the community. In the FE when mothers were asked whom the CHV worked for, all groups said for the benefit of mothers and children in the community. (iii) Health Facilities Strengthening The project works with 52 MINSA staff at the health center and post level. Most health posts are staffed with only one technician, many of whom have received very little training. San Martin is one of the most isolated areas of the country and work conditions are poor. Assignment to the area is usually perceived as a punishment so most people see their work in San Martin as short term, leading to a high rate of turnover. This is somewhat contradicted by findings from the FE of 5 of the 9 staff having more than 4 years in their position. There is a system of supervision for MINSA staff, with guidelines based on each program. Each health facility has a supervision notebook where observations and recommendations of the supervisor are noted. The development of an integrated supervision system could potentially be an area where HOPE’s intervention could strengthen MINSA’s institutional structure. MINSA staff does not prioritize CS project activities because they have other assigned priorities, a heavy workload, and an unrealistic burden of reporting information. Part of the institutional assessment previously recommended should include an assessment of health facilities. National efforts to develop Integrated Management of Childhood Illness (IMCI) programs are being developed in Peru. As a complement to this effort the DIRES of San Martin developed a comprehensive package of AIN (Integrated Attention for the Child) in 1999. The CS project did not directly include the strengthening of these programs within their objectives, but this will be included in the follow-on project. The HOPE staff is collaborating with the ELITES-mobile clinics for disperse populations (formerly SAIZADIP). ELITES have worked in about 20-25% of the same communities as HOPE in Dorado and San Martin for the past year. They Project HOPE in Peru CSXII Final Evaluation 20 work in each community 1 day every 2 months providing medical care, medicines, and community organization. The ELITES staff has been trained in CS project educational methodology and some are former HOPE employees. So far they have coordinated well with the project and offer an incredible service to isolated communities. The sustainability of this group is questionable as it is just being piloted, but HOPE should continue, and expand, the collaboration with this group. (iv) Strengthening Health Worker Performance A good first step in working with community volunteers was to identify existing CHVs and begin working with them. The original goal of the project was to train 400 CHVs (200 couples) in approximately 190 communities. The current database of the project contains 145 CHVs, 32 of which are recognized as inactive, but project staff reports 230 active CHVs. The strategy of working with couples has not been implemented, mainly due to constraints on women to travel to centralized training courses. New strategies need to be implemented to involve women as CHVs- the coordinators of GALMEs is an excellent way to begin as well as selecting in the future the wives of community authorities and current CHVs to receive training. More flexibility in the training plan is required to get these women trained. The roles of the women can be different compared to the men, they don’t both have to do the same thing. Women should be able to receive training in their own communities. The strategy of using CHV couples should be re-evaluated during the second phase of the project. It is difficult to determine exactly how many CHVs have been trained during the life of the project due to a deficient Health Information System (HIS). It was reported in the first annual report that 135 CHVs were trained during the first year, including 118 trained in CDD, FP and nutrition. As these CHVs are not included in the current database (of 145 CHVs), it is unclear whether an additional 145 CHVs have been trained, or if some received various courses. In the DIP, the planned ratio of CHV to families was 1 CHV couple per 16 families, or 1 person per 8 families. This ratio is very unrealistic and the actual ration is about 1 CHV per 50 families. The planned ratio of 1 MINSA staff per 5 couples (or 10 people) was more realistic. Staff estimates that each health post (1 MINSA staff person) has 10-15 CHVs. Of the 23 CHVs interviewed during the FE, 14 have worked as a CHV for three years or more, and of the GALME coordinators 8 out of 16 have worked as coordinator for 2 years or more. The actual turnover rate of CHVs is not known, but the project needs to extend its focus on how to maintain CHVs as active agents within the communities. This needs to be done through improved communication with the CHVs and CHV associations, which have recently been formed. Results of the FE interviews with health personnel showed that the two Project HOPE in Peru CSXII Final Evaluation 21 best ways to motivate the CHVs were to provide incentives and to meet monthly with them. Ideas for incentives presented during the FE included: identification cards from MINSA, free (or reduced cost) medical care, caps, educational materials, and tee-shirts. Knowledge and performance of the CHV is measured through: · Pre- and post-tests at training course · Nutritional surveillance checklists · Completion of nutritional surveillance forms for the database These measures also need to be complemented with a more comprehensive supervision and monitoring system. During interviews with health personnel, a frequent response as to how to support the CHVs and GALME coordinators to improve their work was through frequent follow-up and supervision. Supervision of the CHVs is an area of great weakness in the project, it is not being done on a regular basis and no supervision format is being used, except for the checklist for anthropometric measurement. Project staff use the results of pre/post–tests, non-assistance at training, anthropometric checklist, and not completing nutritional surveillance as ways to prioritize visits to CHVs for follow-up. Given MINSA’s limited budget and mobility, the next phase of the project should be used to identify sustainable ways in which the community volunteers can be supported by MINSA. Establish a system of indirect supervision using regular monthly meetings between health post staff and all CHVs from the post. The meetings would be used for analysis of information, to reinforce training topics, distribution of materials (ORS), discussion of problems and solutions, and to prioritize needs for direct supervision visits. HOPE and MINSA should collaborate on the development of an integrated supervision instrument. Lessons Learned · Initially identify CHVs who have been trained by other organizations · Regular contact is essential to support the work of the volunteer (v) Training The training for the CHVs has been very effective, the principal problems related to training are; · Not all CHVs were trained in all topics · The HIS is inadequate to track who has received what training (during the first 2 years) · Other people who were planned to be trained, did not receive training (Coordinators, teachers, midwives) · More flexibility is needed in training strategies to include women as CHVs Project HOPE in Peru CSXII Final Evaluation 22 The principal training topics for CHVs were Community Organization/Leadership, Nutrition/Micronutrients, Nutritional Surveillance, FP, Breastfeeding, Education, CDD and First Aid and all topics originally planned were carried out. Not all CHVs however have received all courses. Some of the GALMEs coordinators have been trained, but not many (10 of the 16 coordinators interviewed during the FE had not received training). No training has been given for teachers, midwives, or committee members, as was planned in the DIP The project focuses on message delivery, and this has been effective, but more involvement of other decision-makers is needed; men, older women, midwives, and teachers. The strategy of home visits as an educational tool has been underutilized. Use of radio was mentioned in the DIP, but has not been implemented. The project should take more of a gender focus to provide an environment of equity for women to participate in activities. The San Martin area is known as a very male dominated culture and project strategies need to be modified to give women a chance to participate, given the constraints of the culture. One of the principal barriers is travel and staying over night at training courses. Decentralized training at the community level, induction of the wives of current CHVs and community leaders, graduation of Coordinator to become CHVs, are all ways in which more women could be involved. The main reported change in behavior was regarding EBF and waiting to introduce bananas. Fifteen of the 16 coordinators interviewed felt more women where breastfeeding exclusively and community leaders felt that there was less diarrhea and less malnutrition. Male CHVs were not trained in BF because the project thought they would not be the principal source of information concerning this topic, but the CHVs really need to understand these issues so they can influence other men and support women’s actions. There has been a good system used for evaluating the level of understanding of CHVs after receiving training. A system of pre-and post-tests was developed and have been consistently used to improve training and prioritize supervision. Lessons Learned · More emphasis needs to be given on training to train and supervising the replication of training. · Other members of the community need to be trained to create an environment of change · Sustainable training structures must be developed Project HOPE in Peru CSXII Final Evaluation 23 d. Sustainability Strategy Sustainability Goals andObjectives Objectives Goals * Status at Final Evaluation 1. Promoters working for their communities - 400 trained promoters responding to the community - To be recognized by the community Database show 145 CHVs trained (but 32 of these inactive) Other information estimates 230 CHVs, HIS inadequate for tracking CHVs 2. Sustainable community committees conducting health-related activities - 190 communities organized - Regular meetings - Implementation of projects The project reports active committees in 96 communities. 3. MINSA staff involved in community work - Staff from 85 health sites involved in community work - Participation of DIRES-SM staff in the follow-up and strengthening of community work HOPE estimates that 52 MINSA staff in health sites are working with the project 4. Fortified food on sale - Product demand Not on sale, but product developed and marketing just beginning 5. UROCs functioning - 190 UROCs functioning UROCs are present and functioning in 131 communities 6. Support groups or BF counselors - 200 support groups functioning - Establishment of an association in each Health Network Approximately 152 groups are formed, but not all active, most coordinators have received training. No associations formed. 7. Continuing growth monitoring - Growth monitoring of children under 5 in 190 communities - 400 promoters trained in growth monitoring Growth monitoring carried out in 70% of communities Project records show 322 CHVs trained in Growth Monitoring No separate GMVs trained 8. Teachers teaching basic messages - 100 school teachers providing basic messages This project did not work with teachers during the 4 years 9. Adequate sanitary practices in community - 50% of communities with adequate sanitary practices Information not available 10. Participation of birth attendants in the promotion of BF/ FP - 80% of birth attendants encouraging exclusive BF and family planning practices This project did not work with birth attendants during the 4 years 11. Improved dietary practices - 30% of mothers adopt appropriate food practices - Staff from 85 health sites-(MINSA) and 400 trained promoters on adequate dietary practices and adult education - 25% of the communities looking for technical support to improve agriculture KPC showed increase knowledge and practices in some areas. MINSA staff from 52 sites trained, 147 CHVs trained in adult education 308 CHVs trained in nutrition Technical support not being monitored * Goals were modified in the first annual report when total communities decreased from 200 to 162 Further work is required in all areas to meet the sustainability goals of the project. As the role of HOPE changes during the follow-on project, a focus on sustainability must be prioritized. No cost-recovery was planned during this project. Project HOPE in Peru CSXII Final Evaluation 24 Planning for Sustainability During the last year of this project HOPE worked closely with MINSA in a phase-over process which included IVSA: I= Identification of one person who is responsible for the transfer process V= Visits together to the communities S= Supervision in an integrated fashion, design of a supervision instrument and training of MINSA staff in supervision skills A= Articulation of activities involving the HIS Directors of the Health Networks are very involved in the phase-over process. In the future DIRES plans to have one or two specific people assigned to work closely with HOPE. Continuous planning with MINSA is needed to strengthen the structure of the institution and to plan for those elements which can realistically be sustained. Linkage between the communities and governmental/non-governmental organizations There has been good involvement on the part of MINSA and the municipalities in supporting CHV training, both organizations provide some part of the cost of training courses, lodging, transport, food, etc. Both MINSA and municipal authorities have made good efforts to recognize the importance of the CHV and the value of the work they are doing. MINSA sponsored a Day of the Promoter, which was celebrated, with some of the best CHVs being selected to attend a conference. During the first conference, each NGO sent 5 CHVs, for a total of 30 participants. The link between communities and MINSA and municipalities has been strengthened during this project, but further work is required. Increased coordination with other organizations is needed to provide a linkage between organizations (NGOs) and the community. HOPE and MINSA staff needs to gather information regarding the requirements of other organization for presenting projects. This information could be shared with community leaders to strengthen the link between the community and other organizations. Increase the coordination between communities and municipalities and other organizations. Strong community structures with a clear vision of development The foundation has been laid for strengthening community structures, but this needs to be an area of more effort during the second phase of the project. MINSA-HOPE needs to work more with the LHC and leaders so that they assume greater responsibility for the health of the community, and develop an amplified vision of development. Sustainable Training Structure In order for training to be sustainable a system needs to be established which makes training ongoing, taking into account staff turnover within MINSA. During the FE, the evaluation team formulated the following recommendations for improving the sustainability of training: Project HOPE in Peru CSXII Final Evaluation 25 · Create Trainers within MINSA in health, gender and educational methodologies. · Create Trainers within the municipalities and teachers in community organization and project development, leadership and participative methodologies. · Encourage communities to identify alternative resources for training and development activities. · Advocate for the commitment of CHVs who retire from service to train a replacement · Investigate, with the CHV associations, what role they can play in training new CHVs Because of the high turnover within MINSA, two strategies need to be emphasized, 1.) Train trainers at the DIRES level, where staff is relatively stable, who can replicate educational methodology, 2.) Develop a self-guided manual for medical students who work 1 year in the rural area and new MINSA staff on community participation Community demand/responsibility for health services The demand for health services has increased dramatically in response to new national policies which provide free health care for women and children. Child insurance began in 1997 and saw demand far outstrip supply. Maternal insurance is still in the pilot stage, but should be implemented nation-wide this year. These two systems of basic health insurance will change the pattern of use of health care facilities. Another change in government policy will change the community’s ability to influence how services are delivered. CLAS-Local Health Administration Committees are recently being established in the San Martin area at the Health Network level. The committee is composed of MINSA, community members, education sector, and municipal authorities to improve the administration of health services and health facilities. HOPE needs close communication with MINSA to maintain flexibility in how to best work within these new systems. C. Program Management 1. Planning The current DIP was overly optimistic and objectives were formulated which did not take into account factors which are outside the scope of the project, i.e. prevalence of anemia and diarrhea and food consumption. The DIP needs to include more clearly defined objectives and more realistic goals. The DIP was well translated into Spanish, and was used as a guide for project implementation. The selection of the area for project implementation was based on a high-risk focus and even though it is an extremely difficult area to work in the project has shown substantial impact. According to the director of DIRES; HOPE has broken Project HOPE in Peru CSXII Final Evaluation 26 out of the traditional NGO role of choosing more accessible communities and serves as a model for other NGOs. MINSA was involved in the selection of geographic location and interventions. With improved communication via email, HOPE headquarters hopes to be able to involve field staff more in the development of proposals and the DIP. The DIP for the 3 year extension should be developed in a participatory manner, involving community, and MINSA at all levels. 2. Staff Training The Project HOPE staff has received excellent training throughout the four years of the project, unfortunately MINSA staff has not enjoyed equal opportunities. Some of the most notable training was in adult participatory education, both from a local NGO, Kallpa, and an international consultant. Other important skills gained were in negotiation and working with groups. HOPE has had the opportunity to acquire the skills for both technical and professional development. The image of HOPE at the beginning was “the little girls of HOPE”; they lacked credibility due in part to their youth and inexperience. Through outside training, and the formation of a strong supportive team, the HOPE staff has been able to turn around the previous perception and command respect, particularly in educational methodologies. The project was able to identify appropriate training for the staff and sufficient funds were available for training. Project staff has started to participate in national and international training sessions on community-based IMCI. PAHO/Peru invited HOPE Peru staff to participate in the national committee for IMCI implementation. Additional training is needed in IMCI and supervision and monitoring. 3. Supervision of Program Staff A formal system for the supervision of program staff does not exist. There is a supervision format, but it is not adequate for the needs of staff, is not being implemented and should be modified. Staff receives supervision visits 2 – 3 times a year. Through good working relations, the staff responses well to indirect supervision with monthly meetings and frequent contacts. As supervision is one of the aspects that HOPE would like to strengthen within MINSA, a good place to start would be with the development of a model system within HOPE. At all levels the supervision needs to be improved. Project HOPE in Peru CSXII Final Evaluation 27 4. Human Resources and Staff Management The staff, at the time of the FE, was all women, with the exception of the driver and part-time computer person. This situation changes the dynamic of staff cohesiveness, as there is a strong feeling of accomplishment and solidarity. Despite the high morale, staff turnover in the project has been high, of the original 10 positions, only 3 staff are still with the project after four years. There are several reason for this high turnover; the work conditions of the project are arduous and require a high level of adaptability, particularly in the long term; the educational level of the staff is extremely high, for a Child Survival project; most of the staff are not local, but are from Lima. The high level of qualified staff has allowed the project to strengthen the relationship with MINSA by suggesting new strategies and improve the integration of activities. This strategy will continue during the follow-on phase. A key staff, Dr. Luis Benavente, Country Director, was hired by HOPE headquarters and now offers technical assistance to the project in his new role as Associate Director of Maternal and Child Health Programs. The project has a Human Resources Manual, which forms part of the organization’s Procedures Manual. Personnel policies are clearly spelled out and there were no reported problems. All staff is evaluated every 6 months, using a form based on a model distributed by HOPE headquarters. The evaluation is both a self-evaluation and a team evaluation based on established individual goals. Job descriptions have been developed for all positions. The majority of the staff from the CSXII project will be carried over to the follow-on project. Some staff will not be hired for the 3-year extension and good effort has been made by the project to help them find new positions, some have been hired by MINSA. Due to the extensive training that the staff received throughout the project, their skills are in high demand. 5. Financial Management A new financial system was introduced by HOPE headquarters in July 1999 using Quick Books, but is not functioning 100% and communication is slow between headquarters and the local office. On the first of September, the latest budget report, which the local office had received, was for June 2000. To overcome this lack of timely information, a spreadsheet was developed to complement the system and provide the information needed for managing the project locally. HOPE is seeking technical assistance to improve the financial system at their headquarters. Project HOPE in Peru CSXII Final Evaluation 28 A brief financial management manual is part of a HOPE Procedure Manual, which also includes human resources management. The administrator changed in October of 1998. The previous administrator received hands-on training from HOPE Guatemala staff. The new administrator received limited information from the previous administrator at turnover but good support from headquarters. Project HOPE has provided good technical support to the current administrator to overcome problems with the administrative system. A good system of communication exists between the local office and HOPE headquarters via email. The budget was adjusted in 1998 to allow more local staff to be hired. Significant savings during the first two years of project implementation allowed for an increase of local staff, which was approved by USAID/BHR/PVC. Other minimal internal adjustments were made between line items, which did not necessitate USAID approval. The current budget was poorly classified, due to a lack of firm cost figures at the time of proposal development. Local staff was more involved in developing the budget for the extension, and the project has better experience at calculating costs, consequently the extension budget should better support the implementation of the next project. 6. Logistics No major issues were identified during the Final Evaluation relating to logistics. While transportation and communication are extremely limited due to the isolated nature of communities, the project has successfully worked within the constraints of their environment. In some cases, project staff had to travel by car for two hours, take a boat ride for another 3 or 4 hours, and walk up to 9 hours in some instances to reach project target communities. This scenario has been one of the most difficult ones that HOPE has ever faced in a CS project. Yet, logistics barriers were overcome as part of a team effort at HOPE Peru with the continued support from HOPE headquarters. 7. Information Management Most information systems consist of three components: impact surveys, process monitoring and qualitative guidance. The project has a system of KPC, food consumption and biochemical surveys to measure progress toward impact indicators. Most principle objectives were measure three times (baseline, MTE, FE) and summarized in the table on page 3. The project also uses qualitative methodologies to adjust project implementation. The measurement of ongoing activities and process indicators is deficient. Project HOPE in Peru CSXII Final Evaluation 29 Impact surveys Due to changes made in the KPC format, HOPE loses comparability with other Child Survival projects. For example, HOPE’s survey asks at what age a child began receiving (various foods and liquids) the model KPC asks if the child is actually receiving (various foods and liquids). Although the questions are similar, they are not totally comparable. Unless there is compelling evidence that the manner of asking the questions which HOPE employs is superior, it is suggested that the model KPC questions be used. One of the strengths of the KPC developed by CSSP is the ability to compare indicators across various CS projects. Another example of how comparability could be enhanced is with the indicators for the extension project. The indicator for child spacing is: % of mothers who do not want children in the next two years and use modern FP methods. By simply modifying the concept, it could be comparable to the USAID Key indicator: % of mothers who desire no more children in the next two years or are not sure who are using a modern FP method. The project should try and use generic indicators whenever possible. Process Monitoring Ongoing activity reports have not been developed. A monthly format exists for use by the CHV, but its use is limited. Hope staff submits a monthly narrative report by area and each HOPE staff member maintains a notebook to keep track of activities. One of the recommendations from the Analysis Workshop of the FE was to develop a trimester report for GALME coordinators. A community information sheet was developed to measure changes at the community level and has been used during 1997 and repeated in 1999. It includes use of the UROC, formation of LHC, latrines, other projects, and community population. The information from this sheet was not focused enough to be used in guiding project implementation. This could be a valuable tool for project management, but HOPE needs to more clearly define what their information needs are. The census is updated every 4 months with new births (although deaths are not registered) and annually by MINSA. Qualitative Studies Focus groups were carried out on a number of occasions: · to learn about breastfeeding beliefs and practices used to develop educational messages; · to discuss with CHVs problems in one zone, information from these groups was used to improve the supervision system and coordination; · groups with women were used to design educational materials and identify locally available foods. The use of qualitative methods has been one of the strengths of the project. Project HOPE in Peru CSXII Final Evaluation 30 Nutritional Surveillance The nutritional surveillance system being used is ANA, a software package developed by PRISMA and shared with HOPE. HOPE has completed 9 weighing sessions as of May 2000 and currently have 2,287 children registered in the database which includes an analysis of Wt/Ht, Wt/Age, Ht/Age. CHVs and health personnel receive a computerized feedback report of which children are malnourished usually within 1 month. The feedback loop is unnecessary and creates a dependency on the computerized system. CHVs should identify malnourished children, and begin immediate actions including reporting to the LHC and community. To do this, the CHVs need additional support and training in diagnosis and follow-up. The feedback loop should only be used by MINSA staff to insure that the CHV is following-up on all cases. In two Health Networks, MINSA staff is beginning to enter data into ANA and HOPE continues to do the data entry in the other area. Information from the nutritional surveillance has been distributed to MINSA at various levels and has been used for information on the nutritional status at the community level, to verify malnutrition levels in the region, and in the prioritization of areas for food distribution programs, but the widespread impact of information from the project comes more from special studies as previously discussed in B.2.c New Tools/Approaches. The completion of the surveillance is used as an indicator of the level of activity of CHVs and as a way of prioritizing supervision visits. Is the ANA system sustainable? Given the other priorities of health staff and the limited use of the information from the database, which is already available at the community level without a computerized system. The project should seriously evaluate the value of maintaining the ANA database, in terms of sustainability and impact at the household level in solving the problems of malnutrition. Information Management and Use A computer specialist was hired 4 months ago to work with the CS and community bank projects. The management of the HIS is decentralized, with some people working on each database. The principal databases are; Census, Community Information, Training received by CHV, and Nutrition status (ht/wt/age). The computer person has received little training, for example in Epi-Info; he learned mainly by working on the KPC. The project needs a clear vision of what they want to accomplish with the HIS during the next three years, whether the focus is on the community use of information, or support of the MINSA system. Given the lack of progress in this Project HOPE in Peru CSXII Final Evaluation 31 area during the first four years, it is doubtful whether both objectives can be reached. If HOPE opts for working more on community use of information, simple mechanisms need to be developed such as a chart showing the number of malnourished children in the community month by month. Information needs to be put within a context that communities can understand and use. Community maps were introduced, but have not been sufficiently used to reach their potential as a community tool. Community mapping as an information use tool should be strengthened. Sixteen of the 23 CHVs interviewed during the FE reported that they communicate to the community the number of malnourished children, but few communities were motivated to take any specific actions based on the information. Community leaders, committees and volunteers require additional support for identifying concrete steps for resolving health problems. A comprehensive study of needs and an information use plan for the development of an inclusive HIS is urgently needed for the 3-year extension. The development of the HIS should be completed simultaneously with the DIP, to insure that all indicators are measurable with the HIS. 8. Technical and Administrative Support The former HOPE director and UPCH liaison in Peru has now been hired directly by HOPE headquarters’, where he continues to offer technical assistance to the project in technical issues and assistance with the KPC. This relationship provided good continuity throughout the four years of the CS-XII project. A great deal of technical support for the project comes from the relationship with UPCH. The university has been instrumental in providing the CS project with numerous research studies and support for planning and analyzing data from project studies, such as the KPC. The current HOPE director also has professor status at the university, so it is anticipated that this close relationship will continue. HOPE headquarters’ representative has visited the project approximately twice a year during the four years of the grant. No visits were made during the last year, until the FE. The backstopping person is allocated 3pm (person months) for this project, but actually devotes approximately 35% of his time. He is also responsible for partially backstopping projects in Nicaragua, Guatemala, Ecuador, Haiti, and Mozambique More technical assistance is needed in family planning, HIS development and management, and IMCI. HIS development may be possible through an Project HOPE in Peru CSXII Final Evaluation 32 exchange visit by staff from HOPE-Nicaragua or HOPE Guatemala. Technical assistance in IMCI has already been arranged with PAHO during October of this year. 9. Management Lessons Learned · Planning of the project and development of the DIP is crucial and needs more involvement of the field office, all levels within MINSA, and the community · If MINSA is a full partner, they need the same access to training and staff development as HOPE staff · Ongoing planning of activities, including evaluation and modifications of strategies should occur continuously during the life of the project with principal partners. D. Conclusions and Recommendations It is difficult to appreciate the impact of the project due in part to the use of indicators that do not reflect what the project has accomplished in the last four years. Only three of the 13 objectives were actually met during the four-year period. This can be partially explained by an overly ambition proposal and not taking into consideration factors which were outside of the scope of the project, i.e. prevalence of anemia and diarrhea and food consumption. The project has shown positive impact in two very important areas; percentage of women who exclusively breastfeed until 6 months of age and percentage of stunting in children. Other important improvements include frequency of growth monitoring, early initiation of breastfeeding and use of family planning methods. Some of the major achievements of the project include: · the development of a very effective educational methodology · encouraging the valuation of local foods-not donated or processed foods, · improved quality of nutritional surveillance, · establishment of a network of trained CHVs, · development of formula for fortified foods, · studies which have allowed: focused messages i.e. initiation of feeding bananas; better understanding of complex disease states, i.e. anemia; and improved information for use by MINSA officials i.e. iodine content of salt., · development of leadership skills in women and greater solidarity through GALME, · efforts made by the staff to implement a good quality project under very adverse conditions. Major Problems that still need to be addressed during follow-on project: · HIS is not fully functioning · Supervision and monitoring is weak at all levels Project HOPE in Peru CSXII Final Evaluation 33 · Collaboration and communication with MINSA needs to be further improved, the project’s role in supporting MINSA, that means total involvement together-planning, evaluation, implementation · Defining the needs for capacity building and institutional strengthening-find out what is needed and work on systems development · Further strengthening of community structures, including liaison with MINSA, municipalities and other organizations, Hope should serve as a temporary bridge between MINSA and Communities, Communities and other organization, The darker lines represent the sustainable relation which the project should strive for. All levels need to be strengthened, it can not be assumed that strengthening the committee will automatically strengthen women, nor only working at the Health Network level will support the needs of staff at the health post level. Project HOPE has recently launched HOPENET, a web-based Intranet for the exclusive purpose to improve channels of communications within the Foundation in the U.S. and abroad. At this time, all HOPE field offices can access updated information on a variety of subjects pertaining current and past initiatives implemented by HOPE. As part of this effort, lessons learned from all HOPE programs will be posted on the Intranet for all HOPE staff worldwide. HOPENET will be updated on a quarterly basis to include not only information on CS projects but other HOPE initiatives as well. Community: Women Men Committee Authorities MINSA: DIRES HN Center Post Municipalities Other Organizations Project HOPE Project HOPE in Peru CSXII Final Evaluation 34 Lessons Learned: · HOPE, MINSA and the community have to work more closely together. HOPE-MINSA needs to be established as a concept, not just each independently · The development of activities which respect and value the local culture have the greatest impact · Involving MINSA in training activities aids in reaching the communities with health messages. · Unifying the language used in messages promotes better learning · For adequate implementation, the project should involve all partners (MINSA HOPE Community) from the beginning · Continuous follow-up and support are needed for the development of new abilities · The supervision checklist on anthropometric measurement was very effective · The GALME should be for all women, include other topics of interest to the women and include activities with their husbands. · The greatest impact in changing practices comes with clear, simple messages, based on previous study · Take advantage of the felt need of women to meet in their own group, according to their own vision · To decrease the prevalence of diarrheal disease it is necessary to improve sanitation conditions through coordination with other organizations that work in this area. · Educational messages which focus on improving feeding during and after a disease episode will have the greatest impact on changing knowledge and practices · A project can not advance without the support of local authorities. · Communities must be involved as full partners in the development process, including being involved more in planning and evaluating · Don’t divide the work with MINSA, go with them to support the implementation and quality of the work. · The use of checklists clarifies expectations and defines what elements are essential · CHVs should not have an institutional affiliation, but should clearly be of the community. · Initially identify CHVs who have been trained by other organizations · Regular contact is essential to support the work of the volunteer · More emphasis needs to be given on training to train and supervising the replication of training. · Other members of the community need to be trained to create an environment of change · Sustainable training structures must be developed · Planning of the project and development of the DIP is crucial and needs more involvement of the field office, all levels within MINSA, and the community Project HOPE in Peru CSXII Final Evaluation 35 · If MINSA is a full partner, they need the same access to training and staff development as HOPE staff Recommendations: · The project should select one measure for determining nutritional status and focus training on counseling the family on what concrete actions can be taken. · As part of the final evaluation of the follow-on project, the impact of the food dryer and fortified food should be included. · The formation of a link between GALMEs should be established, using one (or more) of the following mechanisms: 1). Formation of a network of GALMEs 2). Integration of the Coordinators of GALMEs into the CHV network 3). Visits between groups for sports events and to exchange ideas about health issues 4). Annual conference of all coordinators to reinforce their work and to define future directions · Elaborate and broadcast basic integrated messages aimed at improving knowledge and practices of mothers with ill children (Diarrhea, respiratory infections, malaria) with a focus on feeding during and after the illness. · During the next phase of the CS project, one focus of education should be aimed at decreasing the use of inappropriate medications for diarrhea. · Alliances should be formed with organizations who have financing and/or offer technical assistance in construction or latrines, water systems, as well as agricultural activities. · Strengthen the UROC as a community structure with an integrated preventative health role in the community through increased community participation. · Implement family planning activities by involving men and husbands in educational activities. · Training for committee members and leaders in project development and community organization should be prioritized during the next 3 years. · Community diagnosis and ongoing meetings to evaluate, analyze and plan should be held together; community (including women), health personnel and HOPE. · CHVs and GALME coordinators need improved skills in more than message delivery but also in counseling, identified barriers to change and negotiating behavior change. Project HOPE in Peru CSXII Final Evaluation 36 · MINSA and HOPE should strengthen follow-up to insure the quality of replication of training courses and the education in the communities. · HOPE should determine a mechanism for acquiring educational materials for all communities and MINSA health posts to support educational messages. · HOPE should seek technical assistance on institutional assessments from other NGOs who have experience in this area. · The anthropometric checklist should be expanded to include diagnosis of malnutrition and counseling of the mother on what actions she can take. The checklist concept should be expanded to include other areas for supervision. · Both a formal and informal system of communication should be developed between HOPE and MINSA: · HOPE should participate in the planning and evaluation meetings of the AIN 2001(Integrated Care of the Child) program. · Regular meetings with the DIRES, UBASS , health center and health post levels · An annual planning and evaluation workshop should be implemented with participation from all levels, to review project achievements and together plan for needed readjustments. · The strategy of using CHV couples should be re-evaluated during the second phase of the project. · Establish a system of indirect supervision using regular monthly meetings between health post staff and all CHVs from the post. The meetings would be used for analysis of information, to reinforce training topics, distribution of materials (ORS), discussion of problems and solutions, and to prioritize needs for direct supervision visits. · HOPE and MINSA should collaborate on the development of an integrated supervision instrument. · Increase the coordination between communities and municipalities and other organizations. · Create Trainers within MINSA in health, gender and educational methodologies. · Create Trainers within the municipalities and teachers in community organization and project development, leadership and participative methodologies. · Encourage communities to identify alternative resources for training and development activities. · Advocate the commitment of CHVs who retire from service to train a replacement Project HOPE in Peru CSXII Final Evaluation 37 · Investigate with the CHV associations what role they can play in training new CHVs · Develop a self-guided manual for medical students who work 1 year in the rural area and new MINSA staff on community participation · The DIP for the 3 year extension should be developed in a participatory manner, involving community, and MINSA at all levels. · A comprehensive study of needs and an information use plan for the development of an inclusive HIS is urgently needed for the 3-year extension. Project HOPE in Peru CSXII Final Evaluation 38 E. Results Highlight COCOA FORTIFIED WITH HEME IRON EFFECTIVE STRATEGY FOR COMBATING IRON DEFICIENCY ANEMIA The Child Survival project implemented in rural communities of northern Peru by Project HOPE and the Ministry of Health encountered a high level of anemia in women and children during the baseline study in 1997. The prevalence was 49% in children under 3 and 21% in women of reproductive age, along with a high incidence of parasites and diarrhea in children. As the main cause of anemia is a deficiency of iron this spurred the project to find a way to increase the availability and consumption of iron. In order to reduce the rate of anemia in women and children, the project looked at fortification as a viable strategy thus the idea of aiding the commercialization of a food product which offered a low cost, acceptable source of micro￾nutrients for vulnerable families was born. The following steps were followed in developing the final product a.) Formulation The first step was to identify a locally available product which was compatible with fortification. A document review of fortification indicated that the most acceptable and available product would be FERRIMIN, a cow’s blood source of protein and iron with 2.7-3.1mg/g, heme iron with a bio-availability of 30% produced in Chile. Laboratory and cost studies evaluated physical characteristics, and products which offered approximately 150% of the daily requirement of a child at an affordable cost. Studies were made of cocoa, banana flour and cookies resulting in a product of 25% Ferrimin and 75% locally grown cocoa (1.25 grams Ferrimin/3.75 gms cocoa). b.) Acceptability The formula underwent acceptability studies in four different products; cocoa mixed with water, mixed with milk, thickened with banana flour and fudge. Taste tests were carried out in the urban and rural areas with men, women, and children. c. ) Effectiveness Four studies were carried out in the project area by students from the University Peruana Cayetano Heredia on the effectiveness of the product in reducing anemia. A summary of the studies shows a high level of recuperation in various population groups: Quijano Alarcón L. Casanova B. Casanova 56% in 30 days 50 % in 56 days 63 % in 56 days 100 % in 28 d Schoolchildren Adults-both sexes Adults-both sexes Women >14y With anti-parasite With anti-parasite With anti-parasite Cutoff points were 13 gr/100ml-men, 12 gr/100ml–women, 11gr/100ml for children and pregnant women. It was concluded that cocoa fortified with Ferrimin had a high level of effic acy with a greater than 50% rate of recuperation of anemia with 1-2 months of treatment. It can be recommended that for use in treating iron-deficiency anemia, the cocoa be used in conjunction with an anti￾parasite drug, particularly in rural tropical areas. d. ) Marketing The project developed a package of technical information on fortified cocoa that was made available to small business entrepreneurs on the efficacy and merits of the product. Fortified cocoa is now available on the local market with limited distribution. The project continues with the challenge of acting as intermediary to encourage the production and distribution to a wider market, particularly those most effected by under nutrition.