World Relief Mozambique Vurhonga II CSXV Child Survival Project 2001 MIDTERM EVALUATION REPORT Evaluation Consultant: Marc Debay, MD, PhD, MPH Johns Hopkins University World Relief Contacts: Anbrasi Edward Raj, PhD,MPH, Director of Maternal Child Health Melanie Morrow, MPH, Child Survival Specialist World Relief Corporation PO Box 868 Baltimore, MD 21203-0868 Tel: (443) 451-1900 Fax: (443) 451-1955 WR Mozambique Contacts: Pieter Ernst, MD Child Survival Project Director Sam Grottis, WR Mozambique Country Director Project Dates: 30 September 1999 – 29 September 2003 Cooperative Agreement #: FAO-A-00-99-00042-00 World Relief Mozambique CSXV Midterm Evaluation Report 2001 i Table of contents I. Executive summary ................................................................................................... 1 II. Assessment of progress towards objectives ............................................................ 3 A. Project implementation overview ............................................................................. 3 B. Technical approaches ................................................................................................ 4 (1) Malaria ............................................................................................................... 4 (2) Diarrhea .............................................................................................................. 6 (3) Nutrition ............................................................................................................. 7 (4) HIV/AIDS and STIs ......................................................................................... 10 C. Cross-cutting approaches ........................................................................................ 12 (1) Community mobilization.................................................................................. 12 (2) Behavior Change Communication ................................................................... 12 D. Partners strengthening ............................................................................................. 14 E. Sustainability........................................................................................................... 17 III. Program management ............................................................................................ 20 A. Planning .................................................................................................................. 20 B. HIS .......................................................................................................................... 21 C. Staff development and management ....................................................................... 23 D. Financial and logistic management ......................................................................... 24 E. Technical and administrative support ..................................................................... 25 IV. Conclusions .............................................................................................................. 26 V. Results highlights .................................................................................................... 27 VI. Action Plan .............................................................................................................. 28 VII. Attachments ............................................................................................................... A. Baseline Information from the DIP ........................................................................ A1 B. MTE Team Members and other persons contacted ................................................ B1 C. Assessment Methodology ....................................................................................... C1 D. Schedule of the midterm evaluation ...................................................................... D1 E. Project implementation timeline: October 1999 – August 2001 ............................ E1 F. Progress towards objectives: malaria ...................................................................... F1 G. Progress towards objectives: diarrhea .................................................................... G1 H. Progress towards objectives: nutrition ................................................................... H1 I. Cause Specific Number of Deaths ........................................................................... I1 J. Response to DIP Review Recommendations ........................................................... J1 World Relief Mozambique CSXV Midterm Evaluation Report 2001 ii Acronyms AIDS Acquired Immune Deficiency Syndrome BCC Behavior Change Communication CG Care Group C-HIS Community-based Health Information System CS Child Survival CSTS Child Survival Technical Support DIP Detailed Implementation Plan EPI Expanded Program of Immunization HIS Health Information System HIV Human Immuno-Deficiency Virus ITN Insecticide Treated Nets KPC Knowledge, Practices and Coverage survey MCH Maternal Child Health MOH Ministry of Health MTE Midterm Evaluation NGO Non-Governmental Organization ORT Oral Rehydration Therapy PVO Private Voluntary Organization QS Quarterly Survey STIs Sexually Transmitted Infections TBA Traditional Birth Attendant USAID United States Agency for International Development VHC Village Health Committee WR World Relief Chef de saude Village Health Representative Socorrista Trained first aid health worker at health post World Relief Mozambique CSXV Midterm Evaluation Report 2001 1 I. Executive summary The Vurhonga II Child Survival XV Project (1999-2003) covers the population of 130,000 living in the rural area of the Chokwe district in the Gaza province of Southern Mozambique. The goals of the project are to (1) reduce morbidity and mortality in children under five and women of fertile age; (2) strengthen the capacity of the Ministry of Health (MOH) to implement child survival interventions; and (3) empower communities to improve their health. The local partners are the district-level management team and the health centers of the MOH and the churches and communities of rural Chokwe. The main strategy of the Vurhonga II project is to help all the communities in the impact area to create Care Groups (CG) of volunteer mothers, and then train these volunteer mothers and the church leaders in their communities as behavior change agents. The child survival behaviors that the project promotes are related to general nutrition, malaria, pneumonia, diarrhea, maternal and newborn care, child spacing, and STI/HIV/AIDS prevention. The project also helps the MOH to expand its network of health centers and health posts. The implementation of the Vurhonga II project began rapidly and effectively despite the disruption caused by the February 2000 flood that affected the entire project area. This successful start in the face of natural disaster is largely due to the very competent and dedicated World Relief (WR) staff transferred from the Vurhonga I project, a previous child survival project implemented from 1995 to 1999 in two districts near Chokwe. In addition, the assistance that several emergency flood relief agencies provided to the beneficiary population of the project area complemented very well that provided by the Vurhonga II project. At midterm, the Vurhonga II project has already established a network of about 200 CGs enabling 2350 volunteer mothers to provide health education through regular home visits in the project area. These volunteer mothers reach all the households in the project area, and appear to have effectively changed the behaviors related to the child survival interventions for which they have been trained: malaria, diarrheal diseases, and nutrition. Overall, all the specific targets for these three interventions are achieved or surpassed. The Midterm Evaluation (MTE) team identified the following constraints and issues that need further attention: - Periodic shortages of chloroquine in the health posts and no assurance yet that the kit for re￾treatment of Insecticide Treated Bednets (ITN) will be available when needed. - Persistence of false beliefs about causes and treatment of convulsions, and about the role of the mother’s milk as a cause of diarrhea - Lack of transport to refer patients from remote villages to the Chokwe district hospital - Possible decrease in latrine use if the construction and maintenance of latrines is neglected - Potential food shortages over the next few months that can override the gains in nutritional status from health education activities and from the Hearth program - Lack of local availability of peanuts as key ingredients of the enriched porridge used in the Hearth program - Possibility that the strategy for HIV/AIDS prevention based abstinence and faithfulness might not be as effective as expected given that promiscuity is often tacitly accepted World Relief Mozambique CSXV Midterm Evaluation Report 2001 2 - Potential conflict between the Vurhonga project and MOH and other agencies supporting the promotion of condoms - Potential negative reactions or resistance by the population to HIV testing The Vurhonga II project made significant progress in building the capacity of its partners: - More than 95% of the beneficiary population has access to primary health care in the health centers of the MOH or in the health posts established by communities - Most of the church leaders in the project area support the Vurhonga II project, and they are aware of and help promoting the key behaviors targeted by the project - Most villages in the project areas have established Village Health Committees (VHC) that meet on a regular basis to review the health issues in their communities, and in particular the information provided by the monthly reports from the CGs This progress in building the capacity of the MOH, the churches, and the VHCs, along with the structure of the CG based on motivated volunteer mothers supported by the community leaders, are promising aspects of the sustainability of the Vurhonga II project. Most community leaders, villagers, and volunteer mothers are well aware that the project will end in a few years, and all express a clear determination to continue supporting the CG activities. However, the VHCs and the health posts are relatively new and still need reinforcement. The MTE team formulated the following main recommendations to the Vurhonga staff and its partners: 1. Strengthen the VHC, and in particular the village leaders, the chef de saude, and the CG leaders; 2. Strengthen the MOH capacity to supervise the community health activities of the CG, the socorristas, and the VHC. 3. Raise selected project targets (early treatment of fever; ORT use; extra food during following diarrhea; counseling and enriched porridge to malnourished children). 4. Conduct a formal assessment and documentation of the impact and sustainability of the CG approach by the end of the Vurhonga II project in 2003. World Relief Mozambique CSXV Midterm Evaluation Report 2001 3 II. Assessment of progress towards objectives A. Project implementation overview Some of the baseline information and the general goals and objectives of the Vurhonga II project included in the Detailed Implementation Plan (DIP) are reproduced in Appendix A. Appendix E presents the implementation timeline of the main activities of the Vurhonga II project from October 1999 to August 2001. The project started by transferring most of the staff from Vurhonga I to rapidly begin work in the new area. It is when project and headquarters staff began the preparation of the DIP in Chokwe in February 2000 that the largest flood in Mozambique for decades submerged the entire project area and facilities. The emergency flood relief operations extended over a period of five months and involved most of the project staff and resources. Given these circumstances, USAID/BHR/PVC accepted to review the Vurhonga II DIP one year later than expected, that is, in June 2001. Therefore the Vurhonga II DIP was finalized in February 2001 and covers more than one year of project implementation. This very complete and well-written document includes all the technical references for the project, which will therefore not be reproduced in the present report except for what is included in Appendix A. The project has established an elaborate health information system that builds on that of Vurhonga I. The Knowledge, Practice, Coverage (KPC) survey was conducted in October 1999, a census of all the women of reproductive age and children under five was completed in January 2000, and a series of quarterly surveys began in August 2000. The community-based information system began in June 2001 after the volunteers were trained in malaria, diarrhea, and nutrition. The animators1 and supervisors began selecting volunteer mothers in the new project area in December 1999. This work involving the communities was slowed down in the flood and its aftermaths, but all the CGs2 had been formed and the volunteer mothers were ready to be trained by June 2000. During the same period, the project staff revised all the training curricula and health educational materials and prepared them for reproduction. The training of volunteers and the subsequent education of the mothers and their families began in June 2000 with malaria. Then the diarrhea and nutrition interventions were phased-in in September and February 2001, respectively. The nutrition intervention included Hearth for growth faltering children and lasted longer than the other two interventions. It was immediately followed by the HIV/AIDS intervention in August 2001. In addition to the intensive education of the mothers of children under five, the Vurhonga II project collaborated with the MOH and other international partners who remained after the flood relief operations to increase the accessibility of the population to primary health care services. A 1 The Vurhonga II project employs 26 animators who provide training and support to 8 Care Groups each. 2 A Care Group comprises 10 to 15 volunteer mothers working in the same area and in charge of up to 10 households. Care Groups are the core structure of the Vurhonga project strategy. World Relief Mozambique CSXV Midterm Evaluation Report 2001 4 total of 18 socorristas who had been selected by the communities and trained by the MOH and the Vurhonga project began working in rehabilitated or newly built health posts during the last quarter of 2000. By the end of year 2000, a total of 28 health posts were functioning in the project area. By February 2001, all but 4 of the 48 villages in the Chokwe District had established a Village Health Committee (VHC). B. Technical approaches The MTE team reviewed the four child survival interventions of the Vurhonga project for which the training of the staff and the health education of the volunteers and mothers had already been conducted: malaria, diarrhea, nutrition, and HIV/AIDS (this last intervention began in August 2001). The progress towards the specific project objectives for each of these interventions is reviewed below. The main achievements and challenges are discussed, and specific recommendations are made. The results of the KPC and the four quarterly surveys for selected monitoring indicators of the malaria, diarrhea, and nutrition interventions are presented in Appendix F, G, and H. (1) Malaria Project activities related to malaria are geared towards prevention, recognition, and early treatment seeking. Formal training of CGs and mothers began in June 2000. Health education has been strengthened by two additional activities: 1) distribution of insecticide treated nets (ITN) as part of flood relief in September 2000, and 2) creation of health posts and training of socorristas so that treatment with chloroquine is now accessible in most villages. Specific measurable objectives, targets, and progress as reported in the August 2001 quarterly survey are presented in the table below. Where appropriate, the new target set by the MTE team is given. Objective with baseline and original target Progress Adjusted Target 1. Increase from 0.3% to 70% the proportion of children under five who sleep under an ITN year￾round. 68% (Peaked at 93% in April 01) NA 2. Increase the percent of children treated within 24 hours for fever (suspected malaria) from 28% to 75%. 86% 80% Notes: - The baseline value (KPC) of the indicator for objective 2 is based on 25 cases of fever (28%; 95% Confidence Interval: 3%-53%) - The indicator of the ITN objective refers to children under five who slept under an ITN the night before the survey (not year-round). It is calculated using the number of children under five who have an ITN as denominator (about 93% in April 01) and therefore overestimates the actual proportion of children who sleep under a net (by 6.5% in April 01). World Relief Mozambique CSXV Midterm Evaluation Report 2001 5 - The target for the care seeking objective was raised but at a lower level than the most recent quarterly survey because staff members felt enough challenge with sustaining the increase already achieved and ensuring that the treatment received is adequate and complete. Achievements A strength of the malaria intervention is the very high coverage of ITNs in households with children under five. Oxfam provided the bed nets during the emergency flood relief operations, and the Vurhonga project contributed to their distribution and impregnation. Quarterly survey data reveal that the percentage of children sleeping under ITNs increased dramatically following ITN distribution and related education of the mothers, but that it recently tapered off during the dry ‘winter’ months. During the site visits, community members explained to the MTE team that they perceived less need to use ITNs during the winter because there were fewer mosquitoes, but that this was progressively changing with the continuing health education activities in the villages. The MTE team noted the widespread awareness of the project health education messages related to malaria. The mere mention of Vurhonga prompted villagers to share what they had learned about malaria, and in particular the importance of seeking care early at the health post. The project target for early treatment seeking for fever was surpassed a few months after the health education on malaria was carried out. Most of the new health posts with trained socorristas were already functioning at that time. Challenges Provision of chloroquine in the villages makes for significant improvement in access to treatment for malaria. However, periodic shortages of chloroquine have been noted since the establishment of the 28 health posts in the project area. The District MOH is well aware of this problem due to the rapid increase in demand for chloroquine (each health post reports on the availability of chloroquine on a monthly basis). Requests for increased supplies for next year have already been made. In the meantime, the MOH has coped with the drug shortages by re￾allocating some chloroquine designated for the hospital to the health posts. Old beliefs about the causes and treatment of convulsions persist, but the need for change is recognized, in particular with respect to the use of health services. As socorristas still must refer patients with convulsions and other complications to the district hospital, the lack of transport from remote villages to Chokwe town often presents challenges to the patients or their families. Six months after ITNs were distributed, UNICEF made use of Vurhonga’s volunteers to provide the re-treatment of nets with insecticide. Though Vurhonga animators and volunteers can serve as distribution channels for re-treatment, responsibility for the supply of re-treatment kits (to be sold at subsidized cost) remains with the government in collaboration with PSI and UNICEF. The next round of re-treatment is fast approaching but kits are not available yet—suggesting that the effectiveness of the bed nets in preventing malaria may decrease in the near future. New nets should also be made available for sale at reduced prices as some mothers report damages to their nets by rats and other causes. World Relief Mozambique CSXV Midterm Evaluation Report 2001 6 Recommendations 1. The Vurhonga project should regularly review the MOH statistics tracking the availability of chloroquine at health posts 2. The Vurhonga animators and volunteers should reinforce messages related to year-round use of ITNs, and in particular for children under five 3. The Vurhonga project should encourage the MOH, UNICEF, and PSI to soon make ITNs and re-treatment available at low cost in the villages 4. The Vurhonga project should encourage the Village Health Committees to address and develop solutions to the issue of referral in their community. 5. The Vurhonga project should consider assessing compliance with malaria treatment received in health posts in one of the HIS surveys. (2) Diarrhea Promotion of hygiene and control of diarrheal diseases began under emergency conditions in the Xiaquelane disaster camp during February and March 2000. The project animators mobilized volunteers to disseminate hygiene messages to the 60,000 people seeking refuge at Xiaquelane. Oxfam constructed latrines in the camp and later assisted with their construction in the villages. Formal training of CGs and mothers in diarrheal diseases control began in September 2000. Specific measurable objectives, targets, and progress as per the August 2001 quarterly survey are presented in the table below. The new target set by the MTE team is given. Objective with baseline and original target Progress Adjusted Target 1. Increase from 53% to 75% the proportion of children with diarrhea treated with ORT by mothers/volunteers. 88% 80% 2. Increase from 19% to 65% the proportion of mothers who give extra food to children for two weeks following diarrhea. 71% 75% Notes: - Both diarrhea project objectives pertaining to diarrhea have already been exceeded, and both end-of-project targets were raised. - The indicator pertaining to increased feeding following diarrhea lacks precision because it refers to children who had diarrhea during the last two weeks and who therefore have not had the time to receive extra food for two weeks following their diarrhea episode. Achievements Mothers are giving more ORT than at the beginning of the project, and they are following up with additional food for two weeks following the episode of diarrhea. ORS is more readily available in the health posts and health centers than chloroquine. The MTE team found evidence for improved household hygiene and sanitation in the villages. Households neatly kept and racks to keep dishes off the ground are a common sight. Latrines are World Relief Mozambique CSXV Midterm Evaluation Report 2001 7 widely available, thanks in part to the Oxfam intervention, as already mentioned. Latrine coverage in the villages has increased from 28% of households just after the flood to over 75% as tracked in the community HIS. Community members commented to the MTE team that mothers were taking more care in the management of their households than ever before, and there was a sense of pride concerning the tidiness of the village. A pastor elaborated on the importance of cleanliness by explaining that children can get diarrhea from putting dirty objects in their mouths. Challenges Villagers agreed that the construction of latrines was feasible for most people in the community and that common excuses including lack of materials were more related to lack of motivation than valid obstacles. However, the construction and maintenance of latrines could become overlooked and latrine use could decrease. Another challenge to the diarrhea intervention is related to breastfeeding. Traditionally, a child with diarrhea might be abruptly weaned due to the belief that the mother’s milk had gone bad and caused the diarrhea. Fortunately this concept seems more commonly mentioned than practiced; consensus among those familiar with the concept, including grannies, was that one should wait for advice from the health post before implementing such a practice. Recommendation 1. The Vurhonga project should continue monitoring latrine coverage and use in the community; volunteers (and village leaders, as necessary) should follow up with families who do not yet have latrines. 2. The Vurhonga project should continue monitoring the weaning of babies with diarrhea to ensure that this harmful practice decreases. (3) Nutrition The nutrition intervention has several components: household nutrition education via CG volunteers, growth monitoring in the health centers and during EPI mobile clinics, counseling of mothers or Hearth program for malnourished children. Food security is a recognized issue but not an intervention per se. The formal training of the CGs and the mothers in nutrition began in February 2001. Two Hearth cycles were organized in May and July 2001. World Relief Mozambique CSXV Midterm Evaluation Report 2001 8 Specific measurable objectives, targets, and progress as per the August 2001 quarterly survey are presented in the table below. Where appropriate, the new target set by the MTE team is given. Objective with baseline and original target Progress Adjusted Target 1. Increase from 75% to 85% the number of children 0- 35 months weighed at least once during last 3 months. 83% NA 2. Increase from 22% to 80% the number of malnourished children’s mothers who receive nutrition counseling. 95% 90% 3. Increase from 50% to 70% the number of malnourished children who received nutritious foods/enriched porridge. 97% 80% 4. At least 70% of children who completed the HEARTH program achieved and sustain adequate or catch-up growth per month during at least 2 months after HEARTH. 77% (At end of second cycle) NA Note: - The baseline value (KPC) of the indicators for the objectives 1, 2, and 3 are based on children 0-23 months; the value for that of objective 2 is based on 46 malnourished children, and that of objective 3 on 10 malnourished children who received nutritional counseling (50%; 95% Confidence Interval: 6%-94%). - Progress in regard to objective 4 could only be assessed with respect to children’s growth by the end of the Hearth cycle because two months had not passed yet to assess if gains were sustained. Achievements The proportion of children 0-35 months weighed during the last three months has remained stable at about 80% since the beginning of the project, but the proportion of malnourished children who receive counseling and of those who receive enriched porridge increased when the related health education of the mothers and of the Hearth program began. Nineteen hundred children with faltering growth plus their mothers participated in the first cycle of Hearth. Those children who did not show satisfactory improvements during the first cycle were included in the second. By the end of the second cycle, 77% had achieved adequate or catch-up growth. The Hearth program has been very much appreciated in the community. Myths concerning malnutrition are being overcome. Traditionally, for instance, growth faltering is attributed to the child’s spirit being ‘sat upon.’ Hearth participants’ improvement was so noticeable that grannies, mothers, and men alike came to see the positive effects of good nutrition. At Conhane, for example, one granny said she used to think a child who wasn’t growing well had been ‘sat upon’ but then Vurhonga showed that the problem was actually due to malnutrition— World Relief Mozambique CSXV Midterm Evaluation Report 2001 9 demonstrated by Hearth. She said she intends to tell other people who don’t know yet about what she learned. Another achievement related to the Hearth program is the development of a recipe for enriched porridge that mothers and children readily adopted. In villages the MTE team visited, people named the ingredients without any prompting. Some further showed off their knowledge by reciting the correct proportions of maize, greens, and peanuts. Knowledge of Hearth was widespread and even those who were not participants had managed to taste the new recipe for enriched porridge. Challenges Despite the Hearth’s apparent success and widespread popularity, gains in nutritional status may be lost in the face of a food shortage projected to worsen over the next five months. The present (and worsening) food shortage is related to a combination of factors. The flood of 2000 was followed by drought and consequent decreases in production of the staple crop, maize. Then, before the meager crop could be harvested in April 2001, a plague of rats consumed some of what had survived the drought. People continue to be at the mercy of the weather as the canal system usually relied upon for irrigation is under repair for flood damage. The soonest people can hope to have sufficient rain for planting again is September—resulting in a December harvest at best and sufficient time for children to become severely malnourished. One difficulty with the enriched porridge recipe is also related to drought. Peanuts are key ingredients in the enriched porridge for both their protein and energy content. Under ordinary circumstances they are grown in the project area—but this year they are very scarce. Though mothers contributed maize, greens, firewood, and water to the Hearth sessions, Vurhonga had to supply the peanuts (and oil in some villages where that was also used). Consequently mothers will be unable to include peanuts when they make the recipe on their own, until the next peanut harvest. In Vurhonga I, the beneficiary population had access to Marula nuts even when other foodstuffs were in short supply—but these trees are uncommon in Chokwe district. In the quarterly surveys, the children 0-35 months below the 3d percentile of the reference weight-for-age as per their Road-to-Health chart are identified as malnourished, and the proportions of those who receive counseling or enriched food are the indicators of the coverage of the nutritional education including the Hearth program. In May 2001, the proportion of children 0-35 months below the 3d percentile was 25%. This indicator may not detect a rapid deterioration of the nutritional status of children under five that can result from a shortage of food. Recommendations: 1. Given the risk of food shortage, the Vurhonga project should define a nutritional surveillance indicator and adopt the related threshold to determine when the malnutrition levels require supplemental feeding. The project should also investigate options for supplemental food via the World Food Program in case a rapid response is needed. World Relief Mozambique CSXV Midterm Evaluation Report 2001 10 2. The Vurhonga project should consider encouraging mothers to make dried greens during the rainy season when they are plentiful—a traditional practice that was discontinued years ago, and promoting marula nut trees in Chokwe district for long-term nutritional benefit. (4) HIV/AIDS and STIs Vurhonga strategy for the prevention of HIV/AIDS and STIs is the promotion of abstinence and marital faithfulness—an approach that is acceptable to the local churches. Formal training of volunteers in the HIV/AIDS and STI intervention was in its first days at the time of the MTE. Thus the MTE team did not look for evidence of impact but got to observe animators teaching the first lesson to volunteers. Specific measurable objectives and targets for the HIV/AIDS and STI intervention are presented below. Objective with baseline and original target 1. Increase from 0.3% to 40% the number of mothers who know 3 ways to prevent transmission of STIs including HIV/AIDS. 2. Increase from 1.l6% to 40% the number of women and partners who recognize at least 3 common symptoms of STIs. 3. Increase from 0.3% to 40% the number of women and partners who recognize that STIs increase risk of HIV/infertility. Notes: - The low level of knowledge of STIs and HIV/AIDS is noticeable. - The indicators for the objectives 2 and 3 are the proportion of women (not women and partners). - The three indicators above relate to knowledge and there are no behavior indicators for the intervention. Achievements As mentioned, the HIV intervention is just being introduced at the village level. Achievements to date include the development of pictorial teaching aids and training of animators in HIV/AIDS. The information is anticipated to be well received for several reasons. To begin with, there is very high perceived need for information about HIV/AIDS. People in the villages have observed the untimely deaths of men who returned from South Africa only to lose weight, die, and have their wives follow suit. They want to know how they can protect themselves and their families from the same fate. In addition to the timeliness of information on HIV, World Relief’s approach to prevention (via abstinence and marital faithfulness) is consistent with values and behaviors esteemed by churches in the project area. This is significant as churches are the most prominent social institutions in the project area, with over 60% of the population identified as Christian. To illustrate, a woman in one village explained that when others have come to promote condom use rather than fidelity, the church rejected them along with their messages. World Relief Mozambique CSXV Midterm Evaluation Report 2001 11 Challenges The Vurhonga staff members are aware of potential challenges that addressing HIV/AIDS in the villages may invoke. To begin with, they are aware that promiscuity is tacitly accepted in the villages, despite fidelity being the ideal. However, the Vurhonga animators have encountered challenges to behavior change in the past and they are confident that they can make a difference in this area too. They say that people cite fidelity as the ideal yet not truly believing it to be possible. The animators intend to work to bring people’s actions into line with their creed. For married couples where one partner is already infected or for those who choose promiscuity, condoms are recommended. The MOH and other agencies working on HIV/AIDS prevention in the Chokwe district support the promotion of condoms. As long as their activities continue and there is no contradiction with the strategies based on abstinence and faithfulness, married discordant couples and those who choose promiscuity should be able to find condoms at the village health posts. Another potential challenge the staff may face in the villages is the association that some people make between HIV/AIDS and witchcraft. During the discussion with the MTE team, however, mothers, pastors, volunteers, and village leaders distinguished between HIV and witchcraft (“If it were witchcraft, the curse could be removed—but AIDS is different because people die no matter what…”). A concern expressed during the various MTE meetings was that HIV testing might create negative reactions and accusations of witchcraft in the community, or might not be accepted, in particular by men. The MTE discussed this issue with the various community members who were all open and demonstrated adequate understanding. Recommendations 1. The Vurhonga project should define behavior change indicators related to the abstinence and faithfulness messages and include them in the quarterly surveys to assess the effectiveness of the related behavior change activities 2. The Vurhonga project should go ahead with its plans to train one or more HIV counselors in each village. 3. The Vurhonga project should consider starting HIV voluntary counseling and testing in one or several villages. 4. The Vurhonga project should review the MOH messages and strategies to be sure the Vurhonga teachings do not contradict the MOH. World Relief Mozambique CSXV Midterm Evaluation Report 2001 12 C. Cross-cutting approaches (1) Community mobilization Achievements Vurhonga has developed an intensive, successful methodology for mobilizing cadres of volunteers in each village to reach every household in the project area with behavior change messages. These activities have been well received by the communities and their leaders as demonstrated by the enthusiastic support for project activities expressed to the MTE team during the various site visits. The animators and supervisors began contacting the village leaders and other community members at the very beginning of the project in October 1999. During these preliminary visits they explained the purpose of the Vurhonga project and began the careful process of selecting volunteers. Most of the targeted total of 2350 volunteer mothers were selected by the end of 1999. This very short start-up time was possible because most supervisors and animators were transferred from Vurhonga I with extensive experience. One of the first activities of the new volunteers was to make a census of all households, women of reproductive age, and children under five in the project area. This activity was abruptly interrupted by the February 2000 flood, which also destroyed all the data already collected at the time. The nascent CG structure and its animators turned out to be very effective during all the emergency relief operations (see detailed account of these contributions in the first annual report). By June 2000, all the CGs were formed and ready to receive their health education training (See section 2 below). In addition to the CG structure to provide health education to the mothers of children under five and their families, the Vurhonga project organizes the communities to create Village Health Committees (VHC) that monitor and make decisions about health issues and priorities (See Section D below). (2) Behavior Change Communication As mentioned above, the core of Vurhonga project is the intensive individual and village-level behavior change activities conducted by a large number of volunteers reaching all households in the project area. The total of 2350 volunteer mothers correspond to maximum of 10 households per volunteer in the entire project area. These volunteers, who live in and are selected by the communities, are carefully trained and supervised. Each one has her copies of educational pictures to use to teach mothers and their families. The MTE team reviewed the materials used so far by the volunteers (malaria, diarrhea, nutrition, and HIV/AIDS) and found their design and messages appropriate for the users and their audience, the majority of them being illiterate. The volunteers too said that they like the teaching materials because they carry vivid messages that are easily understood by the mothers. The pictures are fitted in hanging plastics sheets in a notebook for convenient use and durability. World Relief Mozambique CSXV Midterm Evaluation Report 2001 13 In addition to individual or small group education during household visits, the volunteers use songs, drama and dance that they develop themselves with the guidance of the animators. The mothers pick up on the songs when the volunteers sing them at gatherings. Although the education visits in each household usually concern all the family members, the Vurhonga project specifically reach the grandmothers in the village to ensure their full collaboration and support: grannies still have much influence on their daughters in-law who traditionally live with them. One staff member is assigned to this innovative component of the Vurhonga project and is able to provide training sessions to grannies in all villages on a monthly basis. The MTE team had two meetings with a dozen grannies who all showed much support to the project and good knowledge and understanding of the messages. Another approach to behavior change communication is to train the church leaders in each village in the various child survival interventions at the same time as the mother volunteers. The training of the church leaders varies from that of the volunteers by the different communication skills that are taught. Among the total of 450 church leaders identified in the project area, 60% have attended the training meetings offered to them by the animators on a monthly basis since January 2001. In April and in August 2001, about 50% of mothers who attended church in the last month reported that they had heard health messages (quarterly surveys). Both volunteers and animators commented that initially people were resistant to the teachings of Vurhonga but there is now a better acceptance as they become more familiar with new concepts and have developed trust in the animators and volunteers. Quote from socorrista, in the 25 September village: “I was impressed by what Vurhonga taught us about malaria and diarrhea. We didn’t used to know what caused them and so blamed other people. But now we know. If the child had malnutrition we thought it was an evil spirit—now we know it’s just a simple lack of food.” Also noted by many community members and health workers during the site visits is an increase in health post attendance. Mothers’ attitudes towards health have improved: “They have a better understanding of the importance of care. For instance, mothers tended not to go to health clinic for convulsions but things are changing now that people are more aware of possible treatment. They used to treat them with inhaled smoke or by hitting a spoon on a pan near the ears of the convulsing child but now they take their child to the health post.” The effects of the behavioral change activities at the village and district levels are currently measured through the quarterly surveys (See section 5 below). Recommendations 1. The Vurhonga project should consider conducting an in-depth evaluation of the knowledge and behavior of the volunteers, the mothers, and other community members at the end of the project. World Relief Mozambique CSXV Midterm Evaluation Report 2001 14 D. Partners strengthening The Ministry of Health The Ministry of Health is the primary institutional partner of the Vurhonga project. In particular, the health posts and the district health management team in Chokwe provide services that are critical for the success of the project. Achievements After the emergency flood relief operations, the MOH and WR collaborated with Oxfam and MedAir to strengthen health services in the Chokwe district. By the end of the year 2000, 18 socorristas had been trained and started working in newly built and equipped health posts. In April 2001, 95% of the population in the project area had access to a primary health post within 5 km (quarterly surveys). This higher accessibility to health services increased the number of consultations in the health posts and health centers of the rural district. The MOH statistics show that the average number of consultations in the period May/June/July3 increased from 16,398 in 2001 to 17,766 in 2001-- an 8% increase. In addition, the average number of consultations in the same period of the year increased from 1511 in 6 health posts in 2000 to reach 5533 in the 28 health posts functioning in 2001. This corresponds to a 30% increase in the total number of consultations in health posts and health centers in one year. Challenge The rapid creation of health posts may have resulted in the selection of socorristas with no adequate writing skills or understanding of their roles. One well-recognized problem, for instance, is the tendency of some socorristas to make injections. This is reinforced by the high demand for injections by the population. The rapid increase in the consumption of chloroquine at the district level also explains the relatively frequent stock outs of this drug in the health posts. The MOH has just started providing standard packages of essential drugs to the health centers and the health posts, but the order requested for the new health posts has not arrived yet. According to the MOH policy, the socorristas are entitled to receive the “Kit C” of essential drugs (first-aid kit including chloroquine), which is designed for 250 patients and corresponds to the annual average consumption of a health post. The kits are given free to the socorristas but will soon be available for a fee: socorristas will then have to request an increased financial contribution from the patients. The MOH District recognizes the need for strong supervision of the health centers and the health posts. Given the community-based nature of the health posts, the supervision of the socorristas 3 Comparison between 2000 and 2001 can only be done for this three-month period because of disruption of services from the flood before May 2000. World Relief Mozambique CSXV Midterm Evaluation Report 2001 15 is in principle primarily technical; all managerial aspects of the health posts services need to be addressed by the community through the VHCs. The lack of transport at the District level has prevented the officer in charge from conducting this supervision on a regular basis. This may change in the near future as a motorbike will become available to him and supervision guidelines already exist. Health posts and health centers still have mixed reputations in the project area. Especially the MOH health centers staffed with nurses seem to be overrun with patients. There are frequent complaints about the welcome and wait in the health centers, and nurses may also lack good interpersonal skills. This problem may be less important in the health posts because the number of consultations is lower and the socorristas are selected by their village and paid per patient. In any case, complaints from the population are good signs of demand for health services. The collaboration between the Vurhonga project and the MOH with respect to health information systems is discussed in section 4 below. Recommendation 1. The MOH District should make a supervisory visit to each village at least once a quarter to supervise the socorristas and deal with any issues raised by the VHC or the volunteers. The MOH and the Vurhonga project should define the scope, the tools, and the resources needed for this important task. The MOH should share their monthly report on supervision of socorristas with the Vurhonga project. Vurhonga should assist the MOH with maintenance of the motorbike, office supply, and the management and analysis of the supervision data. 2. The Vurhonga project should continue monitoring the quality of care in the health centers and the health posts, including the availability of drugs, welcome, waiting time and other criteria as identified by the community. The churches The Vurhonga project is strengthening church leaders’ knowledge and ability to disseminate appropriate health messages to the members of their congregations (see section 2 above). Achievements All pastors gave a very warm welcome to the MTE team when they met with them to discuss the project achievements. They recognized that at the beginning there had been problems with bringing the different pastors together, but now the majority understands the project and attend the health meetings organized by the animators. “Vurhonga helps us with the physical and the spiritual aspects of our lives. We used to just pay attention to dreams, but now Vurhonga teaches us to go to the hospital if you have a health problem. It helps us with the children—we know we don’t need traditional roots anymore. We had problems with malnutrition but now we know it is a problem of lack of food and we find the weight of the children increasing. We also get teachings to give. It is a blessing; it makes the work less for us as pastors because the teachings follow exactly what it says in the Bible.” World Relief Mozambique CSXV Midterm Evaluation Report 2001 16 The good collaboration of Vurhonga with most church leaders contributes to decreasing harmful practices such as admitting malaria or other patients in critical conditions to their homes rather than sending them to a health center for immediate and adequate treatment. Church leaders also appreciate the better collaboration among churches of different denominations that results from their common work on the Vurhonga project. Challenges A few pastors are still reluctant to collaborate with the Vurhonga project but all the pastors and volunteers that the MTE team met think that they will progressively understand, join the effort to improve community health, and abandon their harmful practices. A few pastors who attended training meetings with the Vurhonga project indicated that they had not been able to deliver the health messages in their churches because their church leader did not let them do so. Recommendation 1. The VHCs should communicate to the church leaders the importance of letting the pastors trained in health education share the messages with their congregation. 2. The village leaders should follow up with churches that don’t come to teachings or participate in council of churches to collaborate with the Vurhonga project on improving the community’s health. The Village Health Committees Achievements By February 2001, the Vurhonga project helped to set up VHCs in all but 4 of the 48 villages in the project area, and in April and August 2001, 54% and 58% VHCs met at least once during the previous two-month period. VHCs are actively involved in the management of the health posts and support of the socorristas. VHCs also provide a forum to discuss the health information collected by the volunteers and presented to them by the CG leader or the socorristas or nurse. A socorrista asked what advantage the VHC offers his village said: “It helps the community because we get updated.” Another: “If I have a problem, I can take it to the committee so I don’t struggle alone.” A pastor’s comment on the HIS reports during the VHC meeting: “The reports open our eyes—we see how our village and our country are developing bit by bit.” Several VHCs have already demonstrated their ability to solve problems in the community and manage issues related to the health posts and socorristas. The structure proposed for the VHC is appropriate and viable. VHCs are expected to play a critical role in sustaining the community health activities and achievements at the end of the project (see section D below). World Relief Mozambique CSXV Midterm Evaluation Report 2001 17 Challenge The VHC leaders received direction and orientation from the animators, but it appears that they do not always have appropriate understanding of health issues or of their roles in the VHC. Several chefs de saude4 noted they haven’t been trained to be the doctor or the police but to be the link between the health center and the community. “I am an eye of the community—I must look at their needs. I am the ear—I must listen; and I’m the mouth—I must do what I can.” Nevertheless, some chefs de saude are not well aware of their role and of what socorristas can and cannot do. The issue of the pressure on the socorristas to give injections, for instance, could be better addressed at the community level with the help of the chefs de saude. There may also be rivalry sometime between the health workers and the chef de saude that could be prevented by a careful definition and communication of their respective roles. Recommendation 1. The Vurhonga project and the MOH should prepare a written guideline on the creation and the operations of the VHCs that defines the role of the individual members. This guideline will be useful for the VHC members, the MOH socorristas supervisor, and other agencies interested in developing VHC outside of Chokwe. 2. The Vurhonga project should consider providing more training to village leaders and to the chefs de saude so they could be more involved and better serve the VHCs. 3. The animators should first give health lessons to VHC so they are informed in advance of what is being taught in the community. The chefs de saude could use training on the content of the health topics themselves as well as on leadership skills relevant to running the VHC meetings and managing the volunteers, and on the role of socorristas. E. Sustainability Achievements The Vurhonga project design at the village level based on CGs, VHCs, and health post or centers is promising in terms of sustainability. It appears possible to withdraw the animators and supervisors and have the volunteers and VHCs continue their activities. The project vehicles and motorbikes are now critical for the current operations to ensure that all the animators go back and forth to their villages every week, and that the supervisors can travel from village to village as needed, but the work of the volunteers does not directly depend on this logistical support. The question of what will happen at the end of the project was regularly addressed during the meetings that the MTE conducted at the village level. The village leaders, chefs de saude, and volunteers all said they would continue their current involvement in community health. Mothers in one group stated that one way they could encourage the volunteers to continue their involvement was to accept their instruction and follow their advice. The attitude and interest shown by the socorrista/nurse, the village leader and the Chef de Saude also motivate the 4 A Chef de Saude is the health representative of a village. World Relief Mozambique CSXV Midterm Evaluation Report 2001 18 volunteers to continue their work. One supervisor shared: “In Vurhonga I, the people saw the progress they had made and did not want to slide backwards, so they continued. The village leader and the Chef de Saude also had strong influenced on this. If all these factors are working together, the activities will not stop because of a lack of finances.” Challenges The 5 supervisors and 26 animators have gained valuable expertise and skills, but it is not clear yet that they will able to build on this experience at the end of the project. Several options have been identified during the various MTE meetings: establish themselves as a local NGO specializing in community health education and mobilization; become socorristas; and be transferred on another WR project. The VHCs are still relatively new and their meeting attendance and activities still need reinforcement. The health posts too were created after the emergency flood relief operations, and there may be a slowdown in the enthusiasm of the socorristas and the community after some time working on their own, when the drugs are not free anymore, or when the new facilities and equipment deteriorate. Several community leaders and members indicated that there should be a clear designation of one person to take the lead for community health at the village level. It is not clear so far who among the chef de saude, the CG leader, and the village leader would be that person and would get higher-level training from the Vurhonga project to ensure that the activities continue. No matter how well the project activities are carried out at the community level, the District MOH will need to follow up on this program, and at least through effective supervision of the socorristas and nurses. The project activities are currently under the responsibility of one competent and motivated person, and this at the demand of WR. Another District MOH staff member is formally in charge of “community health” but is currently involved in many activities other than those of the project (immunization, malaria, tuberculose, water and sanitation, sanitary inspection of restaurants and shops, etc). The staff of Vurhonga II visited the sites of Vurhonga I in June this year to assess the level of activities of the volunteers and give them some support. Overall, the animators were satisfied by the continuation of the activities of the volunteers, but noted a certain decrease in the CG attendance. Recommendations 1. The Vurhonga project should prepare a detailed phase-out plan that includes:  Activities with the MOH at the District level to develop: - The policy of explicitly giving over community health activities when health posts or district staff member are transferred - The capacity to effectively supervise community health activities - The community-based health information system - The leadership of the person in-charge of community health. That person should have all the information and skills needed, and be progressively given the authority and responsibility to apply them. World Relief Mozambique CSXV Midterm Evaluation Report 2001 19  Activities to prepare for the future employment of the animators and supervisors  Activities at the Provincial and National level including advocacy and information dissemination (documentation; attendance to meetings; exchange visits with other community health and child survival projects)  Activities to empower the CG leaders and the chefs de saude. During the refresher training planned for the end of the project, for instance, only the CG leaders could be trained by the animators so that they can learn how to transfer their skills to the other volunteers.  Activites to prepare a complete documentation of the Vurhonga project by 2003 2. The Vurhonga II project should continue monitoring the volunteer activities in the Vurhonga I project to document and learn about the sustainability of the CG approach. In particular, the animators should continue collecting mortality and other indicator data perhaps once a year. 3. WR should plan on including the Vurhonga I area and achievements in the final evaluation of Vurhonga II in 2003. World Relief Mozambique CSXV Midterm Evaluation Report 2001 20 III. Program management A. Planning Building on the successful achievements of Vurhonga I, WR prepared its proposal with close participation of the District MOH and the field staff (director, supervisors, and animators). A one-week meeting was organized to launch the Vurhonga II project in October 1999. The DIP was prepared by the project director, WR headquarters staff, and an external consultant but its sound and well-articulated strategies clearly reflect this collaboration with the MOH and project field staff. In addition to the socorristas and nurses, the MOH District Director and one officer are directly involved in the Vurhonga project activites. Their participation in the MTE demonstrated their good understanding of the project, but it appeared that they do not have a well-defined reference document for the project available in Portuguese or Shangaan (the DIP is not translated). The project field staff conducted the KPC survey and the subsequent quarterly surveys. These results are largely discussed during the weekly meetings of the project director and the supervisors and animators. They all have a clear understanding of the objectives and strategies of the project. The results of the surveys are also presented and discussed in the VHC and CG meetings. As the MTE team met with community members in the villages (mothers, volunteers, grannies, village and church leaders) and asked about their knowledge of the Vurhonga project, the responses demonstrated a clear understanding of the objectives of the project in terms of reduction of morbidity and mortality as well as in terms of the strategies of community health education and participation. They all were clearly aware that the project would end in about two years and that they would be expected to continue these activities on their own. As mentioned before, the work plan in the DIP was prepared in February 2001, that is, about fifteen months after the beginning of the project and only about seven months before the MTE. Thus at this point there is no delay in the activities as planned in the DIP. Recommendation 1. The Vurhonga project staff should prepare and translate a summary of the key technical elements of the DIP including of the child survival interventions, the capacity building and partnership approaches, the project objectives and indicators, and the monitoring and evaluation plan. This document should be shared with the project and MOH staff, and possibly with other partners in Chokwe and at the Provincial and National level. World Relief Mozambique CSXV Midterm Evaluation Report 2001 21 B. HIS The Vurhonga project has developed an elaborate health information system made of three components: quarterly surveys; monthly reports by the animators; and a community based health information system. In addition, the MOH collects services statistics that are relevant to the monitoring and evaluation of the Vurhonga project. The quarterly surveys Starting in September 2000, the Vurhonga project has conducted a series of quarterly surveys similar to that of the KPC surveys with respect to the questions but different by the sampling scheme. Each animator surveys all the households of one CG chosen at random in another animator’s area. This represents about one eighth of the total population of the project area, and between 100 and 150 women of reproductive age per animators. The quarterly survey data are tabulated manually by each animator and then compiled and discussed during their weekly meetings with the project director and the supervisors. Since each animator’s area is most likely homogeneous, the successive quarterly surveys are considered to be representative of their area and to allow for monitoring their performance and for cross-comparison among them. The total sample is considered representative of the entire population of the project area and the successive estimates are used to monitor progress towards project objectives. The confidence intervals on the estimates are not routinely calculated but would be far smaller than those obtained with the KPC survey since the sample size is much larger. In the May 2001 quarterly survey, for instance, the number of mothers of children under two was 944 as opposed to 300 in the October 1999 baseline KPC. Selected results for the project area are plotted on graphs displayed on the wall of the Vurhonga meeting room. The results are also informally shared with the MOH, and the animators report the results to the VHCs. The three graphs on the malaria, diarrhea, and nutrition progress towards objectives presented in Appendices F, G and H are based on these quarterly survey data. The monthly animator reports Starting in March 2000, the animators have been reporting on a monthly basis the number of deaths of children under five in all the households in their area. These data are collected during the CG meetings but also by actively visiting and questioning the volunteers who are absent. For each death under five, the age group (< 1, and 1-4) and the cause is also recorded. Although the number of deaths and the age group at death are probably fairly reliable, the establishment of the causes of death lacks standardized criteria. Each death is discussed during the CG meeting. These data will now be included in the community-based HIS (see below), but the high coverage and reliability achieved by the animators may be difficult to replicate. The number of births is not recorded nor is migration (the census made at the beginning of the project is not updated). Appendix I presents the cause-specific number of deaths under five as reported by the Vurhonga animators from March 2000 to July 2001, a 17-month period of observation (the data for the months of December is missing). The dates of key programmatic events are represented on the graph for easy reference. Five causes of death are considered: malaria, diarrhea, pneumonia, malnutrition, and all the other causes. The graph shows two peaks in May 2000 and 2001, which World Relief Mozambique CSXV Midterm Evaluation Report 2001 22 corresponds to the rainy seasons with higher intensity of malaria transmission. During the period of March through August, the number of deaths in each calendar month is lower in 2001 than in 2000. This decrease is largely due to a lower number of malaria and secondarily of diarrhea deaths, but this is somewhat compensated by an increase in the number of deaths assigned to malnutrition and to other causes. With the reasonable assumption of no decrease in the number of births and no net migration out of the project area, the decrease from 253 to 154 deaths under five between the periods of March through August 2000 and 2001 (a 40% decrease) may be due to the alleviation of the hardship conditions due to the flood, to the effectiveness of the Vurhonga project, or both. An unintentional shift in the assignment of the causes of deaths from malaria and diarrhea to malnutrition and others causes cannot be excluded. The sharp decrease in malaria deaths between May and June 2000 corresponds to the beginning of the malaria health education of the volunteers and the mothers and to the reestablishment of the pre-flood level of attendance in the MOH health centers (MOH data). However, it occurred several months earlier than the massive distribution of bed nets in September 2000 and than the beginning of the work of the 18 new socorristas in October 2000 (MOH data). The community-based health information system The community-based HIS as described in the DIP and based on that of Vurhonga I has just begun as the volunteers have learned the various child survival interventions and how to report on specific health conditions. The information is reported orally by the volunteers during their monthly CG meeting and transcribed by the CG leader. This systematic reporting and sharing of information during the meeting is an opportunity to double-check the information provided by each volunteer and to discuss the implications and recommended actions for the community and the volunteers. The CG leader then sends the written report to the socorrista or nurse who can review it and make the appropriate actions. As mentioned earlier, there may be some difficulty at this stage of the process as many CG leaders are illiterate. In addition, some health workers may look down on the data from the CG leaders and ignore them. Aware of this weakness, the Vurhonga project is considering providing additional training to the CG leaders to improve their reporting capacity, and to the health workers to ensure that they fully understand the value of the CG reports. The socorristas and nurse then compile the reports from the different CG in their zone, and present them for discussion at the monthly VHC meeting. A village leader noted: “The socorrista or the chef de saude gives a written report of what’s been done in the community. It is helpful to know better what is going on in the community. The information includes deaths, diseases, deliveries at home or at the hospital,…” The summary reports by the nurse or socorrista are also sent to the District MOH officer who can check and detect discrepancies among the various health posts. No district-level compilation is done yet, but this will begin as soon as the coverage of the C-HIS improves. The District MOH officer commented that the information from the Vurhonga community-based HIS is very useful and that they want to keep collecting information on morbidity and mortality in the community, World Relief Mozambique CSXV Midterm Evaluation Report 2001 23 not just in the hospital and health centers. He did not think managing and analyzing these data would be a problem for the District even when it will come from all the 28 health posts and 15 health centers, but mentioned that some help with paper and photocopies from time to time would be useful while Vurhonga is still around. The MOH services statistics The MOH routinely collects various services statistics that can be used for project management and evaluation. The MTE team received tables with the number of consultations by health centers and health posts that demonstrate the increase in the frequentation of health services in the district over the last 17 months (see section D.3. —MOH). If data on frequentation by age (under five) and cause (malaria, diarrhea, pneumonia, etc) exist and are available for analysis, they could show better the relationship with the project activities. Data on the MCH services (FP, deliveries, ANC, and EPI) and on STIs exist that would be particularly useful to monitor the effect of the next interventions of the Vurhonga project (STI, maternal care, birth spacing). Recommendations 1. The Vurhonga project should work with the District MOH on managing and analyzing health services statistics. As these and the Vurhonga project data accumulate, it becomes necessary to develop a structured database that allows access to data a few years backward and to link data from different sources. This will support the management of the project and health services in the District, and will allow for a complete documentation of the project by 2003. 2. The Vurhonga project should monitor carefully the development of the promising community-based health information system to assess what can be sustained at the end of the project. The animators and the MOH staff should then train and supervise the socorristas and chefs de grupo with respect to recording and reporting the community HIS data. C. Staff development and management The Vurhonga project has an effective system to develop the capacity of its staff to implement the project while acquiring technical and leadership skills that will last beyond the project. First, the project organizes regular training camps for the supervisors and animators that address the technical aspects of the child survival interventions as well as the leadership skills necessary to achieve effective behavior change at the community level. The animators then provide training, support and monitoring to the volunteers twice a month during the CG meetings. The animators also accompany volunteers on their home visits to assess the accuracy of the messages they deliver and the appropriateness of their teaching skills. Peer approaches are used whereby weaker volunteers are paired with stronger ones during their home visits--this has proved a very effective way of supporting volunteers who appreciate this approach. Finally, the animators provide the same training instructions for the volunteers to pastors, but with a special emphasis on teaching methods appropriate to church meetings. Given the large number of animators and volunteers, the project has developed an efficient supervision system. The five supervisors submit their weekly plan of action to the project director, and these are discussed with the animators during their regular meetings held on World Relief Mozambique CSXV Midterm Evaluation Report 2001 24 Fridays. Each supervisor follows up on three animators per week to assess their teaching skills and performance. They use a checklist that details areas requiring reinforcement. The supervisors also track volunteer attendance records, which directly reflect the animator’s performance. The well-attended Friday meetings appear critical to the effective exchange of information between the project director and the volunteers and other actors at the community level: the excellent knowledge of the health issues in their village makes it possible for them to collect and disseminate information very easily. One issue discussed during Friday meeting with the MTE team, for instance, was the training of TBAs at the Chokwe hospital: a few questions to the animators quickly provided the information needed to plan this training for all the old and new TBAs in the project area. Conversely, the animators bring information from these meetings to the volunteers in the CG or to the VHCs (quarterly survey data; new teachings or instructions; etc). The Vurhonga I and II projects have developed a strong and committed human resources base and the staff retention is very high. Personnel policies and procedures were reviewed during the final evaluation of Vurhonga I and are in line with standards of other local NGOs. Each year, the project staff at all levels undertakes a participatory and anonymous evaluation whereby everyone rates each other. The outcomes are reviewed instantly. This exercise promotes motivation and high performance because everyone knows that he or she will be evaluated by everyone else, and those who gets low score attempt to improve their attitude and work style. Recommendation 1. The Vurhonga project should prepare a training of trainers (animators) manual that include all the technical educational materials used by the volunteers and all the training, managerial and leadership skills that the Vurhonga project teaches them. D. Financial and logistic management Financial management occurs at the field level in Chokwe, even though the financial reports are maintained in the WR office in Maputo. The WR bookkeeper in Chokwe prepares and submits a monthly budget request to the office in Maputo who transfers the funds to the general WR account in Chokwe. The system is working well. As the activities at the community-level do not require financial inputs, it is not necessary to plan for financial management sustainability. The project has 19 motorbikes and 6 vehicles, two of which were purchased for the current project. The motorbikes are used by the animators, and five vehicles are used by the supervisors. The staff reviews the use and fuel consumption of the project vehicles during their weekly meeting. The availability and good use of these vehicles contributes to the effectiveness of the project. Travel and transportation of the animators and supervisors has been very efficient and the supervision is done on a timely basis. Occasionally, the project also supports the MOH with transport to conduct maternal and child health activities such as EPI and growth monitoring in the villages. World Relief Mozambique CSXV Midterm Evaluation Report 2001 25 E. Technical and administrative support According to the Project Director, Vurhonga II has received more technical and administrative support from PVO headquarters than Vurhonga I. The project received technical assistance from the WR Africa Regional Office on HIV/AIDS programs and AIDS home-based care in preparation for the training of volunteers and AIDS counselors. The project receives continued technical support from the Director of International Health Programs and the CS specialist who have already made a total of 5 trips to Chokwe. WR Maputo staff visited the Vurhonga project in June 2001, and the project director goes to the Maputo office on a quarterly basis to meet with the WR managers, share reports, and discuss administrative issues. The Vurhonga project director participated in several local workshops and conferences, and the meetings of the Global Health Council and Christian Connections for International Health in Washington DC in June 2001. World Relief Mozambique CSXV Midterm Evaluation Report 2001 26 IV. Conclusions Following on Vurhonga I (1995-1999), the implementation of the Vurhonga II project began rapidly and effectively despite the disruption caused by the largest flood in Mozambique for decades, and that covered the entire project area and facilities. This successful start in the face of natural disaster is largely due to the very competent and dedicated WR staff transferred from Vurhonga I. Several emergency flood relief agencies also provided assistance to the beneficiary population of the project area that complemented very well that of the Vurhonga project. In particular, education and hygiene (latrines; wells) interventions helped prevent cases of diarrheal diseases and cholera outbreaks; education and distribution of ITNs helped controlling malaria epidemics; and rehabilitation of health posts and training of socorristas ensured access to primary health care services within 5 km for 95% of the population just a few months after the flood. The Vurhonga project is well designed and implemented. The CG structure enabling 2350 volunteer mothers to provide health education through home visits at least every two weeks to all the households in the project area is very well accepted by the communities. It appears to have effectively changed the behaviors targeted by the project and related to malaria, diarrheal diseases, and nutrition. To ensure the participation of the community in the work of the CG and that of the socorristas in the health posts, the Vurhonga project has been able to mobilize most villages in the project areas to create Village Health Committees. These VHCs meet on a regular basis to review the health issues in their communities, and in particular the information provided by the monthly reports from the CGs. The planning and management of the Vurhonga project is rigorous. This is reflected by the timely and complete implementation of activities clearly laid out in the DIP, and by the continuous collection of data to monitor progress toward the objectives of the project. With a six-year perspective in two neighboring areas (Vurhonga I and II - 1995-2001), the CG approach is becoming a model for implementation of child survival interventions at the community level that WR has already begun replicating in other countries. The MTE team formulated a series of specific recommendations to the Vurhonga staff and its partners. These include 1) strengthening the VHC, and in particular the village leaders, the chef de saude, and the CG leaders; and 2) strengthening the MOH capacity to supervise the community health activities of the CG, the socorristas, and the VHC. Given the rapid progress towards the project objectives of several interventions, the MTE team also recommended to slightly raise selected targets (early treatment of fever; ORT use; extra food during following diarrhea; counseling and enriched porridge to malnourished children). Finally, WR should prepare for a formal assessment of the impact and sustainability of the approach by the end of the Vurhonga II project in 2003. This requires beginning a systematic documentation of activities and management of data for this evaluative purpose in addition to the current one for project management and monitoring. The outputs of these additional efforts will be valuable to WR and other agencies involved in community health or child survival programs in Mozambique and many other countries. World Relief Mozambique CSXV Midterm Evaluation Report 2001 27 V. Results highlights The Vurhonga Community Health Information Systems The Vurhonga II Child Survival project (1999-2003) covers a total population of 130,000 in the rural area of Chokwe District in southern Mozambique. The two main strategies are to (1) help 48 communities create 220 Care Groups (CG) of 10-15 volunteer mothers working as behavior change agents through regular visits of about 10 households each; and (2) help the MOH District expand its network of health posts so that 95% of the population lives within 5 km of a trained health care provider (nurse or first-aid worker--socorrista). The Vurhonga project promotes malaria, diarrheal diseases, nutrition, respiratory infections, STI/HIV/AIDS, child spacing, and maternal and neonatal care interventions. The Vurhonga project has developed an elaborate health information system that comprise quarterly surveys, monthly reports by animators, and a community-based health information system (C-HIS). The MOH also collects data on health services. At the end of the Vurhonga project in 2003, only the C-HIS will remain incorporated in the MOH system. The Vurhonga C-HIS was designed to register births and deaths and to track children and mothers of reproductive age who do not receive specific health services such as immunization, antenatal and delivery care, and nutritional counseling. The data are primarily collected and used by the volunteer mothers. During the biweekly meetings of the CG, the volunteer mothers report the data orally on the basis of their knowledge of the families they are in charge of. This systematic group reporting is an opportunity to double-check the information provided by each volunteer and to discuss the recommended actions for each case. It takes about 20 minutes per CG meeting to let all the volunteers report their information, which is transcribed by the CG leader. Every month, the CG leader sends a written report of the data collected from the volunteer mothers to the local health post. The nurse or socorrista of the health post can review this report and take appropriate action. S/he then compiles the reports from the different CGs in their zone. This report is discussed at the monthly Village Health Committee meeting. The nurse or socorrista also sends this summary report to the District officer who can detect discrepancies among the health posts. The Vurhonga C-HIS is not connected to a planning and supervision system yet, but this could be done at the District MOH level as soon as the completeness of the reporting makes it possible. The Vurhonga C-HIS is very appropriate to the population of the Chokwe District because it does not rely on literacy skills of the volunteers in a region where the female literacy is only 25%. It has an excellent coverage because there are volunteer mothers visiting all households in the project area who meet on a biweekly basis to report, reflect and act on results. World Relief Mozambique CSXV Midterm Evaluation Report 2001 28 VI. Action Plan MTE RECOMMENDATIONS ACTION PLAN – September’01 Recommendations Responsibility Target Date A Malaria 1. The Vurhonga project should regularly review the MOH statistics tracking the availability of chloroquine at health posts. Proj. director With each HIS 2. The Vurhonga animators and volunteers should reinforce messages related to year-round use of ITNs, and in particular for children under five. Animators and volunteers Jan’02 3. The Vurhonga project should follow up with the MOH, UNICEF, and PSI to encourage them to soon make ITNs and re-treatment available at low cost in the villages. Proj. director 27 Sep 01 4. The Vurhonga project should encourage the Village Health Committees to address and develop solutions to the issue of referral in their community. Animators Dec.’01 5. The Vurhonga project should consider assessing compliance with malaria treatment received in health posts in one of the HIS surveys. Animators Dec.’01 B Diarrhea 1. The Vurhonga project should continue monitoring latrine coverage and use in the community; volunteers (and village leaders, as necessary) should follow up with families who do not yet have latrines. Animators and VHC’s Monthly 2. The Vurhonga project should continue monitoring the weaning babies with diarrhea to ensure that this harmful practice decreases. Animators VHC’s Volunteers Ongoing C Nutrition 1. Given the risk of food shortage, the Vurhonga project should define a nutritional surveillance indicator and adopt the related threshold to determine when the malnutrition levels require supplemental feeding. The project should also investigate options for supplemental food via the World Food Program in case a rapid response is needed. N/A as UNICEF and WFP are already involved in doing it. 1. The Vurhonga project should consider encouraging mothers to make dried greens during the rainy season when they are plentiful—a traditional practice that was discontinued years ago, and promoting marula nut trees in Chokwe district for long-term nutritional benefit. Supervisors Animators Volunteers Jan.’02 World Relief Mozambique CSXV Midterm Evaluation Report 2001 29 D MOH Strengthening 1. a)The MOH District should make a supervisory visit to each village at least once a quarter to supervise the socorristas and chefs de saude and deal with any issues from the VHC or the volunteers. The MOH and the Vurhonga project should define the scope, the tools, and the resources needed for this important task. The MOH should share their monthly report on supervision of socorristas with the Vurhonga project. b)The Vurhonga should assist the MOH with maintenance of the motorbike, office supply, and the c)management and analysis of the supervision data. a)MOH H/P supervisor b)W/R mechanic c)Proj. director a)Quarterly b)As needed c)Nov.’01 2. a)The Vurhonga project should continue monitoring the quality of care in the health centers and the health posts, including the availability of drugs, welcome, waiting time and b)other criteria as identified by the community. a)MOH H/P Supervisor b)Animators a)Quarterly b)Quarterly with HIS E Church Strengthening 1. The VHCs should communicate to the church leaders the importance of letting the pastors trained in health education share the messages with their congregation. Supervisors and Animators Oct.’01 2. The village leaders should follow up with churches that don’t come to teachings or participate in council of churches to collaborate with the Vurhonga project on improving the community’s health. Village leaders Ongoing F VHC Strengthening 1. The Vurhonga project and the MOH should prepare a written guideline on the creation and the operations of the VHCs that defines the role of the individual members. This guideline will be useful for the VHC members, the MOH socorristas supervisor, and other agencies interested in developing VHC outside of Chokwe. Proj. director Dec.’01 2. The Vurhonga project should consider providing more training to village leaders and to the chefs de saude so they could be more involved and better serve the VHCs. Proj. director Supervisoras Animadoras April’02 2. The animators should first give health lessons to VHC so they are informed in advance of what is being taught in the community. The chefs de saude could use training on the content of the health topics themselves as well as on leadership skills relevant to running the VHC meetings and managing the volunteers, on the role of socorristas. Animators Monthly World Relief Mozambique CSXV Midterm Evaluation Report 2001 30 G MOH Service Statistics (Under HIS) 1. The Vurhonga project should work with the MOH District on managing and analyzing health services statistics. As these and the Vurhonga project data accumulate, it becomes necessary to develop a structured database that allows access to data a few years backward and to link data from different sources. This will support the management of the project and health services in the District, and will allow for a complete documentation of the project by 2003. 2. The Vurhonga project should monitor carefully the development of the promising community-based health information system to assess what can be sustained at the end of the project. The animators and the MOH staff should then train and supervise the socorristas and chefs de grupo with respect to recording and reporting the community HIS data. Proj. director Animators Dec.’01 Ongoing H Sustainability 1.The Vurhonga project should prepare a detailed phase￾out plan that includes: Proj. director  Activities with the MOH at the District level to develop: - The policy of explicitly giving over community health activities when health posts or district staff member are transferred MOH District director As needed - The capacity to effectively supervise community health activities See D-1 - The community-based health information system See G-2 - The leadership of the person in-charge of community health. That person should have all the information and skills needed, and be progressively given the authority and responsibility to apply them. MOH H/P supervisor Dec.’02  Activities to prepare for the future employment of the animators and supervisors Administrator Supervisors Dec.’02  Activities at the Provincial and National level including advocacy and information dissemination (documentation; attendance to meetings; exchange visits with other community health and child survival projects) Proj. director Admin. assistant Ongoing  Activities to empower the CG leaders and the chefs de saude. During the refresher training planned for the end of the project, for instance, only the CG leaders could be trained by the animators so that they can learn how to transfer their skills to the other volunteers. Supervisors Animators April ’02 – July ‘03  Activities to prepare a complete documentation of the Vurhonga project by 2003, Proj. director Ongoing till end of project World Relief Mozambique CSXV Midterm Evaluation Report 2001 31 2. The Vurhonga 2 project should continue monitoring the volunteer activities in the Vurhonga 1 project to document and learn from the sustainability of the CG approach. In particular, the animators should continue collecting mortality and other indicator data maybe once a year. Supervisors and Animators Yearly 3. WR should plan on including the Vurhonga 1 area and achievements in the final evaluation of the Vurhonga 2 in 2003. Evaluation team Aug.’03 I Program Management (Planning) 1. The Vurhonga project staff should prepare and translate a summary of the key technical elements of the DIP including of the child survival interventions, the capacity building and partnership approaches, the project objectifs and indicators, and the monitoring and evaluation plan. This document should be shared with the project and MOH staff, and possibly with other partners in Chokwe and at the Provincial and National level. Proj. director Translator Febr.’02 J Staff Development and Management 2. The Vurhonga project should prepare a training of trainers (animators) manual that include all the technical educational materials used by the volunteers and all the training, managerial and leadership skills that the Vurhonga project teaches them. Admin. assistant Dec.’02 World Relief Mozambique CSXV Midterm Evaluation Report 2001 Attachments VII. ATTACHMENTS A. Baseline Information from DIP B. MTE Team Members C. Assessment Methodology D. MTE Schedule E. Project Implementation Timeline F. CHART: Progress towards objectives: Malaria G. CHART: Progress towards objectives: Diarrhea H. CHART: Progress towards objectives: Nutrition I. CHART: Cause specific deaths World Relief Mozambique CSXV Midterm Evaluation Report 2001 A1 A. Baseline Information from the DIP 1. FIELD PROGRAM SUMMARY Potential Beneficiaries There are 53,418 potential beneficiaries for Vurhonga II, including 31,338 women age 15-49 years old and 22,080 children under five years of age. Table 1: Vurhonga II Intervention Mix Component Child Survival Intervention: Level of Effort Nutrition General Nutrition: 30% CDD Control of Diarrheal Disease 10% Malaria and Pneumonia Malaria Control 20% Pneumonia Case Management 10% Reproductive Health Maternal and Newborn Care 10% Child Spacing 10% STI/HIV/AIDS Prevention 10% 2. PROGRAM GOALS AND OBJECTIVES Table 19: Vurhonga II Objectives, Indicators and Major Activities Objectives Indicators Measurement Methods Major Planned Activities Nutrition: 1a. Increase from 75% to 85% the number of children 0-35 months weighed at least once during last 3 months. 1a. Children 0-23 months with documented weighing at least once in last 3 months (by card). 1. Project quarterly Monitoring and Evaluation (M&E) Plan. (See description in monitoring and evaluation section.) 2. KPC surveys (conducted three times during project- at the baseline, and midterm & final evaluations.) 1. Volunteers trained to assist MOH staff at monthly EPI/GMC sessions, mobilize community for attendance. 2. Train MOH health center and health post staff to do nutritional counseling. 3. Volunteers trained to give nutritional counseling on home visits and at GMC sessions, especially to malnourished children. 4. Mothers trained by volunteers in how to make and why to give 1b. Increase from 22% to 80% the number of malnourished children's mothers who received nutr. counseling 1b. Mothers of malnourished children 0-23 months who received nutritional counseling by volunteers, animators, or MOH personnel. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A2 Table 19: Vurhonga II Objectives, Indicators and Major Activities Objectives Indicators Measurement Methods Major Planned Activities 1c. Increase from 50% to 70% the number of malnourished children who received nutritious foods/enriched porridge 1c. Mothers of malnourished children 0-23 months who receive nutrition counseling and stated they give nutritious weaning foods/ enriched porridge to their child at least once per day. enriched porridge. 2. At least 70% of children who completed the HEARTH program achieve and sustain adequate or catch-up growth per month during at least 2 months after HEARTH. 2. Malnourished children 0-3 years old who enter 12 day Hearth program and complete 9-12 days. 2b. Children who complete Hearth program and achieve adequate or catch-up growth. 2c. Children who continue to gain weight at a rate equal to the international standard for their age at 6 months and 1 year following the Hearth. 2a. Hearth program registers data, constantly updated. 2b. Hearth program children follow-up weights taken 6 and 12 months after completing program. 1. Volunteers trained in Hearth methodology conduct 2 Hearth cycles per year in the first 2 years, repeat in Years 3 and 4 as needed. 2. Maintain high coverage in bimonthly GMC sessions. 3. Animators maintain Hearth registers until child turns 5. Maternal Care: 3. Increase from 45% to 70% the number of mothers who eat the same amount or more food during pregnancy. 3. Pregnant mothers who say they ate the same amount or more food now as they did before their pregnancy. (For KPC only: the mothers of children 0-23 months who report that they ate the same amount or more food during their last pregnancy than they did when they were not pregnant.) 1. Project quarterly M&E. 2. KPC surveys 1. Train volunteers and mothers about the importance of eating more during pregnancy. Malaria and Pneumonia: 4a. Increase from 28% to 75% the number of children treated within 24h for fever (suspected malaria) at any health facility. 4. Children 0-23 months old who had fever (suspected malaria) in the past 2 weeks that were treated within 24 hours at appropriate health facility. 1. Project quarterly M&E. 2. KPC surveys . 1. Volunteers and mothers trained in: a) recognition of fever (and other signs) of malaria and b) the importance of rapid (within 24 hours) treatment. 4b. Increase from 0.3% tot 70% the proportion of children under 5 who use insecticide treated nets year round. Percentage of children under two who slept under an insecticide treated net the previous night. 1. Project quarterly M&E. 2. KPC surveys 1. Distribution of ITN bednets 2. Retreatment of bednets World Relief Mozambique CSXV Midterm Evaluation Report 2001 A3 Table 19: Vurhonga II Objectives, Indicators and Major Activities Objectives Indicators Measurement Methods Major Planned Activities 4c. Increase from 2% to 50% the number of children treated within 24h for rapid, difficult breathing at appropriate health facility 4b. Children 0-23 months old who had cough/ rapid, difficult breathing, with or without fever (suspected pneumonia) in the past 2 weeks that were treated within 24 hours at appropriate health facility. 1. Project quarterly M&E. 2. KPC surveys 1. Volunteers and mothers trained in: a) recognition of cough and fast, difficult breathing as signs of pneumonia and b) the importance of rapid (within 24 hours) treatment. 4d. Increase to 100% the health facility-based providers who have received continuing education in malaria and pneumonia protocols during the project. (MOH Capacity Building) 4c. Number of Health facility-based providers who received continuing education in malaria and pneumonia protocols by the MTE of the project. 1. Project training records. 2. MTE interviews with MOH staff. 1. Project Director (PD) trained in IMCI or equivalent. 2. PD gives in-service training to Chokwe District MOH health providers in IMCI / malaria and pneumonia protocols. (assuming MOH Chokwe participates in IMCI protocols. Diarrhea Case Management 5a. Increase from 53% to 80% the proportion of children with diarrhea treated with ORT by mothers/volunteers. 5a. Children 0-23 months with diarrhea in the past 2 weeks who received ORT. 1. Project quarterly M&E. 2. KPC surveys 1. Volunteers train mothers to give ORT to children during diarrhea. 2. Volunteers train mothers to give extra food to a child recovering from diarrhea. 3. Assure ORS packet supply. 5b. Increase from 19% to 65% the proportion of mothers who give extra food to children for 2 weeks following diarrhea. 5b. Children 0-23 months who received extra food after their last diarrheal episode. Reproductive Health: 6a. Increase from 0.3% to 50% the number of mothers who know 3 ways to prevent transmission of STI's including HIV/AIDS. 6a. Mothers of children 0-23 months who know 3 ways to prevent transmission of STIs including HIV/AIDS. 1. Project quarterly M&E. 2. KPC surveys 3. District MOH statistics. 1. Couples trained in STI transmission and prevention. 2. Couples trained in a) how to recognize at least 3 common STI symptoms, b) importance of rapid treatment. 3. Couples trained in how STIs increase risk of HIV transmission and infertility. 6b. Increase from 1.6% to 50% the number of women and partners who recognize at least 3 common symptoms of STI's. 6b. Mothers of children 0-23 months who know at least three common symptoms of STIs other than HIV/AIDS. 6c. Increase from 0.3% to 50% the number of women and partners who recognize that STI's increase risk of HIV/infertility. 6c. Mothers of children 0-23 months who know that STIs increase the risk of HIV transmission and infertility. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A4 Table 19: Vurhonga II Objectives, Indicators and Major Activities Objectives Indicators Measurement Methods Major Planned Activities 7. Increase from 65% to 70% the number of mothers who deliver child by trained provider. 7. Mothers of children 0-23 months who had their youngest child delivered by a trained health worker (includes trained TBA.) 1. Project quarterly M&E. 2. KPC surveys 3. District MOH statistics. 4. Yearly client satisfaction surveys. 1. Workshops with MOH staff on QA (with TA from SEATS) to improve client perception of treatment at health centers. 8. Increase from 7% to 20% the number of women who are using a modern method of birth spacing. 8. Mothers of children 0-23 months who are not pregnant and who are using a modern method of birth spacing. 1. Project quarterly M&E. 2. KPC surveys 3. District MOH statistics. 1. Couples trained in benefits of spacing births and methods available. 2. Train Socorristas as CBD to improve access 3. Village leaders and pastors address men in community about Child Spacing. SUSTAINIABILITY & CAPACITY BUILDING: 1. MOH / District Level 1a. All beneficiaries will have access to primary health care centers. At least 95% of beneficiaries in the project area will live within 5 km of a health facility staffed with trained personnel and equipped with essential supplies. Project, VHC and MOH records. 1. Select and train Socorristas for un￾staffed health posts. 2. Monitor health posts through VHCs. 3. Assist community in the construction and equipping of health posts with Oxfam funding 1b. MOH personnel at district level will creatively address shortages of materials to provide continuity of services. Chloroquine will not be out of stock for more than three days per month in at least 90% of health posts. Project, VHC and MOH records and meetings. 1. As shortages arise, catalyze a joint response between MOH personnel and community to solve the immediate problem without WR resources. 2. Establish contingency plans for repeated problems. 1c. All Socorristas will receive regular supervision. 75% of Socorristas will have received a supervisory visit from Chokwe District staff or Vurhonga project director within the previous quarter. Supervision reports Mtgs with Chokwe DHMT 1. A motorbike will be made available to the MOH Socorrista supervisor. 2. A monthly report on health posts visited and their current status will be completed. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A5 Table 19: Vurhonga II Objectives, Indicators and Major Activities Objectives Indicators Measurement Methods Major Planned Activities 1d. Community groups will maintain linkages with the MOH. 70% of VHCs will have met at once during the last two months. Health Post staff will attend at least 80% of the meetings of VHCs M&E, MTE, Project records 1. Socorrista supervisor monitors VHC meetings and participation of health post staff. 1e. Community members will expect and demand provision of basic services. 1. 50% of beneficiaries will report that they are satisfied with their last visit to an MOH facility as measured by a satisfaction index M&E, MTE, Project records 1. Churches, community groups, or VHCs will learn to approach the MOH concerning disrupted or poor quality of services. 2. Community–Level 2a. VHCs will monitor, plan and evaluate maternal and child health for their areas. 1. 95% of villages will have established VHCs by EOP. 2. 70% of VHCs will meet at least once in the last two months. Staff reports, VHC meeting records, M&E. 1. Animators facilitate formation of the VHCs/ 2. Animators orient and train the VHCs in their responsibilities and use of meeting agenda guide (MAG) 3. Pair strong CGs and VHCs with weaker ones for instruction. 2b. Care Groups are thoroughly trained in CS intervention. 100% of Care Groups will “graduate” during the fourth year of project. Project records See description of Care Group “graduation” in project design section. 2c. Churches in the project area will actively promote behavioral changes specified in the interventions. 1. The monthly attendance rate at church leaders Care Group meetings will be at least 60%. 2. 50% of mothers who attended church during the past month report that they heard a health message. Church Care Group meeting records, M&E, 1. Conduct census of congregations within project area. 2. Approach pastors and church leaders to participate in church Care Groups. 3. Adapt health education for selected interventions to church use, e.g., for sermon outlines. 4. Through animator and supervisor visits to church leaders, promote cooperation across denominations. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A6 Table 19: Vurhonga II Objectives, Indicators and Major Activities Objectives Indicators Measurement Methods Major Planned Activities iii. Vounteer and Family Level 3a. Volunteers in the Care Groups will develop a sense of efficacy and pride in promoting changed behavior. 1. At least 60% of families have been visited by their volunteer during the past two weeks. 2. The attrition rate of volunteers for reasons other than death, disability, or movement out of the project area will be less than 10%. Project records, M&E plan 1. Care Groups will celebrate successes. 2. Results of the base line, MTE and M&E will be communicated to the volunteers in the Care Groups to chart their success. 3. Vurhonga II staff of at least supervisor level and “important visitors” will regularly praise volunteers before community leaders. 3b. Families will collectively demonstrate and sustain positive health behaviors Use indicators for specific CS interventions, e.g., participation in growth monitoring, EPI coverage, etc. Ministry of Health HIS Quarterly HIS 1. BCC by volunteers emphasizes the importance of sustaining positive health behaviors 3. PROGRAM SITE INFORMATION 1. Program location A map of Chokwe district and the project impact area is included in the front of this report. The map includes the names of all towns and villages, and indicates the location of existing hospitals, health centers, and health posts, including 27 health posts constructed since the beginning of Vurhonga II. The project map also indicates the five geographic areas of responsibility for the five project supervisors. Large poster-size versions of this map are used for planning purposes by the project offices and the MOH district office. Situated on the southern border of the Limpopo River about three hours (200 km) north from Maputo (capital of Mozambique), Chokwe is one of the most densely populated districts in Gaza province. The project site for Vurhonga I was in the districts of Guija and Mabalane just on the other side of the Limpopo River. The project impact area for Vurhonga II includes all towns and villages in Chokwe district except for the urban center of Chokwe town and the more rural town of Hokwe. Hokwe has been excluded because it has been assisted, since February 1999, by a project of the Nazarene Church. The Hokwe project was designed using the same development approach as the Vurhonga project, and even uses the same health education materials. 2. Population and beneficiaries The estimated population for Chokwe is 202,000. The project area, however, excludes Chokwe town (60,000 inhabitants) and the village of Hokwe (12,000 inhabitants). The total population for the project area is therefore estimated to be 130,500. A project census of the impact areas was completed in early 2000, and calculated that the total number of potential beneficiaries would be 53,418, i.e., 31,338 women age 15-49 years old and 22,080 children under five years of age. The estimated number of births to occur during the life of the project is 19,836 (130.500 x 3.8 births/100 x 4 years). World Relief Mozambique CSXV Midterm Evaluation Report 2001 A7 3. Health status Rural Mozambique is a dangerous place for women and children. The national infant mortality rate is 133, the under 5 mortality rate is 214 and the total fertility rate is 6.2.5 Weakened by malnutrition, children die most frequently of malaria, diarrhea and pneumonia. According to national DHS data, malaria and diarrhea are the top two killers of children under 5. The 1997 Demographic Health Survey (DHS) found high incidence of diarrhea (1 in 5 prevalence within the past 2 weeks) and low levels ORT knowledge (35%). Among children who died after admission to Chokwe Hospital in the last six months, hospital records attributed 50% of the deaths to malnutrition, 16% to malaria, and 15% each to pneumonia and diarrhea.6 Maternal mortality for Mozambique (1990 data) is estimated as 1,500/100,000 live births.7 USAID estimates a maternal mortality rate of 631/100,000 live births in their focus areas.8 About one-third of the project effort will address reproductive health issues (birth spacing, maternal care, and HIV/STI education) highly relevant to child survival in Chokwe. 4. Other factors influence health status a. Economic characteristics of the population The rural economy of the project area lags behind the increased commerce of the towns. Most of the families in the project area raise only enough food from their farms to sustain themselves. The land is flat and susceptible to seasonal droughts and floods. Even with the financial assistance from the many men who work in South African mines and industry, six in ten people live below the level of absolute poverty. b. Social characteristics Shangan is the mother tongue and ethnic identity of 97% of the people of Chokwe. Most (60%) identify themselves as Christians of various denominations. The remaining 40% follow traditional African beliefs. Because the Shangan people are both patriarchal and patrilocal, women are among the most powerless. The freedom of young women of child-bearing age to adopt new health behaviors is constrained by long hours of hard work each day and allegiance to their husbands and the older women of their husband’s household. Fewer than four in ten women are literate. Further, the disproportionate ratio of women (58%) to men (42%) because of war and male migration to South Africa encourages polygamy and multiple sexual partners. Because women’s sense of value and future security depends upon childbearing, the incidence of sexually transmitted infections increases even as their power relative to men decreases. c. Practices regarding the care of infants A mother provides complete care of her child during the first several months. She keeps the child with her at all time, even when going to the fields, so that she can breastfeed the child on demand. Weaning is often done suddenly. When the child’s grandmother feels that the right time has come, or when the child’s mother again becomes pregnant, the child is removed from the breast and often sent to the grandmother’s home to completely separate the child from the mother for a time. Children who are able to put food in their mouths are placed on the floor with the food dish and expected to eat with their fingers. Usually during mealtimes small children are only supervised by older siblings. The mother also frequently leaves a child under the care of siblings when she goes to the field for the day. The Grandmother, if available, may supervise all the children, but not necessarily provide direct care for the youngest child. 5 State of the World’s Children 1998. UNICEF. Oxford University Press, 1998. 6 MOH Chokwe District health office, 1998. 7 State of the World’s Children 1998. UNICEF. Oxford University Press, 1998. 8 USAID Mozambique Mission Strategic Objectives, 1998. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A8 A child will usually be given food left over from the previous day, and competes poorly with older siblings for the available food. Rarely do young children receive protein or vitamin-rich foods in their porridge, and they do not eat enough meals per day. No differences exist between the manner that young boys and girls are fed, and they generally eat out of the same dish. Typically, men receive the first choice of food, with the rest left for the other family members to eat. d. Potential geographic, economic, political, educational, and cultural constraints Most villages in Chokwe district are situated along roads that are accessible year round. Floods in Chokwe town and neighboring villages are a rare but an all too real and potential constraint for the project. The last floods of February 2000 left the town of Chokwe under two meters of water, and interrupted the project activities of Vurhonga II for more than six months. There are a number of traditional beliefs that represent cultural constraints for the promotion of improved health practices. For example, traditionally mothers attribute the visible signs and symptoms of malnutrition not to poor nutritional practices but to oppression (being ‘sat upon’) by malevolent spirits and therefore cannot see the relationship between malnutrition and feeding behaviors. 5. Existing status of health services The existing health infrastructure for Chokwe district is summarized in Table 2. At the beginning of Vurhonga II the MOH infrastructure included the hospital of Chokwe, three health centers with nurses and 14 health posts with either a nurse or Socorrista. There were 34 villages without health posts. Population access to health services within 5 km of the village was 65%. Table 2: Health Infrastructure of Chokwe District (1999-2001) Health Facilities (construction materials + type of staff) Chokwe District 1999 2001 Health Center (durable + nurses + midwife) 3 3 Health Post (durable + nurse and/or midwife) 10 10 Health Post (durable + Socorrista) 2 3 Health Post (local materials + Socorrista) 2 28 Total 17 44 Population within 5 kms of a Health Post 84,630 132,084 Total Population in the Project Area 130,492 132,480 % of population within 5 km of a health post 64.8% 99.7% In addition the MOH health facilities, other health providers in the Chokwe District include:  a Catholic hospital in Chokwe that tests and treats AIDS patients;  the health program of the Nazarene Church in the community of Hokwe;  Med-Air assistance to a number of health posts;  Socorristas managing health posts under the supervision of the MOH  Traditional Healers (Inyanga spiritists) The population pays a nominal fee for services at health posts and health centers. At MOH health posts and health centers there is a 1000 mt. (.06 USD) charge for service and an additional charge for medicines of approximately 500 mt. (.03 USD). The total payment per episode is therefore around 1500 mt. or nine cents in U.S. currency. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A9 At health posts with a Socorrista the level of cost recovery is somewhat higher, i.e., around 2000 to 3000 mt. per episode treated depending on the agreement at the village level. An average workload of a Socorrista is five to ten patients per day. These funds are kept entirely at the health post level and constitute the salary of the Socorrista. The only other significant cost for Socorrista is transportation cost to travel to the district office or to the nearest health post with a nurse at least once a month. The Socorrista cover this cost out of their salary. The morale of MOH health workers has traditionally been low, with a correspondingly low utilization of services by the population, who did not feel that they were well received. However, the general economic improvements throughout the country and salary increases for health personnel make the current situation more positive than it was during Vurhonga I. 6. Identify any groups in the program site that you consider disadvantaged One disadvantaged group within Chokwe district consists of villages that lack good access to EPI services. While the mobile team theoretically serves these villages from Chokwe, the frequent unavailability of a vehicle has resulted in a relatively low EPI coverage for Chokwe district. 4. PROGRAM DESIGN The overall goals of the Vurhonga II project are to: 1) Reduce morbidity and mortality in children under 5 and women of fertile age, 2) Strengthen the capacity of the MOH to implement CS interventions; and 3) Empower communities to improve their health. The specific purposes and strategies within the above goals are to:  Reduce malnutrition by 50% by growth monitoring , counseling and weaning foods;  Improve maternal nutrition by increasing food consumption during pregnancy and lactation;  Reduce malaria death rate in children by 50% by early recognition and treatment;  Reduce pneumonia deaths by early recognition and treatment;  Reduce deaths and speed recovery from diarrhea by ORT and improved nutrition;  Reduce transmission of HIV/AIDS through improved recognition and treatment of STIs;  Double the number of couples spacing their children with a modern method;  Increase the number of mothers delivering children by a trained provider;  Increase access to health care to 100% of the population within 10 km of a trained provider;  Improve the management of childhood malaria and pneumonia at health posts;  Establish sustainable village health committees to monitor health activities; and • Mobilize and train over 2,000 volunteers, churches and community leaders. Program Design and Stakeholders World Relief has designed Vurhonga II with USAID Mission Strategic Objectives (SO) in mind. All proposed program interventions share Mission SO3 goals of improving access, quality and management of MOH services while also mobilizing community demand for basic health services. National government policies for child survival are basically sound and still developing. Vurhonga II strategies are consistent with MOH policies. Where there are differences the project will train them in new protocols, e.g., malaria and pneumonia. Vurhonga II will also assist the MOH in improving their capacity to plan, implement and supervise child survival interventions and community-based activities. Vurhonga II will help 47 communities create 220 Care Groups to train 2,800 volunteer mothers and church leaders as behavior change agents to all households for nutrition; malaria, pneumonia, and diarrhea case World Relief Mozambique CSXV Midterm Evaluation Report 2001 A10 management; maternal and newborn care; child spacing; and STI/HIV/AIDS prevention. The project will help the MOH expand its network of health posts so that 95% of the population lives within 5 kms of a trained health care provider (nurse or Socorrista). The project will also establish communication and coordination links between the communities and health posts. 5. PARTNERSHIPS The local partners for Vurhonga II are the Ministry of Health and 47 local communities. The working relationship with these partners will build on an important lesson learned from Vurhonga I related to working in partnership with the MOH. Lesson Learned: Working in collaboration with the MOH from the beginning and with a discipline to respect each others’ roles creates a synergy that both improves health care and is also sustainable. Project interventions for Vurhonga I showed dramatic improvement for all interventions. Project goals were not only met, but greatly surpassed. This success was attributed to a combination of 1) enhanced awareness of health issues at the household level; 2) improved access to health services; 3) increased availability of supplies; and 4) cooperation and communication with the MOH. The Vurhonga I project increased the demand for services by focusing on communication for behavior change at the household level. The MOH increased access to and quality of health services with the creation of new health posts and the training of Socorristas. These approaches, and the links between them, had a synergistic impact that was greater than either could have achieved alone. All child survival interventions were carried out in collaboration. There was a demonstrated willingness and capacity by both the MOH and community to continue and sustain these services. In addition to working successfully with the MOH as a partner, the Vurhonga I project also worked in partnership at the community level. The final evaluation of Vurhonga I found that Knowledge of child survival has significantly increased at the household level. What is more important, changes to positive health behaviors are clearly sustainable. The project’s strategy to use a participative methodology, “mentored” training, home visits and demonstrations have resulted in very effective and sustainable educational. The Care Group structure created a dynamic support structure for CS interventions and behavior change. “People are doing things differently now” and “It is now part of our daily lives” were heard repeatedly during evaluation visits. Therefore, the two major partners for Vurhonga II are the MOH and the communities, especially the Care Group structures. The respective roles of these partners and the project staff are outlined in the Table 6. See also Table 5 regarding the relationship with other health providers. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A11 Table 6: Roles of Major Partners by Child Survival Intervention Intervention MOH Community Project Staff Nutrition Growth Monitoring - Growth Monitoring -Vit A distribution -Assist with GM and HEARTH -Training in GM Malaria/Pneumonia - Diagnosis and treatment - Stock of Essential Meds -BCC -Distribute Bednets - Bednet retreatment -Train Socorrista & nurses -Procure bednets -Plan Bednet Retreating Reproductive Health Maternal Health Child Spacing STI/HIV/AIDS -Stock of Contraceptives Train communities CDD -Stock of ORS packets -Training in treatment - Immunizations Maintain Cold Chain Mobilize for EPI Support as needed 6. HIS There are four processes – formal evaluations, intervention specific research, cycles of organizational learning, and the promotion of personalized feedback loops – which WRC will combine to provide a richly detailed picture of the Vurhonga II program. a. Formal Evaluative and Monitoring Cycles. These include the proposal and DIP reviews, the midterm and final evaluations led by external evaluators and the three KPC surveys that punctuate the progress of the project. By choosing external evaluators who are not only technically competent but also highly participatory and carefully examining and responding to the recommendations, the Vurhonga II team will incorporate these inputs into program formation and change. The Vurhonga II project will use the quarterly HIS M&E developed in Vurhonga I. See detailed description later in this section. b. Intervention Specific Research. Each of the project interventions will require specific qualitative approaches to inform program and BCC strategy. The Hearth program, for instance, will use 24 hour diet recalls in each village to identify locally sustainable nutritious diets, that are continually assessed to take into account regional and seasonal changes. The quality of care programs in reproductive health will involve exit interviews, focus groups and other procedures. The feedback loop for intervention specific research is short and extremely practical. Knowledge about health services and the needs and practices of people will be used to revise objectives and plans at several key junctures: 1) during objective-setting at the DIP, after the KPC survey, 2) during development of curriculum, (TOT with animators, and when they train Care Groups 3) after midterm evaluation, monitoring visits, and to a lesser extent after each quarterly M&E. c. Organizational Learning Cycles. The formal evaluation and monitoring system is only one of the organizational learning cycles built into Vurhonga II. Each work week will be sandwiched between meetings every Monday morning and Friday afternoon when animators, supervisors and technical staff discuss progress, results and problems. These meetings function as quality assurance (QA) meetings where project staff identify opportunities for improvement, define problems, establish desired outcomes and plan steps to achieve them. During these World Relief Mozambique CSXV Midterm Evaluation Report 2001 A12 meetings animators will get their questions answered, review progress toward goals and develop weekly action plans that will be left up on the board for discussion the next week. In monthly cycles, the Project Director will attend district MOH staff meetings to share feedback. The M&E and training camps mark quarterly cycles of information gathering and organizational reflection. The Project Director is responsible to collect and analyze data on limiting factors and will plan corrective action and monitor results. The animator uses a weekly monitoring and reporting form that includes columns for each intervention. She writes down how many groups she met with and what interventions she discussed. This includes meeting with and individual or group, MOH, NGO’s, or church leaders. On back of the form the animator may comment on any significant issue the supervisor needs to know. She fills in this form daily, and reports the results to supervisor at the end of each week, but keeps the form. The supervisor in turn reports results to the project director. d. Personalized Feedback Loops. A unique strength of the Vurhonga I program was the highly personalized feedback loops. Unlike many CSP’s, the project language from top management to the beneficiaries is Shangan, greatly enhancing communication and cultural understanding. Personal feedback is essential to the cultural fit and flexibility of program interventions as well as to continuous cycles of quality assessment. In Vurhonga II, technical staff and supervisors will live with animators in the field and assess progress through interviews and observations, guided by supervisory forms and project indicators. The Project Director (PD) will continue to work in the district hospital in Chokwe one day a week, allowing him to personally assess coverage, quality and needs of the health system and access MOH statistics (malaria and pneumonia cases treated, FP acceptors, EPI coverage, mortalities and morbidities). The project director and supervisors can assess quality of MOH services in visits to clinics and Socorristas. Needs of existing health services are discussed in routine local visits and in monthly district meetings with MOH, which also provide an ideal opportunity for the Director to share M&E results, report on progress, issue invitations to TOT camps, etc. Quality of care can also be assessed through exit interviews, focus groups and other methods learned at the Quality of Care workshop sponsored by SEATS as part of Vurhonga I. Animators and volunteers will discover needs in the community and monitor client satisfaction by their constant contact with Care Groups representing cross-sections of the population. Qualitative personal feedback from the animators will be essential to the program. The animators, who share the culture of the people, encode key intervention messages in song, using communal learning and support to facilitate change and develop accountability. They will address behavior change in a shared context of community norms, with culturally appropriate communication models. Each animator will track volunteer attendance at Care Group meetings as an indicator of collective volunteer performance. Care Groups must have 60% average attendance to be considered viable. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A13 3. Quarterly HIS Monitoring and Evaluation The quality, effectiveness and coverage of the interventions are monitored through these quarterly rapid assessment surveys (the M&E plan), which use KPC survey questions to monitor most knowledge, practices and coverage indicators. Sample survey forms are shown in Figures 4 below and Figure 5 on the following page. Figure 4: HIS Survey Form: Questions for Mothers Every three or four months, one Care Group from each of the 26 animators is randomly chosen and all the families, approximately 150 households, that they are responsible for, are surveyed. To preserve objectivity, the animators trade areas to conduct the home surveys. The resulting data set of 2,500+ interviews is hand-tabulated and summarized by the animators, then analyzed and reported project-wide to chart progress on objectives. Animators and supervisors will also use this time as an educational opportunity; the volunteers benefit from immediate feedback, and the results are left with each animator for them to review with their Care Group. Thus in addition to routine informal visits from animators and supervisors, every home will be surveyed every two years. The collected information is used to track progress towards objectives, and the animator gives feedback to village health committees. World Relief Mozambique CSXV Midterm Evaluation Report 2001 A14 Figure 5: HIS Survey Form: Questions about Children The quarterly HIS system used by Vurhonga I proved to me quite accurate in matching the results of the formal KPC process. World Relief Mozambique CSXV Midterm Evaluation Report 2001 B1 B. MTE Team Members and other persons contacted MTE Team members NAME POSITION AFFILIATION Project staff Pieter Ernst Director World Relief Margarida Massingwe Secretary World Relief Crestina Cossa Area 1 Supervisor World Relief Veronica Cima Area 2 Supervisor World Relief Clara Javana Area 3 Supervisor World Relief Selina Hlonga Area 4 Supervisor World Relief Delfina Maluleque Area 5 Supervisor World Relief Partner representatives Vitorino Luis District Director MOH Inacio Xitlangu District Training Coordinator; Socorrista Supervisor and In-charge of Statistics. MOH Americo Mathe Pastor Presbyterian Church Antonio Cossa Pastor United Methodist Church Albino Chambal Village Leader (Xiaquelane) Government Simao Mabessa Village Leader (Xilembene) Government Translator Sybil Balayi Child Evangelism Director World Relief External evaluators Melanie Morrow Child Survival Specialist World Relief headquarters Victor Kabaghe Child Survival Project Director World Relief Malawi Marc Debay External consultant Johns Hopkins University Other persons contacted NAME POSITION AFFILIATION Ilka Esquivel Child Survival and USAID/Mozambique Reproductive Health Results Team Leader World Relief Mozambique CSXV Midterm Evaluation Report 2001 C1 C. Assessment Methodology The MTE of the Vurhonga II Child Survival project ran from Tuesday August 21st through to August 31st 2001. The MTE team comprised 7 project staff members (the director, the administrative assistant, and the five supervisors) and 3 external evaluators (the World Relief (WR) Headquarters child survival specialist in charge of the project, the project director of the WR/Malawi Child Survival project, and an external consultant). One representative of the Ministry of Health (MOH) was available for 75% of his time during the evaluation period and participated in all field visits. Two church leaders and two village leaders representatives participated in the inception and in the final meetings. The MOH District Director participated in the final meeting. One WR staff member from another project in Chokwe graciously provided translation assistance during all the meetings and site visits. The list of participants of the MTE is in Appendix B. The schedule of the MTE reflects the methodology adopted by the team (See Appendix B). The first day was used to identify the MTE team members and clarify the USAID/BHR/PVC guidelines for the midterm evaluation of the PVO Child Survival Grants Program projects. During the first meeting of the MTE team, which involved only project staff and external evaluators, the external consultant presented the principles of the evaluation (participation, reference to the DIP, and use of project data) and the time and logistical constraints to take into account in scheduling the MTE activities. Then he led a group review of the technical approaches of the project to identify the main problems or questions encountered so far in their implementation. The emphasis was placed on malaria, diarrheal diseases, nutrition and HIV/AIDS as the interventions already implemented (as opposed to reproductive health and pneumonia to be phased in later). A second MTE team meeting the next day provided the opportunity to the MOH, church, and village representatives to catch up with the evaluation process, and contribute to the group discussions on the crosscutting approaches of the project. Five main themes were proposed to channel these discussions: community mobilization, behavior change communication, partners strengthening, health information system, and sustainability. The first day of site visits was scheduled after the main questions of the MTE were identified. The MTE team decided that two similar teams of six evaluators would go to two separate villages and meet representatives of the various types of actors in the Vurhonga projects. The two villages were purposely chosen among all those that had a Care Group (CG) meeting planned for that day, and so that one village would represent a poor and the other village a high level of project performance. The MTE team then developed interview and observation guides to investigate the questions identified earlier. These tools all included a similar list of topics but addressed them in a somewhat different way depending on the type of informants. The two teams of evaluators were able to meet all the group and individual informants as organized by the animators in charge of the two villages. This demonstrates how responsive to the project are the community members. In each village, individual interviews were first conducted with the village leader, and then group interviews were World Relief Mozambique CSXV Midterm Evaluation Report 2001 C2 conducted with pastors and with volunteer mothers. The interviews with the volunteer mothers began with a short drama on a health topic performed by the supervisors to facilitate communication and test the knowledge of the related messages. The visits of the health post included a rapid observation of the facility and an interview with the health workers (socorristas or nurse). Although one team member usually conducted the interview following the guidelines, all the members followed the discussions, contributed questions, and took personal notes. The two teams met the following day to share their findings and draw common conclusions. The purpose of a second day of field visits was discussed with respect to these initial conclusions. The MTE team first considered investigating further the effectiveness of the health education on the behaviors of the mothers and caretakers but concluded that available HIS indicators provided enough evidence a positive effect. The quality of care in the health posts was also considered but the MTE team concluded that the main issues were known by the project supervisors and the MOH, and that there was no need of further investigation (see section D.3). A consensus rapidly emerged that two areas needed further investigation: 1) the impact of the training of grannies, and 2) the current and future roles of the chefs de saude. The MTE team then developed guides for group interviews of these two types of informants. During the second day of field visits, two teams of six evaluators each met a group of about fifteen grannies in different villages. The entire MTE team then met with six chefs de saude from different villages and areas that the animators gathered in one meeting point. The findings from these meeting were shared and common conclusions drawn in a similar way as for the first day of field visits. The external evaluators had the opportunity to attend one of the weekly meetings of the 26 animators with the 5 supervisors and the project director. They first observed a discussion of ongoing project issues—one being the organization of TBA training (See Section III-C). During the second part of the meeting, the MTE team reviewed with the animators the various issues and questions that they had been investigating so far. The contribution of the animators provided valuable insight to this analysis. A subject of particular interest was that of their future after the project ends, and various options were considered (See Section II-E). The MTE team or selected members also reviewed a few additional aspects of the project to complete the midterm evaluation, primarily through review of documents and discussion with the WR local and headquarters staff: the content and user-friendliness of the educational materials; various aspects of the management of the project; the type, coverage and data of the project and MOH HIS; and the progress towards the project objectives and the need for changes in the indicators and targets. Finally, the external evaluators prepared preliminary findings and recommendations that were presented for discussion in a last meeting with the entire MTE team including the partner representatives. This concluded the participatory component of the MTE. The external evaluators then began writing the MTE report, and prepared for the debriefing at the USAID mission that took place in Maputo on Friday 31. World Relief Mozambique CSXV Midterm Evaluation Report 2001 D1 D. Schedule of the midterm evaluation DAY DATE ACTIVITY Tuesday August 21 MTE team meeting: - MTE principles and schedule - Review of technical approaches Wednesday August 22 MTE team meeting (including partner representatives): - Review of cross-cutting approaches - Development of tools for site visits Thursday August 23 Site visits to Lionde and to 25 September: - Meetings with village leaders, pastors, and volunteers - Observations of Care Groups - Visits of health posts and interview of health workers Friday August 24 MTE team meeting: - Sharing and summarizing first site visits findings MTE team meeting with all project animators (26) Saturday August 25 Review project records (HIS, training and educational materials, project timeline) MTE team meeting: - Planning second site visits - Review of progress towards project objectives Sunday August 26 REST Monday August 27 Site visits: - Meeting with grannies in 2 villages - Meeting with chefs de Saude - Visit of health post MTE team meeting: - Sharing and summarizing of second site visits Tuesday August 28 Preparation of preliminary findings and recommendations MTE team meeting (including partner representatives): - Discussing preliminary findings and recommendations Wednesday August 29 Report Writing Review of program management Thursday August 30 Report Writing Friday August 31 Travel to Maputo Debriefing meeting at USAID mission World Relief Mozambique CSXV Midterm Evaluation Report 2001 E1 E. Project implementation timeline: October 1999 – August 2001 Selected Project Activity Timeline to Date 1999 2000 2001 O N D J F M A M J J A S O N D J F M A M J J A Project management landmarks Project begins October 1, 1999 X Emergency Flood Relief X X XXX DIP Preparation and Review X Midterm Evaluation X Health Information System Baseline KPC X Census of WRA and children <5 X Quarterly surveys X X X X Community health information system XXX Rapid Catch Survey X Community health education Volunteer selection X X X X Care group formation XXXXXX Malaria education X X X ITN distribution and retreatment X X Diarrhea education X X X X Nutrition education XXX Hearth (2 cycles) X X HIV education (beginning) X Strengthening of health posts 18 new health posts completed X Socorristas trained and in place XXX Village Health Committees in place X World Relief Mozambique CSXV Midterm Evaluation Report 2001 F1 F. Progress towards objectives: malaria 0% 20% 40% 60% 80% 100% 120% Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 Feb-00 Mar-00 Apr-00 May-00 Jun-00 Jul-00 Aug-00 Sep-00 Oct-00 Nov-00 Dec-00 Jan-01 Feb-01 Mar-01 Apr-01 May-01 Jun-01 Jul-01 Aug-01 ITN Treatment<24h malaria education ITN distribution FLOOD RELIEF 18 new health posts functioning Series label Objectives Indicators ITN Increase from 0.3% tot 70% the proportion of children under 5 who use insecticide treated nets year round. Percentage of children under two who slept under an insecticide treated net the previous night. Treatment<2 4h Increase from 28% to 75% the number of children treated within 24h for fever (suspected malaria) at any health facility. Children 0-23 months old who had fever (suspected malaria) in the past 2 weeks that were treated within 24 hours at appropriate health facility. World Relief Mozambique CSXV Midterm Evaluation Report 2001 G1 G. Progress towards objectives: diarrhea 0% 20% 40% 60% 80% 100% 120% Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 Feb-00 Mar-00 Apr-00 May-00 Jun-00 Jul-00 Aug-00 Sep-00 Oct-00 Nov-00 Dec-00 Jan-01 Feb-01 Mar-01 Apr-01 May-01 Jun-01 Jul-01 Aug-01 ORT Food diarrhea education FLOOD RELIEF Series label Objectives Indicators ORT Increase from 53% to 80% the proportion of children with diarrhea treated with ORT by mothers/volunteers. Children 0-23 months with diarrhea in the past 2 weeks who received ORT. Food Increase from 19% to 65% the proportion of mothers who give extra food to children for 2 weeks following diarrhea. Children 0-23 months who received extra food after their last diarrheal episode. World Relief Mozambique CSXV Midterm Evaluation Report 2001 H1 H. Progress towards objectives: nutrition 0% 20% 40% 60% 80% 100% 120% Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 Feb-00 Mar-00 Apr-00 May-00 Jun-00 Jul-00 Aug-00 Sep-00 Oct-00 Nov-00 Dec-00 Jan-01 Feb-01 Mar-01 Apr-01 May-01 Jun-01 Jul-01 Aug-01 weight counseling food nutrition education hearth 1 hearth 2 FLOOD RELIEF Series label Objectives Indicators Weight Increase from 75% to 85% the number of children 0-35 months weighed at least once during last 3 months. Children 0-[23]35 months with documented weighing at least once in last 3 months (by card). Couseling Increase from 22% to 80% the number of malnourished children's mothers who received nutr. counseling Mothers of malnourished children 0-23 months who received nutritional counseling by volunteers, animators, or MOH personnel. Food Increase from 50% to 70% the number of malnourished children who received nutritious foods/enriched porridge Mothers of malnourished children 0-23 months who receive nutrition counseling and stated they give nutritious weaning foods/ enriched porridge to their child at least once per day. World Relief Mozambique CSXV Midterm Evaluation Report 2001 I1 I. Cause Specific Number of Deaths Cause-specific number of deaths under five Vurhonga II project, Chokwe district, March 2000 - July 2001 0 10 20 30 40 50 60 70 80 Sep-99 Oct-99 Nov-99 Dec-99 Jan-00 Feb-00 Mar-00 Apr-00 May-00 Jun-00 Jul-00 Aug-00 Sep-00 Oct-00 Nov-00 Dec-00 Jan-01 Feb-01 Mar-01 Apr-01 May-01 Jun-01 Jul-01 Aug-01 Number of deaths Malaria Diarrhea Pneumonia Malnutrition Other malaria education ITN distribution diarrhea education nutrition education Hearth 1 Hearth 2 FLOOD RELIEF 28 health posts functioning KPC QS 1 QS 2 QS 3 QS 4 KPC: Baseline Knowledge, Practice and Coverage survey; QS: Quaterly surveys World Relief Mozambique CSXV Midterm Evaluation Report 2001 J1 J. Response to DIP Review Recommendations Issue #1: Revise budget, if necessary. Response: Revised budget for last three years of program including Lionde region expansion submitted to Technical Officer. Issue #2: Obtain the standard IMCI materials and update the standard case management materials to be used in your program Response: We did have received the updated IMCI materials and videos and have already trained all the community health workers (nurses and socorristas) in it. Issue #3: Develop a contingency plan for possible future floods a) A communication network is in place through radio phones where we are able to contact the centers where the staff stay during the week so that they would be able to spread the message quickly and evacuate in time in case of a sudden threatening flood. b) Back-ups of all files have been made and are still being made on diskettes for quick removal if necessary c) A security fence has been put up around the WR office to help safeguard and prevent any looting that usually accompany such disasters. d) Easy access to high roofs through steps in the office area have been made available to elevate any important objects in time to prevent damage through flooding. e) An elevated container has been equipped to serve as a hiding place for the guards so that the office area would not be left alone during and after a flood. f) Next time we would rather flee with our laptop computers than with any other important items. g) Any information received from relevant authorities will be taken seriously.