MIDTERM EVALUATION REPORT FOR THE TUBARAMURE PM2A PROGRAM Cooperative Agreement No: AID-FFP-09-00004-00 JULY 2012 i ACKNOWLEDGEMENTS First and foremost, the evaluation team would like to express its appreciation to all the beneficiaries and Tubaramure partners we met in Cancuzo and Ruyigi Provinces, including the Government of Burundi administration, local leaders, and the Ministry of Health at the provincial and facility levels. Their willingness to spend time with us and the insights they provided into the program were key to helping us understand the impact of the activities. We are also very grateful to all the Tubaramure staff at the national, provincial and commune levels for the many long hours they devoted to answering our questions, accompanying us to the field, and providing the numerous documents we requested for an in-depth understanding of the program. As with most evaluations, those implementing the program were the best source of ideas for making midcourse corrections; this report reflects many of their observations and recommendations. We cannot thank enough the three people who served as our interpreters: Mme. Marie Rose Hatungimana, Félicien Harerimana, and Jean Claude Ndikumazambo. They were with us throughout the process, serving not only as interpreters but also as cultural guides. Their contributions to the daily review sessions were also much appreciated. Dr. Térence Hatorimana was an important addition to the evaluation team during the field work. His knowledge of the Ministry of Health provided critical background information and helped us to understand the history of community health in Burundi and the constraints encountered by the program. His presence greatly facilitated access to the Ministry of Health personnel in the program area and his dry sense of humor enlivened our meetings. Four other essential members of the team were the drivers who transported us safely, ran errands, and kept everything running smoothly. Our thanks go to Macias Bigirimana, André Nderagakura, Anselme Ndikubwayo, and Miburo Ladislas. The CRS/Burundi office provided outstanding support throughout the evaluation. We especially appreciated the efforts of Mme. Scholastique Ntakirutimana who organized an office for us; kept us supplied with money, equipment and office supplies; provided excellent logistical support; and anticipated all our needs. Thanks also to Mme. Estella Mazuru ensured that our travel and hotel reservations were made. And a final thank you goes to Dr. Raphael Bajay Tchumah, the Tubaramure Chief of Party, for the stellar support and guidance, which started well before we arrived in Burundi. From the initial meeting with the Tubaramure consortium members in Bujumbura, Dr. Bajay set the right tone, stressing the need for candor and availability from all team members. Everything was exceptionally well-organized from beginning to end, a result of his foresight, meticulous planning, and problem-solving skills. Kathy Tilford, Team Leader Ange Tingbo, Commodity Supply Chain Expert Dr. Vera Bensmann, Primary Health Care Expert ii iii ACRONYMS AND TERMS ASC Agent de Santé Communautaire (= CHW) BCC Behavior Change Communication BDS Bureau de District Sanitaire (= District Health Department) BPS Bureau Provincial de Santé (= Provincial Health Department) CG Care Group CHW Community Health Worker C-IMCI Community-Integrated Management of Childhood Illnesses CMAM Community Management of Acute Malnutrition CMU Commodity and Monetization Unit CNTA Centre National de Technologie Agro-alimentaire or National Center for Food Technology Colline Literally, “hill”: an administrative unit COSA Comité de Santé CRS Catholic Relief Services CSB Corn Soy Blend DIP Detailed Implementation Plan EBF Exclusive Breastfeeding EHA Essential Hygiene Actions ENA Essential Nutrition Actions EPI Expanded Program on Immunization EUC End-use Checker FANTA Food and Nutrition Technical Assistance Project FARN Foyer d'Apprentissage et de Réhabilitation Nutritionnelle FBu Burundi Francs FFP Food for Peace FH Food for the Hungry FY Fiscal Year GoB Government of Burundi GM Growth Monitoring HF Health Facility HH Household iv HW Hand washing IEC Information, Education, Communication IFPRI International Food Policy Research Institute IGA Income-generating Activity IMC International Medical Corps IMCI Integrated Management of Childhood Illnesses IPT Intermittent Preventive Treatment IPTT Indicator Performance Tracking Table IR Intermediate Result ISTEEBU Institut de Statistiques et d'Etudes Economiques du Burundi (Burundian Institute for Statistics and Economic Studies) ITN Insecticide Treated Net IYCF Infant and Young Child Feeding kg. kilogram km. kilometer LM Leader Mother M&E Monitoring and Evaluation MAE Ministry of Agriculture and Animal Husbandry MoH Ministry of Health MT Metric Ton MTE Midterm Evaluation MUAC Mid-Upper Arm Circumference MYAP Multi-year Assistance Program NA Not available ORS Oral Rehydration Salts OTP Outpatient Therapeutic Program PD-Hearth Positive Deviance/ Hearth Model PM2A Preventing Malnutrition in Children Under 2 Approach PMP Program Management Plan PMTCT Prevention of Mother to Child Transmission PRONIANUT Programme National Intégré pour l’Alimentation et la Nutrition (= National Food and Nutrition Program) PSN Promoteurs de Santé et Nutrition (= Tubaramure Health Promotors) v Q2 Quarter 2 RUTF Ready to Use Therapeutic Food SAM Severe Acute Malnutrition SILC Savings and Internal Lending Communities THP Tubaramure Health Promoter (PSN in French) TOR Terms of Reference TPS Technicien de Promotion de la Santé (= Public Health Technician) USAID United States Agency for International Development USD U.S. dollars USDA United States Department of Agriculture WFP World Food Program WHO World Health Organization TABLE OF CONTENTS ACKNOWLEDGEMENTS ............................................................................................................. I ACRONYMS AND TERMS........................................................................................................ III EXECUTIVE SUMMARY ............................................................................................................ 1 TERMS OF REFERENCE FOR THE MIDTERM EVALUATION ............................................. 4 OVERVIEW OF THE TUBARAMURE PM2A PROJECT .......................................................... 5 Table 1. Framework for the Tubaramure program (from proposal) ....................................... 6 INTERMEDIATE RESULT 1: WOMEN AND CHILDREN UNDER 5 ACCESS QUALITY NUTRITION AND HEALTH SERVICES .................................................................................... 8 I. IR-1 ACTIVITIES ............................................................................................................ 8 A. Training of Personnel ......................................................................................................... 8 B. Material Support to the MoH ........................................................................................... 11 C. Monitoring and Evaluation............................................................................................... 11 II. PROGRESS TOWARD PROGRAM OUTCOMES AND OUTPUTS ..................... 13 A. Performance Indicators for Measuring Outcomes ........................................................... 13 B. Progress on Indicators for Measuring Outputs ................................................................. 17 III. STRENGTHS, WEAKNESSES, THREATS AND OPPORTUNITIES .................... 24 A. Strengths........................................................................................................................... 24 B. Weaknesses ...................................................................................................................... 24 C. Threats .............................................................................................................................. 25 D. Opportunities .................................................................................................................... 26 IV. RECOMMENDATIONS AND SUGGESTIONS ........................................................ 26 A. Recommendations ............................................................................................................ 26 B. Suggestions....................................................................................................................... 28 INTERMEDIATE RESULT 2: HOUSEHOLDS PRACTICE APPROPRIATE HEALTH AND NUTRITION BEHAVIORS ......................................................................................................... 29 I. IR-2 ACTIVITIES .......................................................................................................... 29 A. Establishment of Care Groups of Leader Mothers ........................................................... 29 B. Development of BCC Materials ....................................................................................... 30 C. Cascade Training .............................................................................................................. 31 D. Complementary BCC Activities ...................................................................................... 31 II. PROGRESS TOWARD PROGRAM OUTCOMES AND OUTPUTS ..................... 34 A. Performance Indicators for Measuring Outcomes ........................................................... 34 B. Progress on Indicators for Measuring Outputs ................................................................. 35 C. Issues to Address to Ensure Continued Success .............................................................. 39 III. STRENGTHS, WEAKNESSES, OPPORTUNITIES AND THREATS .................... 40 A. Strengths........................................................................................................................... 40 B. Weaknesses ...................................................................................................................... 40 C. Opportunities .................................................................................................................... 40 D. Threats .............................................................................................................................. 41 IV. RECOMMENDATIONS AND SUGGESTIONS ........................................................ 41 A. Recommendations ............................................................................................................ 41 B. Suggestions....................................................................................................................... 41 INTERMEDIATE RESULT 3: ELIGIBLE WOMEN AND CHILDREN HAVE INCREASED INTAKE OF NUTRIENT-RICH, DIVERSE FOODS ................................................................. 42 I. IR-3 ACTIVITIES .......................................................................................................... 42 A. Ensuring the Proper Utilization of Title II Rations .......................................................... 42 B. Promoting Increased Consumption of Appropriate Local Foods..................................... 43 II. PROGRESS TOWARD PROGRAM OUTCOMES AND OUTPUTS ..................... 45 A. Performance Indicator for Measuring the Program Outcome .......................................... 45 B. Progress on Indicators for Measuring Outputs ................................................................. 45 III. STRENGTHS, WEAKNESSES, OPPORTUNITIES AND THREATS .................... 48 A. Strengths........................................................................................................................... 48 B. Weaknesses ...................................................................................................................... 48 C. Opportunities .................................................................................................................... 48 D. Threats .............................................................................................................................. 48 IV. RECOMMENDATIONS AND SUGGESTIONS ........................................................ 49 A. Recommendations ............................................................................................................ 49 TITLE II FOOD COMMODITIES MANAGEMENT ................................................................. 49 I. THE FOOD LOGISTICS SYSTEM ............................................................................. 50 A. Title II Food Movement from Dar-Es-Salaam to Burundi .............................................. 50 B. Food Distribution to Caritas Warehouses - Methodology ............................................... 56 II. FINDINGS AND RECOMMENDATIONS FROM THE FIELD VISITS ............... 59 A. Findings at Food Distribution Sites ................................................................................. 59 III. SUMMARY OF RECOMMENDATIONS FOR COMMODITY MANAGEMENT 69 CROSS-CUTTING THEMES ...................................................................................................... 70 I. MANAGEMENT, COORDINATION AND COMMUNICATION .......................... 70 II. MONITORING AND EVALUTION ............................................................................ 71 III. GENDER ......................................................................................................................... 72 IV. TRACKING GRADUATES .......................................................................................... 72 LESSONS LEARNED.................................................................................................................. 73 SUMMARY OF PRINCIPAL RECOMMENDATIONS ............................................................ 75 ANNEXES Annex A: Program for Each Consultant Annex B: Terms of Reference for Ministry of Health Expert Annex C: Terms of Reference for the Midterm Evaluation Annex D: Summary Responses to Evaluation Questions Annex E: List of Key Informants Annex F: List of Documents Consulted Annex G: Map of Program Area Annex H: Example of Materials Provided Annex I: Example of Donation Annex J: List of Lessons in BCC Modules Annex K: List of Eligibility Conditions for Beneficiaries Annex L: Most Recent Commodities Ledger Annex M: Revised Program Management Plan (proposed) Annex N: Revised IPTT (proposed) 1 EXECUTIVE SUMMARY This report covers the qualitative midterm evaluation of a five-year USAID-financed Multi-Year Assistance Program (MYAP) entitled Tubaramure (“Let’s Help Them Grow” in Kirundi). Three external consultants conducted the evaluation over a four-week period in May-June 2012, with assistance from an Expert from the Ministry of Health (MoH). Catholic Relief Services (CRS) is the lead agency for Tubaramure and works in partnership with three other organizations through sub-awards: International Medical Corps (IMC), Food for the Hungry (FH), and CED-Caritas/Burundi. The program is designed around the “Preventing Malnutrition in Children under Two Approach” (PM2A), an approach that was first identified by the International Food Policy Research Institute (IFPRI) as more effective in reducing malnutrition than historically favored remedial methods. Tubaramure is being implemented in a total of 268 collines (literally, hills) in the 12 communes of Cankuzo and Ruyigi Provinces. Approximately 51,075 mother-child pairs are targeted over the five years (July 2009 to October 2014) of the $45 million grant; by April 2012 the program had already reached 49,652 mother-child pairs. The overall goal is to prevent malnutrition in children under two. To achieve this, activities are organized around three Intermediate Results (IR):  IR-1: Improving the capacity of health facilities/staff and Community Health Workers (CHWs) to deliver high quality general health and nutrition services  IR-2: Promoting household and community level behaviors that prevent maternal and childhood illnesses and support good nutrition using Care Groups (CGs) as the delivery mechanism  IR-3: Improving the caloric and nutrient intake of mothers and children under two through culinary demonstrations and monthly delivery of Title II commodities Overall, the evaluation team concluded that the program is well on its way toward meeting the majority of its objectives and targets and has already achieved visible results in terms of people trained, changes in behavior at the household and community level, improved services at health facilities, and better nutritional status among young children. Key findings for each IR are summarized below. IR-1: The evaluation team determined that the IR-1 component of the Tubaramure program is relevant, effective, efficient and sustainable. It is well-integrated into the local health system, has a strong emphasis on capacity building, and is technically sound in terms of health, nutrition, and monitoring and evaluation (M&E). The activities are well-designed, using the MoH’s policies, structures and systems, and targets are being met. At the half-way point in the program one can already see a significant increase in the number of 1) women attending pre- and postnatal health services, 2) children participating in growth-monitoring activities, and 3) nurses and CHWs demonstrating improved detection and treatment of childhood illnesses. Areas to focus on for the second half of the program include: providing monitoring and nutrition advice for children identified with mild or moderate malnutrition; scaling up the community component and promoting greater synergy between the CHWs and the Leader Mothers (LMs) in the Care 2 Groups; and reinforcing training and community awareness around the prevention and treatment of malaria, the major cause of childhood morbidity and mortality in the program area. IR-2: Significant progress has been made in achieving the program outcomes for this IR, designed to promote behavior change for improved maternal, child and community health. This component is highly relevant to the overall Tubaramure goal of preventing malnutrition in children under 2. Where IR-1 works to improve health services and outcomes, IR-2 focuses on what individuals, households and communities themselves can do to prevent illness and to sustain improvements in health. The development of well-researched, culturally appropriate Behavior Change Communication (BCC) materials is one of the successes of this component. The midterm evaluation results showed: increased levels of awareness and observable changes in behaviors, especially for infant and young child feeding practices; more timely health-seeking behavior for pregnant women and young children; and widespread improvements in household￾level hygiene and sanitation (e.g., latrines, hand washing stations and compost pits). Issues which should be addressed before the end of the program include: 1) finding ways to encourage those who have graduated from the ration distribution to continue participating in the BCC activities; 2) ensuring that barriers to behavior change continue to be identified and addressed directly; and 3) involving men more fully in activities. IR-3: The evaluation team noted that definite progress has been made in improving the diets of pregnant and lactating women and children under two. An anthropometric survey conducted during the same period as the midterm evaluation showed that Tubaramure is having a positive impact on the nutritional status of children who are enrolled in the program, corroborating the positive findings of the evaluation team. A well-run commodity supply chain ensures the timely distribution of rations and meets current standards and best practices for management of Title II commodities; the evaluator for this component commended the commodity management team for the notable improvements made since the distribution activities began. It was evident that these commodities are reaching the right beneficiaries and are being prepared correctly. The IR-3 activities have also been successful in promoting the consumption of a diverse, nutrient-rich diet. Beneficiaries and non-beneficiaries alike could cite the importance of a good diet for young children and give examples of a well-balanced meal. However, a constant refrain from those interviewed is that they lack the means to either produce or buy nutrient-rich foods in sufficient quantities and this poses a major threat to improving their diets in the absence of commodities. In fact, one of the key assumptions of the program design is that “Food security continues to improve” and if a lack of local foods turned out to be a problem, the proposed solution was to link with other projects. The evaluation team agreed that in the context of a program designed to prevent malnutrition in areas such as Cankuzo and Ruyigi Provinces, it is important to ensure that there are either strong links to other programs or a component to 1) improve food production (e.g., agriculture and/or animal husbandry) and/or 2) increase household revenues (e.g., credit schemes and/or income-generating activities) so that families can buy more nutritious local foods. Improving health services and promoting behavior change, however successful these interventions are, may not be sufficient in the long term to prevent malnutrition once the Title II ration distribution is finished. An IR-3 related evaluation question posed by USAID/Food for Peace was: What are the opportunity costs of providing the protection ration for other members of the household so that the mother-child ration is not consumed by everyone in the family? The midterm evaluation 3 clearly showed the importance of the protection ration. There was unanimous agreement among all those interviewed that the monthly mother/child ration would only last a few days if it was not accompanied by the protection ration. This was also the conclusion reached by the evaluation team. It is simply not acceptable or feasible for a mother not to share available food with other family members. In this respect, the protection ration definitely contributes to the overall program goal of preventing malnutrition in children under two. Strengths: In addition to the achievements for the individual IRs described above, Tubaramure has a number of strong points that have facilitated progress to date:  The program is very well-managed, with strong leadership that provides timely guidance and resolves problems quickly.  Tubaramure has excellent M&E systems that provide up-to-date information on program status. In addition, all four consortium members prepare regular reports to document progress, problems and trends.  The consortium members form a cohesive group, with comprehensive coordination and communication systems in place; this helps to ensure smooth implementation.  The development of excellent working relationships with the Government of Burundi administration, the Ministry of Health, local leaders and other government ministries facilitates activities; these close partnerships are key to ensuring sustainability.  The synergy among the three IRs - and among the consortium members - is evident and well-understood by program staff, beneficiaries and partners; those interviewed readily gave examples of how the three IRs complement each other. Gender: The original Tubaramure design discusses gender in several places, focusing on 1) involving men in health and nutrition activities and 2) empowering women to have more decision-making authority where the family’s diet is concerned. Competing program priorities and the absence of a well-defined plan at start-up meant that the gender focus was often missing from the program and staff have not fully exploited the opportunities to ensure that men are fully involved and that women are empowered to make household decisions. Program management is beginning to remedy this situation, bringing in a gender consultant, organizing gender training for staff, and making tentative efforts to include men in trainings and peer education. Conclusion: The evaluation team concluded that although there are issues to be addressed if Tubaramure is to achieve all of its objectives and targets, the program has made significant progress in strengthening the quality and delivery of health services, promoting sustained behavior change, and improving the health and nutritional status of pregnant and lactating women and children under two. The activities are highly relevant to the needs of the beneficiaries and to the priorities of the Government of Burundi and the USAID Mission. The program is effective, already demonstrating an impact across all three IRs. It is clear that much of the progress will be sustainable (e.g., personnel trained, services improved, and behaviors changed), but the overall degree of sustainability will depend in large part on addressing the issues raised in this report and developing a realistic exit strategy. 4 TERMS OF REFERENCE FOR THE MIDTERM EVALUATION The midterm evaluation of the Tubaramure program was designed as a qualitative exercise. It took place over a month-long period in May-June 2012 and was carried out by a team composed of three external consultants: a Public Health/Nutrition Evaluator who was also the team leader (Kathy Tilford); a Commodity Supply Chain Specialist (Ange Tingbo); and a Primary Health Care Expert (Dr. Vera Bensmann). The Burundian Ministry of Health (MoH) appointed an Expert, Dr. Térence Hatorimana as an observer; he was with the team in the field through June 9 and provided valuable insights and background information. See Annex A for each consultant’s individual program and Annex B for the Terms of Reference (TOR) for the MoH Expert. The overall objectives of the Tubaramure midterm evaluation were to assess achievements against targets; to determine progress against the strategic objective; to identify the program’s strengths and weaknesses from inception to mid-term; and to provide recommendations for improving implementation. Specifically, the TOR presented in Annex C asked the team to look at four evaluation areas:  Relevance: Investigate in detail the extent to which the objectives of the program are consistent with the needs of the beneficiaries, the recipients, the host country and donor.  Effectiveness: Look into whether or not the proposed program strategy will achieve its long term goals. Verify whether the program is meeting targets. .  Efficiency: Examine how economic inputs (resources, expertise, time, etc.) are converted into outputs.  Sustainability: Assess the likelihood that the positive effects of the Preventing Malnutrition in Children Under 2 Approach (PM2A) such as assets, skills, facilities or improved services will persist for an extended period after the end of the financial assistance. Included in the TOR were individual sets of questions for each consultant to use to guide the work. These questions have been answered in the main body of the report and summary responses are also provided in Annex D. In addition to the TOR prepared by the Tubaramure consortium, USAID and Food for Peace (FFP) proposed four additional points for examination: 1. Beneficiaries take home four Corn Soy Blend(CSB)/vegetable oil rations for their family members over and above their own CSB/vegoil ration. We would like to know how/whether/why these additional take-home rations for family members contribute to the program’s objectives. 2. What are the opportunity costs in making this investment in family food rations in light of other potential investments? 3. Beyond dietary diversity, is the program considering recommending the substitution of local foods for the CSB/vegoil rations once the program ends (or as a phasing out of the program during its second half)? 5 4. Does the program have a strategy to replicate lessons learned? Questions 1-3 are addressed in the chapter on Intermediate Result 3 (IR-3). Question 4 is discussed in the chapter on lessons learned. Given the qualitative nature of the evaluation, the team relied on a methodology that stressed individual interviews and guided group discussions with beneficiaries, program staff and partners at all levels. See Annex E for a list of Key Informants. Observations of ongoing activities as well as visits to health facilities, warehouses, food distribution sites and households gave additional insight into how the program was performing. A thorough review and analysis of the documents listed in Annex F provided valuable information on the history of the Tubaramure program and the Burundian context. Although this was a qualitative evaluation, the team did review the results of the quantitative anthropometric survey carried out by the Institut de Statistiques et d’Etudes Economiques du Burundi (ISTEEBU) in the program area in May-June 2012. This survey measured 928 children of whom 838 were 6-59 months: weight for age, weight for height, and height for age. It is the first large-scale anthropometric survey in the Tubaramure area since the baseline survey.1 The following report contains both recommendations and suggestions. The recommendations are considered more important for success and to the extent that the consortium concurs with them, should be addressed as soon as possible during the second half of the program. OVERVIEW OF THE TUBARAMURE PM2A PROJECT Catholic Relief Services/Burundi (CRS) is the lead agency for this five-year USAID-financed Multi-Year Assistance Program (MYAP) entitled Tubaramure (“Let’s Help Them Grow” in Kirundi) with sub-awards to International Medical Corps (IMC), Food for the Hungry (FH) and CED-Caritas/Burundi, hereinafter referred to as Caritas. The program is built around PM2A, an approach that was first identified by the International Food Policy Research Institute (IFPRI) et al.2 as more effective in reducing malnutrition than historically favored remedial methods. The objectives of the Tubaramure program are to improve the health and nutritional status of pregnant and lactating women and children less than two years of age and to strengthen the quality and delivery of health care services. The program is being implemented in a total of 268 collines (literally, hills) in the 12 communes of Cankuzo and Ruyigi Provinces. See Annex G for 1 Enquête Anthropometrique en Provinces Cankuzo et Ruyigi: Evaluation du programme PM2A￾Tubaramure; ISTEEBU. June 2012. 2 The International Food Policy Research Institute (IFPRI), in collaboration with World Vision￾Haiti, Cornell University, and the Food and Nutrition Technical Assistance Project (FANTA), provided the first programmatic evidence that the blanket targeting of a food assisted Maternal and Child Health and Nutrition program to all children 6-24 months old (preventive approach) was more effective in reducing the prevalence of stunting, wasting, and being underweight than the traditional recuperative approach based on targeting underweight children under five years of age (Ruel et al. 2008). 6 map.) Table 1 below summarizes the three Intermediate Results (IR) and the planned outputs for each IR. Table 1. Framework for the Tubaramure program (from proposal) Strategic Objective: Malnutrition in children under 2 years of age is prevented Intermediate Results (IR) Outputs 1. Women and children under 5 access quality nutrition and health services. Technical Lead: IMC 1.1: Pregnant and lactating women access pre and postnatal care services 1.2: Implementation of national IMCI plan is supported 1.3: Health facilities supported in providing Growth Monitoring (GM) 1.4: Severe Acute Malnutrition (SAM) is detected and referred for treatment 2. Households practice appropriate health and nutrition behaviors. Technical Lead: FH 2.1: Households (HH) adopt Essential Nutrition Actions (ENA) 2.2: HHs adopt Essential Hygiene Actions (EHA) 2.3: HHs adopt prevention and management behaviors for maternal and childhood illnesses 3. Eligible women and children have increased intake of nutrient-rich, diverse foods. Lead: CRS in partnership with Caritas 3.1: FFP rations distributed to eligible women and children at community level 3.2: Mothers and children use FFP rations appropriately 3.3: HHs use appropriate local foods in addition to FFP ration Program activities include: improving the capacity of health facilities/staff and Community Health Workers (CHWs) to deliver high quality general health and nutrition services; promoting household and community-level behaviors that prevent maternal and childhood illnesses and support good nutrition using Care Groups (CGs) as the delivery mechanism; and improving the caloric and nutrient intake of mothers and children under two through culinary demonstrations and monthly delivery of Title II commodities. Approximately 51,075 mother-child pairs are targeted over the five years (July 2009 to October 2014) of the $45 million grant. By April 2012 the program had already reached 49,652 mother-child pairs. Integrated into the Tubaramure program is a research program conducted by IFPRI with funding from USAID through the Food and Nutrition Technical Assistance II (FANTA-2) and FANTA-2 Bridge projects. IFPRI is conducting a series of studies in 60 collines to assess the impact and cost effectiveness of Tubaramure on child nutritional status, as well as to evaluate the differential and absolute impact of varying the duration of receiving food rations. The following table shows the principal implementers at the community level: Table 2. Core actors at the health facility and community level (adapted from proposal) 7 Actor Job Description Roles and responsibilities Core Field Staff/Volunteers Leader Mothers (LMs) Are 4,866 as of June 2012 Selected by households (1 for every 10-15 households) based on trust, respect, leadership skills and mothering knowledge. Form in groups of 10-12 into a Care Group led and trained by a THP in techniques for promoting behavior change in ENA, EHA and early detection and referral to pre-post natal services, referral and proper home management of childhood illnesses. Care Group (CG) Are 425 as of June 2012 Groups of Leader Mothers trained by the Tubaramure Health Promoters. Carry out peer to peer activities and BCC at community level and through home visits; work with CHW in referring cases. Tubaramure Health Promoters (THP) Are 40 as of June 2012 Commune-level CRS staff. Coordinate commune-level project activities; provide training to CG, LMs, CHWs; supervised by the Care Group Supervisors (CGS) and report to the CRS head of office in each province Care Group Supervisors FH staff Supervise the THPs; provide training; ensure that the CGs are adequately supported and that the BCC strategy is implemented according to plan. IMC Technical Staff IMC staff Based in each provincial office, these four￾member teams provide the technical expertise for IR-1; an IMC site manager coordinates each team and reports to the IMC technical lead in Bujumbura. Tubaramure Provincial Technical Advisors In each province: Tubaramure technical team composed of BCC supervisors (FH); general health focal point (IMC); and food utilization focal point (CRS). Supervised by the Provincial Coordinator. Responsible for supporting THPs to deliver high quality PM2A interventions; coordinate with provincial MoH and other stakeholders in planning and implementing training, coordinating activities; participate in yearly programming reviews and planning; supervised by Tubaramure technical leads Health Workers MoH Health Facility Staff Doctors at the hospital level; nurses at the health centers; mid-wives; provincial and district medical coordinators; Public Health Technicians (TPS) Receive specific trainings; support referral processes; engaged in M&E, planning and review; participate in surveys or other assessment activities Community Health Workers (CHWs) Community level volunteers sponsored by MOH, trained by the TPS to support basic health services at the community level. MOH protocol recommends 1 CHW for every 10-15 households. Carry out consultations with households; refer cases for service; support Care Group; receive training and support from project Local Authorities 8 Heads of collines Elected officials responsible for administration Organize community meetings; support Leader Mothers; serve as conduit between Tubaramure and their constituency The principal Government of Burundi (GoB) partners in the program include:  The MoH at the national, provincial, district and health facility level  The government administration, especially at the provincial, commune, colline and zone levels  The Ministry of Agriculture and Animal Husbandry (MAE) at the provincial, commune, colline and zone levels INTERMEDIATE RESULT 1: WOMEN AND CHILDREN UNDER 5 ACCESS QUALITY NUTRITION AND HEALTH SERVICES 1. Women and children under 5 access quality nutrition and health services. Technical Lead: IMC Output 1.1: Pregnant and lactating women access pre and postnatal care services Output 1.2: Implementation of national IMCI plan is supported Output 1.3: Health facilities supported in providing Growth Monitoring (GM) Output 1.4: Severe Acute Malnutrition (SAM) is detected and referred for treatment IMC is responsible for the implementation of the first intermediate result: “Women and children under 5 access quality nutrition and health services”. IR-1 covers 50 health centers, 23 in Cankuzo and 27 in Ruyigi. Initially there were 40 health centers (16 in Cankuzo and 24 in Ruyigi), but since the start of the program the government has added 10 health centers to the two provinces. I. IR-1 ACTIVITIES For IR-1 the program activities have been divided into 3 axes: training of personnel, material support, and monitoring and evaluation (M&E). A. Training of Personnel The program facilitates and supports the training of MoH personnel and CHWs, using the MoH’s available protocols, trainers, and training materials. Where modules and materials were not yet available (e.g., for training in postnatal consultations), IMC-Tubaramure worked with MOH colleagues to develop the module and materials. Training modules have been tested, validated and accredited by the MoH. Topics for training have been: 9 - Nutrition and Malnutrition: For the nurses, doctors and clinical assistants, the training covered the management of malnutrition and how to conduct therapeutic feeding services for severely malnourished children. For the Techniciens de Promotion de la Santé (Public Health Technicians or TPS), the CHWs, and the THPs, the emphasis of the training has been on identifying malnourished children in the community and referring them to the health center. All groups were also taught the basics of good nutrition practices and how to teach mothers to provide wholesome nutrition with the foods that are available locally. - Prenatal and Postnatal Consultations: For the nurses this was a clinical training, focused in depth on the danger signs and risks in pregnancy and childbirth. For the THPs and CHWs it was more basic, aimed at letting pregnant women know the importance of regular check-ups and why they need to get their vaccinations and supplements during pregnancy and after childbirth. - Growth Monitoring: In this course health workers not only learned how to do and interpret the anthropometric measures of a child (CHWs learned how to measure the mid-upper arm circumference or MUAC, while health center staff and TPS learned to use the scales and measuring boards), but were also taught the basics of nutrition and signs of malnutrition. Trainees learned to check whether children had been vaccinated and de-wormed and if they had received any micro-nutrient supplements. They were also taught how to give a presentation in the community. For the THPs there was an additional class for community cooking demonstrations. - Clinical Integrated Management of Childhood Illnesses (IMCI): This is a course on the integrated management of newborn and childhood illnesses, which comes with a set of algorithms for the observation and clinical examination of a child and questions for the mother to quickly recognize danger signs when a child comes with cough, diarrhea, fever, etc. The training is quite elaborate and requires previous medical schooling. - Community IMCI: The program has only just started with this course for CHWs and TPS. Community workers are taught simple and basic messages such as how to cool a child with fever, how and when to give oral rehydration, and when to refer a child to the health center. Table 3. Number of people trained by April 2012 TOPIC: Nutrition and malnutrition Pre- & postnatal consultation Growth monitoring Clinical IMCI Community IMCI Nurse 47 104 105 113 9 Clinical assistant 10 0 71 0 0 Public Health Technician (TPS) 22 22 22 0 14 CHW 534 534 534 0 534 Doctor 0 0 0 0 2 Tubaramure Health Promoter 35 35 35 0 35 TOTAL: 648 695 767 113 594 People that have been trained include: 10 - Nurses: The norm established by the MoH is to have six nurses in each health center, but in Cankuzo and Ruyigi Provinces there are only two or three. A minimum of two nurses have been trained for each health center, though many of these were transferred to other provinces after they completed the training. Since the program began, the MoH has added 10 new health centers to the two provinces. Additional training courses for the new nurses have already been scheduled in the Detailed Implementation Plan (DIP) for 2012. - Clinical assistants: These are paramedics (aides soignants in French) without any medical schooling. Most clinical assistants have completed secondary school and perform simple tasks in the health center under the supervision of the nurses. - Public Health Technicians or TPS: The TPS are employed by the MoH to manage a group of community health workers and to work in all aspects of public health. There are two TPS per commune, which means that each TPS is affiliated with two or three health centers. Most of the TPS are trained nurses who have chosen a career in public health rather than in clinical medicine. (Note: Only 14 TPS have been trained in community-IMCI by the program because the eight TPS in Cankuzo Province received community-IMCI training from UNICEF.) - Community Health Workers (CHWs): These “Agents de Santé Communautaire” started working as volunteers for the MoH in 2003 and have participated in many ‘vertical’ programs. They are called upon for national immunization days, for campaigns to distribute mosquito nets, for HIV awareness raising, etc. The CHWs receive a Fanta and some money for transport during training sessions, but otherwise they work for free. - Doctors: The pediatricians at the hospitals were trained through the program, but have since left. The doctors within the health administration (the médecins chefs des districts and médecins chefs des provinces) are very much involved in the program. They have participated in several courses; they often join on supervision rounds; and they conduct the quarterly coordination meetings. - Tubaramure Health Promoters: The THPs (Promoteurs de Santé et Nutrition or PSN in French) are salaried Tubaramure outreach workers who work across all three IRs with a particular focus on supporting the LMs and the BCC activities. The work of the THPs is very similar to that of the MoH’s TPS as they manage the LMs in a similar fashion as the TPS manage the CHWs. Therefore the TPS and the Tubaramure Health Promoters have received the same training modules through IR-1 (and of course they have a whole additional training schedule through the program in IR-2). All the health workers that were interviewed were happy with the training modules and indicated they would like to attend a refresher course. It remains to be seen whether a refresher course is really necessary as they appeared to have retained the knowledge from their schooling and the trainings. All modules use a pre-test and post-test to gauge the trainees’ knowledge before and directly after the training. Although the joint supervision activities capture the training uptake by health workers, it would be interesting to conduct a test among all the nurses in the health facilities, similar to the post-test of the training module, so that one can see 1) whether those trained have retained the knowledge and 2) the level of knowledge of those who have not yet received any training through the program The training of CHWs has only started recently, and their training is very basic. Simple and concise messages are used in their training and it was interesting to see that CHWs from different 11 areas all used the same hand motions and gestures explain how one should cool a child with fever. For the CHWs to actually use their knowledge it is important to find some kind of incentive for them to go out into the community and visit more children in their homes. The CHWs themselves suggested they would like to have their own visual aids (pictorials) to take into the communities and homes, instead of borrowing the large and heavy image box (boîte à images) from the health center. Furthermore they suggested they should receive T-shirts and/or caps to make them recognizable; boots to facilitate access to muddy areas; and bicycles to get from the colline to the health center. (IMC-Tubaramure is already providing T-shirts, bags, registers and pens.) The CHWs noted that they would also like more training. B. Material Support to the MoH At the start of Tubaramure, the program provided the health centers with basic materials and equipment such as stethoscopes, scales, washbasins, measuring boards, cups, plates, buckets, etc. All the equipment that was donated is listed in the Burundian standard requirements for the inventory of a health center. An inventory of all centers was made at the start of the program and the health centers made a request to their district health department for the things they needed. The requests were evaluated and the equipment was given to the two provincial health departments who distributed the items to the respective health centers. An example of the materials provided can be found in Annex H. The program also occasionally supplies the health centers with medicines. These are basic antibiotics, anti-malarials, painkillers, deworming agents, etc. All are on the WHO essential medicines list and approved for use in Burundi. An example of a donation can be found in Annex I. The frequency and the quantity of the donations depend on the gifts in kind that the program receives from overseas. All donations are given to the provincial health departments, which distribute them to the health centers according to need. They are then disbursed to pregnant women and children under five free of charge. The items given are much appreciated by the centers and staff know how to use the equipment. All items are being used properly except for some otoscopes that were delivered without batteries. The spoons that were given for the supplementary feeding centers are not being used as the supplementary feeding centers closed soon after the start of the program. Besides the donations of drugs and equipment, IMC-Tubaramure also provides support in terms of fuel. Fuel is provided 1) to the ambulances for the transport of severely malnourished children to the hospitals; 2) to the provincial and district health departments during their joint monthly supervision; and 3) for transport to coordination meetings. The program also provides food for the caregivers (usually the mother) of severely malnourished children who are admitted in the hospitals’ nutrition stabilization centers. The hospital provides the food and is reimbursed by the program. C. Monitoring and Evaluation The program has created a thorough system to supervise and follow up the IR-1 component, which consists of the following three types of supervision and coordination. C.1. Supportive Supervision 12 The technical people within IMC’s teams in Cankuzo and Ruyigi each have their own specialty within the program for training on-the-job. The IMC-Tubaramure supervisors look at activities for 1) therapeutic feeding, 2) growth monitoring, 3) pre-and post natal consultations, and 4) community outreach. In each province an IMC Site Manager oversee all activities and focuses on quality implementation of the IMCI modules. During the support visits IMC-Tubaramure collects monthly data on malnutrition; growth monitoring; IMCI; pre-and postnatal consultations; and the number of cases of malaria, diarrhea and acute respiratory infections in children under five. Often a topic is chosen to discuss with the health center workers during the supervision. C.2. Joint Supervision IMC-Tubaramure staff make monthly visits to all health centers, together with the supervisors of the district and provincial health departments. Sometimes the health directors (the médecins chefs des districts and médecins chefs des provinces) join in as well. All health centers need to compile their monthly data before these visits and hand in their summary sheets. All health centers have at least14 different registers (ledgers) to record their activities: 1. Register for children under five (contains personal data, signs, examination done, diagnosis, therapy) 2. Register for adults and children five years and older 3. Entry Register (for everyone who comes into the center) 4. Hospitalization Register (for those who stay overnight) 5. Birth Register 6. Death Register 7. Register for Prenatal Consultations 8. Delivery Register 9. Register for Postnatal Consultations 10. Growth Monitoring Register 11. Register for Out-patient Nutrition Services 12. Vaccination Register 13. HIV Testing Register 14. Family Planning Register In addition, there are numerous charts (fiches) used to compile weekly and monthly data from the different registers. During the MoH supervision visits, a 15-page form3 is used to record 1) population data; 2) curative activities; 3) prevention activities; 4) outreach activities; 5) supervisory visits; 6) resource management (medicines, vaccines, laboratory reagents, material and equipment, staff, and finances); and 7) morbidity and mortality. Findings from these joint visits are discussed during the quarterly coordination meetings. C.3. Coordination Meetings 3 SIS Burundi: Rapport Mensuel du Centre de Santé 13 These consist of 1) quarterly meetings with the MoH in which all ‘vertical’ programs are discussed (i.e. malaria, Expanded Program on Immunization [EPI], malnutrition, family planning, HIV, etc.); 2) quarterly meetings with the COSAs (the Comités de Santé or Health Committees that consist of community members and health center staff); and 3) internal meetings within IMC-Tubaramure to exchange information between the two provinces and to discuss progress; and 4) meetings with the other consortium members. The program has done a good job of using the supervisory visits and all the data collected to adjust and modify the IR-1 implementation plan. For example, when program staff saw that the MoH’s district supervisors focused mostly on the administrative tasks of the health centers and did not spend adequate time on clinical skills and patient outcomes, they decided to organize a workshop for all supervisors and their bosses (the médecins chefs des districts and médecins chefs des provinces), to improve the quality of supervisory visits. At this point in time one can say that the program staff has a very good understanding of how they are performing within IR-1. However, with the existing data collected at the health center level it is not possible to look specifically at malnutrition prevented in children under two. The age brackets for reporting morbidity and mortality within the MoH system are 0-11 months, 12- 59 months, 5-14 years, and 15 years and above. Furthermore, it is not possible to see in the data from the health centers whether a pregnant woman or a child is enrolled in the Tubaramure PM2A program or not. At the health centers one cannot see which mothers and children are receiving food rations and are part of a CG. If at the health center one could distinguish between those enrolled in the program and those not enrolled, it would be easier to attribute results and to provide feedback for the community component of IR-2. To overcome this problem, the program has been conducting performance surveys on all Intermediate Results every three months using a random sample of beneficiaries and health centers. These quarterly surveys to monitor the performance of the whole program are now being replaced by semi-annual surveys to measure the performance towards the strategic objective (malnutrition prevented). This will include an anthropometric study of a random sample of children conducted every six months by. ISTEEBU implemented the first of these anthropometric studies in June 2012. II. PROGRESS TOWARD PROGRAM OUTCOMES AND OUTPUTS This section on program outcomes and outputs for IR-1 is divided into two parts: 1) a discussion of the four performance indicators for measuring outcomes and 2) a review of the four outputs and their respective indicators. A. Performance Indicators for Measuring Outcomes Table 4 below shows progress for the outcome indicators at the end of FY2011 and at the end of the first semester of FY2012 (March 2012). Table 4. Tracking Outcome Indicators 14 Minimal package of prenatal visits includes: vitamin A and iron supplementation, weight gain monitoring, and at least 3 prenatal visits. Minimal package of postnatal visits includes: vitamin A and iron supplementation, and at least 2 postnatal visits. IR-1: Women and children under 5 access quality nutrition and health services Baseline Target FY2011 Achieved FY2011 Achieved Q2 2012 Original indicator Indicator 1.1. % of pregnant women completing package of prenatal visits 29% 98% 61% 61% 94% Indicator 1.2. % of mothers completing package of postnatal visits 5% 50% 8% 12% 68% Indicator 1.3. % of children 0-59 months attending growth monitoring at least once in a two-month period (as recorded on card) 16% 65% 61% 63% NA Indicator 1.4. % of health providers (facilities/ CHW) accurately assessing a child using IMCI protocols 0% 40% 88% NA NA A.1. Indictors 1.1.and 1.2.: % of pregnant women completing package of prenatal and postnatal visits: During the course of program implementation the indicators for pre- and postnatal consultations were modified when the definition of the “package” was changed. In the original proposal (Appendix 20: Performance Management Plan or PMP) the “packages” of pre- and postnatal visits were defined as follows: The last column in Table 4 above shows the performance for the first semester of 2012, using the original indicators, i.e., three prenatal visits and two postnatal visits. According to the national protocol, women are supposed to get a dose of vitamin A and iron supplements once during pregnancy and once after delivery. It is marked in the consultation register when these supplements are given. However, it is not recorded in the compiled monthly data for pre-and postnatal consultations and therefore hard to measure. And according to the national protocol, pregnant women are supposed to complete four prenatal and three postnatal visits. These two performance indicators were changed during the course of the program to follow the national recommendations. Despite increased efforts, only 61% of pregnant women are completing four prenatal visits and only 12% are completing three postnatal visits. A suggestion was made by the Tubaramure team to lower the end of program targets. However, it is better to keep the targets high (i.e., at 98% for prenatal consultations and 75% for postnatal consultations) and revert instead to the original indicators of three prenatal and two postnatal visits. Most of the women register for prenatal services in the second trimester of their pregnancy, which leaves sufficient time to complete three visits. If one should insist on four visits, efforts should be made to capture women in their first trimester. 15 For the first two Outcome Indicators the recommendation is to leave out the word “package” and to change the wording of the indicators as follows: 1.1. % of pregnant women completing 3 prenatal visits 1.2. % of mothers completing 2 postnatal visits For this outcome indicator the recommendation is to change the age to 36 months: 1.3. % of children 0-36 months attending growth monitoring at least once in a two￾month period (as recorded on card) According to the health center staff, most women now deliver at the health center and have their first postnatal check-up upon exit. And many women who deliver at home come to the clinic within three days of delivery for their first postnatal consultation. If everything was all right during this first visit and they don’t have any complaints, most women do not see the need for a second or third postnatal consultation. However, almost all these women do come to the clinic with their babies to receive the first immunization (BCG); this would be a good opportunity to capture these women for their second postnatal consultation. It should also be noted that the prenatal and postnatal consultations are a requirement for women to remain enrolled in the Tubaramure program. Attendance is good: 94% of pregnant women complete three or more prenatal visits and 68% of the new mothers come for two postnatal consultations. In the coming year one can see whether these rates will be sustained, as there will be no new enrollments into the Tubaramure program. To ensure the quality of these pre- and postnatal visits, it would be good to check the registers of the health centers regularly to see whether supplements are given and to monitor 1) whether women are weighed during pregnancy, 2) if they receive a mosquito-net, and 3) if they get their tetanus toxoid vaccination (all three are requirements in the national protocol). The information gathered will be useful for advocacy purposes to stimulate the MoH to comply with its own protocols. The data collected will not influence the measurement of the Tubaramure program performance. A.2. Indicator 1.3: % of children 0-59 months attending growth monitoring at least once in a two-month period (as recorded on card) This indicator currently measures the number of children 0-59 months participating in monthly growth monitoring divided by the total number of children 0-59 months in the target population. However, growth monitoring is only done up to 36 months. This means that the program is actually achieving a better result than is shown in the performance tracker because they are only measuring children under three, while using the population of children under five as the denominator. To further increase the number of children attending growth monitoring in their first year, it would be good to combine growth monitoring activities with routine immunizations. Children come for BCG and Polio-0 in the first week; for Pentavalent (DTP-HepB-Hib) and Polio-1-2-3 in weeks 6, 10 and 14; and Measles at 9 months. At the moment many clinics have their 16 For this Outcome Indicator the recommendation is to change the wording as follows: 1.4. % of nurses accurately diagnosing and treating children under five vaccination days on Tuesdays and Thursdays, while growth monitoring is done on a Wednesday. If growth monitoring was conducted during vaccination days, mothers would not need to come to the health center twice in the same month. A.3. Indicator 1.4: % of health providers (facilities/CHW) accurately assessing a child using IMCI protocols Although the indicator mentions percentage of health providers accurately assessing a child, what is actually measured is the number of health facilities implementing IMCI protocols, divided by the total number of health facilities in the target area (See Appendix 20 for the PMP of the original proposal). At the moment the program measures the number of health centers that apply the IMCI protocol. The criteria used are whether staff can produce the protocol (i.e., the IMCI manual with its algorithms) and if there is someone in the health center who knows how to use the protocol. If this were indeed the purpose of this indicator, the target should be that all the health centers have the IMCI protocol and have at least one person who knows how to use it. It would be more useful to know how many nurses can accurately diagnose and treat children under five. However, this is a lot more difficult to measure than measuring the number of health centers that apply the IMCI protocol. Other ways to obtain this information: One could think of doing a test (similar to the post-test used during the training modules) at semi-annual intervals. Or one could look at the number of nurses who have correctly completed their five algorithms out of all the nurses in the 50 clinics. This could be done during the supportive supervision visits but may be quite cumbersome to measure. At the end of FY 2011, 40 health facilities were implementing IMCI protocols, as scheduled initially, though this was divided by the total number of health facilities in the target area at the end of FY2011 (which was then 46). With the increase in the denominator from 40 to 46 health centers, the result was 22% lower. Since the end of FY2011, another four health centers have been added to the target area and the denominator is now 50 centers. It is very good that the MoH keeps adding health centers and the MoH should include the staff in the new centers in their training. Hence, a repeat course for clinical IMCI has already been scheduled for 2012. However, if the MoH keeps adding more centers and the denominator keeps increasing, this may look as if the program is performing poorly, which it is not. Furthermore, the program has trained a huge number of CHWs, which was supposed to be reflected in this indicator as well: The original indicator says: “% of health providers (facilities/CHW)...” However, the CHWs’ performance is not being measured here. As Indicator 1.7 already measures the number of health centers using IMCI and Indicator 1.8 measures the number of CHWs trained, it is recommended that Indicator 1.4 measure the percentage of nurses who can accurately diagnose and treat a child. 17 During the evaluation the nurses in the health centers seemed very competent at diagnosing and treating children with the most common illnesses such as malaria, acute respiratory infections and diarrhea. However, not all nurses understand how to use the algorithms and questionnaires that are provided with the IMCI module. But as long as the nurses diagnose and treat children correctly, one can wonder whether these IMCI algorithms and questionnaires are really necessary. At the present time the MoH supervisors have decided that the nurses only need to complete all the IMCI paperwork for five children per day. This is to help the nurses to practice using the forms and to understand the reasoning within the algorithms. B. Progress on Indicators for Measuring Outputs The table below shows progress on achieving the four IR-1 outputs at the end of FY 2011 and for the first semester of FY2012. Table 5. Tracking Output Indicators Indicator Baseline Target FY2011 Achieved FY2011 Achieved Q2 of FY2012 Target end of Program Output I.1: Pregnant and lactating women access pre- and postnatal services Indicator 1.5. % increase of women registered for prenatal services by the sixth month of pregnancy 80.1% 95% 94% 96% 95% Indicator 1.6. % of health facilities with 2 or more staff who completed in-service training in pre-post natal services 0% 40% 42% NA 75% Output I.2: Implementation of national IMCI plan is supported Indicator 1.7. % of health facilities with two staff members trained in IMCI protocol through MOH's IMCI office 15% 70% 48% NA 100% Indicator 1.8. % of trained CHW in IMCI in target areas through MOH's IMCI office 0% 40% 99% NA 70% Output I.3: Health facilities supported in providing growth monitoring Indicator 1.9. Number of communes with at least one functioning community based growth monitoring center after year 3 0 NA NA 6 (+12) Indicator 1.10. % of health facilities with upgraded growth monitoring equipment. 0% 40% 100% NA 75% Output I.4: SAM is detected and referred for treatment Indicator 1.11. % children referred to CMAM actually admitted for treatment 72.7% 85% 96% NA 85% 18 For Output Indicator 1.5 the recommendation is to take out the word ‘increase’: 1.5. % of women registered for prenatal services by the sixth month of pregnancy Output Indicator 1.7 can remain unchanged: 1.7. % of health facilities with two staff members trained in IMCI protocol through MOH's IMCI office Output Indicator 1.6 can remain unchanged: 1.6. % of health facilities with 2 or more staff who completed in-service training in pre￾post natal services Indicator Baseline Target FY2011 Achieved FY2011 Achieved Q2 of FY2012 Target end of Program Indicator 1.12. % of children recovered from acute malnutrition being followed by the CHWs or LM. 7.1% 60% 0% NA 90% B.1. Output 1.1: Pregnant and lactating women access pre- and postnatal services The program is performing very well on this indicator; with an achievement of 96% it has exceeded its end of program target. However, one should note here that the program is measuring the number of women who are registered for prenatal services by their sixth month of pregnancy out of all the pregnant women who attend prenatal consultations. In other words the denominator is the number of women who are in the health centers’ prenatal consultation registers. It would be more interesting to know how many pregnant women are registered for prenatal services by their sixth month of pregnancy out of all the pregnant women in the target area. The denominator could be the number of women who are in the health centers’ delivery register (or the birth register). The program is on target for this indicator: At the end of FY2011, 19 out of the then 46 health centers (42%) had two or more nurses who had completed the course on pre-and postnatal consultations. As explained for Indicator 1.4, this was meant to be 19 out of 40 health centers and the denominator for FY2012 has already increased to 50 health centers. With the expanded denominator and with the high turnover of nurses in the two provinces, it may be a challenge to reach the 75% proposed for the end of project target, i.e., 38 out of 50 health centers. However, the team recognizes this and prefers to keep the indicator and the target high in order to stimulate the MoH to continue to train more nurses. The MoH is already looking into the problem of high staff turnover: At the moment they are decentralizing their hiring process for health center staff, which means nurses will no longer be hired centrally and sent to places they don’t want to go to. Instead they will be hired at the district level and will stay there without frequent transfers to other provinces. B.2. Output 1.2: Implementation of national IMCI plan is supported 19 Output Indicator 1.9 can remain unchanged: 1.9. Number of communes with at least one functioning community based growth￾monitoring center after year 3 For Output Indicator 1.8 the recommendation is to change the wording as follows: 1.8. % of collines with 2 or more CHWs trained in IMCI in target areas through MOH's IMCI office The program is behind on this indicator: At the end of FY2011 22 out of the then 46 health centers (48%) had two or more nurses who completed the course on IMCI. The reasons for not achieving the FY2011 target are two-fold: the denominator has increased from 40 to 50 health centers (22 out of 40 would have been an achievement of 55%) and there has been a high turnover of nurses who were trained. (Note: The courses on IMCI started in FY2010 whereas those on pre-and postnatal consultations started in FY2011 so more nurses who have done the IMCI course have left after they have been trained.) The new nursing schools in Cankuzo and Ruyigi will have their first graduates next year. In theory IMCI is now part of the regular nursing curriculum so hopefully there will be 50 newly graduated nurses with IMCI skills coming into each of the two provinces. It may be a good idea to check whether the MoH’s IMCI office indeed provides training through the nursing schools and whether they would need support from the program. The program has exceeded its target for this indicator: Almost all (534) of the current Community Health Workers have been trained thus far, which is about two per colline (there are 268 collines in the two provinces). The indicator for this output is currently measured by the number of CHWs trained in IMCI, divided by the total number of CHWs in the target areas. According to the national public health policy, there should be three CHWs per colline and the MoH has pledged to train more CHWs to meet this norm. As the program has only recently started the training of CHWs in community-IMCI, it may not be possible to train an additional 268 people within the current time-frame and budget. The additional CHWs are needed to cover their target areas but if the denominator increases, this should not reflect negatively on IMC￾Tubaramure’s performance. B.3. Output 1.3: Heath facilities supported in provide growth monitoring The program is on target for this indicator. The activities for this output were only started this year and already six out of 12 communes have functioning community-based growth monitoring centers. During these activities the CHWs give a short talk about nutrition before the TPS and the IMC supervisor start weighing and recording all children. Output Indicator 1.10 can remain unchanged: 1.10. % of health facilities with upgraded growth monitoring equipment. 20 For Output 1.4 the recommendation is to change the wording as follows: Malnourished children are detected and followed-up The program has exceeded its target for this indicator: All 50 health centers have upgraded equipment for growth monitoring. B.4. Output 1.4: SAM is detected and referred for treatment Children with Severe and Acute Malnutrition (SAM) are detected through the growth monitoring activities and through the increased efforts of CHWs; they are referred to the health centers for follow-up and treatment. However, children with moderate or mild malnutrition are no longer covered. It is important to mention here that one of the critical assumptions of the program design was that “UNICEF and the MoH will provide CMAM in the project area.” Since the supplementary feeding centers were closed, there has been no more monitoring, registration or follow-up of children who are moderately or mildly malnourished. There should be some kind of follow-up for children who are moderately or mildly malnourished to prevent them from becoming severely malnourished. In the growth monitoring data provided by IMC’s M&E Specialist for January 2011 to April 2012, one can see a monthly average of about 1,100 children (11% of all children measured) who are marked in the “yellow zone”, indicating moderate or mild malnutrition, or children at risk for SAM. The graph below is a representation of these children by age group. Figure 1. Results from Growth Monitoring from January 2011-April 2012 Unfortunately it is not possible with the existing data to distinguish between the 50,000 children in the Tubaramure program (i.e. children under two who receive CSB and vegetable oil (VO), with mothers receiving food, nutritional advice and hygienic measures), and those who are not. Not all children under two are benefitting from the full PM2A program because there are 60 collines in the IFPRI study and children enroll into and exit from the program at different times. Enrollment starts with pregnancy for the mother and children benefit from the full PM2A 21 For Output Indicator 1.11 the recommendation is to change the wording as follows: 1.11. % children with malnutrition referred from the community who are enrolled in nutrition services program until they exit at their second birthday. As not all babies are born at the same time, there is not one fixed age cohort one can follow: i.e. there will be children in the 25-36 month age group who have already ‘graduated’ and as of this year there will be babies born after the closing date for enrollment. For this indicator the program measures the number of children enrolled in the nutritional services, divided by the total number of children who were referred from the community to these services. At the moment only children with severe acute malnutrition (SAM) are referred and 96% of these children were enrolled in the outpatient therapeutic feeding services. The program should ensure that children with moderate or mild malnutrition are also identified, registered and followed up in the community as was planned in the original proposal. (See Tables 6 and 7 below.) Table 6. Community screening of moderate and mild malnutrition (from original proposal) Presenting with moderate or mild malnutrition not receiving PM2A supplements Presenting with moderate or mild malnutrition enrolled in Tubaramure (Mother or child had received PM2A supplements) Children 0 to 24 months with moderate or mild malnutrition at start of Tubaramure (Mothers were not pregnant or lactating at project start-up, therefore the children are not enrolled)  detection  home visits by CHWs and LMs with specific messages for catch-up feeding  C-IMCI screening and referral to health facility (HF) for other services including CMAM Children 0-24 months with moderate or mild malnutrition (Mother started in Tubaramure at time of pregnancy)  detection  home visits by CHWs and LMs  Use of Title II food - Use of additional foods during illness - Counseling on hygiene - Counseling on home management of illnesses  C-IMCI screening and referral to other health services Children 25 to 59 months with moderate or mild malnutrition at start of Tubaramure)  detection  referral for CMAM  home visits by CHWs and LMs with specific messages for catch-up feeding  C-IMCI screening and referral to HF for other services Tubaramure graduates 25 to 59 months with mild or moderate malnutrition  detection  home visits by CHWs and LMs - Reinforcing learning from Tubaramure - Use of additional foods during illness - Counseling on hygiene and home management of illnesses 22 Table 7. Community-based detection and referral of SAM (from original proposal) Children with SAM not receiving supplements Children with SAM enrolled in Tubaramure (mother or child has received supplements) Children +6 to 24 months with moderate or mild malnutrition at start of Tubaramure  detection  home visits by CHWs and LMs with specific messages for catch-up feeding  C-IMCI screening and referral to HF for other services including CMAM Children 0-24 months with SAM (Mother started in Tubaramure at time of pregnancy)  detection  referral for treatment  follow up through CHWs and LMs Children 25 to 59 months with SAM at start of Tubaramure  detection  referral for treatment  follow up through CHW and LM Tubaramure graduates 25 to 59 months with SAM  detection  referral for treatment  follow up through CHWs and LMs The program uses the acronym CMAM (Community Management of Acute Malnutrition) to refer to all nutritional services. This can be confusing - it is not always clear which of the services is meant as there are three types of nutritional services within the MoH: Supplementary Feeding, Outpatient Treatment, and Inpatient Stabilization: 1) Supplementary feeding for children with moderate or mild malnutrition Due to shortages in CSB and vegetable oil from the World Food Program (WFP), the supplementary feeding services have been closed and subsequently there has been no more monitoring, registration or follow-up of children moderate or mild malnutrition. According to USAID, WFP will provide food for supplementary feeding centers again shortly, though it is unclear when this will be. In the absence of functioning supplementary feeding services, the MoH and IMC-Tubaramure are looking into the possibilities (and feasibility) of community-based solutions for treatment of malnourished children such as PD-Hearth (Positive Deviance - Hearth Model) or FARN (Foyers d'Apprentissage et de Réhabilitation Nutritionnelle, which is the Burundian equivalent of PD￾Hearth). These options may be more sustainable though they are difficult to implement correctly and should be explored with caution. 2) Outpatient Treatment Centers for severely malnourished children There are 28 outpatient treatment centers for severely malnourished children (eight in Cankuzo and 20 in Ruyigi). Children are seen in these centers on a weekly base, and receive antibiotics, vitamin A, folic acid, deworming agents, and Plumpy’nut© (a ready-to-use therapeutic food [RUTF] based on peanut butter, skimmed milk, sugar, maltose, dextrose, vitamins and minerals). 23 For Output Indicator 1.12 it is recommended to change the wording as follows: 1.12. % of children recovered from acute malnutrition visited at home by the CHWs at least twice a month for three months. Referrals to these centers by the CHWs are working well; the CHWs know how to use the MUAC measurement tapes and use printed referral notes. On these notes it should be indicated whether the child is enrolled in Tubaramure or not, but this information is not recorded in the health center or the outpatient nutrition service. Stock outs of Plumpy’nut have resulted in severely malnourished children staying in treatment longer than expected. The outpatient services are open on a fixed day in the week and if on that particular day there are no Plumpy’nut sachets available to take home, a child goes without supplementary food for the whole week. These stock outs are mostly due to problems with transport between Bujumbura and Ruyigi and from Ruyigi to the treatment centers. The MoH and UNICEF need to solve this but meanwhile, one could look into stock management and think of alternatives for transport such as delivering the Plumpy’nut during joint support visits. In the long run, one might consider the possibility for local production of RUTFs as groundnuts are available in abundance. 3) In-patient Stabilization Centers The in-patient stabilization centers are linked to the hospitals of Ruyigi, Kinyinya, and Cankuzo. Initially there was supposed to be a fourth stabilization center at the Murore hospital in Cankuzo Province, but the MoH has no funds to establish it. Children with severe malnutrition with complications are admitted in these centers and are supposed to be seen by the pediatrician on intake and during his or her daily ward rounds. In Cankuzo, however, the stabilization center is in a wooden shack adjacent to the hospital. This used to be a temporary structure set up by Médecins Sans Frontières as a therapeutic feeding center during the crisis and is not visited by any doctor. The nurse in charge is competent but says he has only had three days of training in management of malnutrition and he often finds his cases too complicated. The director of the hospital would like to build a new pediatrics ward that includes facilities for malnourished children in a more permanent structure and he is looking for funds to do so. Though this request does not fit within PM2A, his request is not unreasonable as the wooden shack is not very hygienic, and it gets very cold at night, which is not helping the recovery of these often hypothermic, malnourished and sick children. The director of the hospital in Ruyigi made a request for blankets, which is a good solution for the short term. At the moment the program measures the “number of children recovered (released from CMAM) from acute malnutrition being followed by CHWs / total number of children recovered from acute malnutrition”. (See Appendix 20 of the original proposal for the PMP). The LMs who follow-up on children are not included in this measurement because IMC-Tubaramure currently only has the tools to follow-up on CHWs through the MoH. So either the LMs should be taken out of the indicator or the means to follow-up on LMs should be put into place within IR-1. 24 Furthermore, the indicator did not specify what ‘being followed’ means: Is that one visit? Weekly visits? Or monthly visits? The National Coordinator for IMC thought it would be feasible to set this at two home visits per month for three months. III. STRENGTHS, WEAKNESSES, THREATS AND OPPORTUNITIES A. Strengths The IR-1 component of the Tubaramure program is relevant, effective, efficient and sustainable. It is well-integrated into the local health system, has a strong emphasis on capacity building, and is technically sound in terms of health, nutrition and M&E.  Relevance: Pregnant women and mothers are accessing the health centers with their children, and are receiving quality service. The IR-1 component of the program is still very relevant as malnutrition is often caused and/or exacerbated by disease; prevention and treatment of childhood and maternal illnesses will have an impact on low-birth weight and childhood malnutrition.  Effectiveness: The IR-1 component of the program is well designed, using the MoH’s policies, structures and systems, and is meeting its targets. At the half-way point in the program one can already see a significant increase in the number of 1) women attending pre- and postnatal health services, 2) children participating in growth-monitoring activities, and 3) nurses and CHWs trained in IMCI for the improved detection and treatment of childhood illnesses.  Efficiency: IMC-Tubaramure has been very efficient in its use of resources. With very few technical staff in the field, they manage to support and supervise 50 health centers and 534 CHWs. The IMC staff in Cankuzo and Ruyigi use motorbikes to visit the health centers and share a rental car between the two provinces. However, one wonders if they could accomplish even more if each of the provincial teams had its own car as some of the health centers are far and remote.  Sustainability: The IR-1 component of the program is very well embedded in the Burundian health system and structure and there is a strong emphasis on capacity building and systems strengthening. During the second half of the program IMC will design an exit strategy together with the MoH; the strategy will include scaling down contributions in fuel and other material support to the MoH. B. Weaknesses B.1. Children with Moderate or Mild Malnutrition Children with moderate or mild malnutrition are being detected through the growth monitoring activities, but are not referred to the health centers or followed-up since the supplementary feeding centers are closed. Even if there is no supplementary food available for these children, they and their mothers should still be enrolled in some sort of activity for monitoring, follow-up, and nutrition advice. 25 It should be noted, however, that we are looking at a large number of children for the health centers to handle. In the period from January 2011 to April 2012 160,334 children under three were weighed during the growth monitoring activities in the health centers, of which 17,493 had moderate or mild malnutrition. To put this in perspective, this is about 1,100 children per month, or 200 children per health center per month. With the current staff shortages in the health centers, a strategy needs to be well thought out on how all these children could be registered, monitored, and followed up. It seems obvious that this activity should be decentralized to the community level (we are now talking about 4 children per colline per month, which seems more manageable). However, the community component under IR-1 with the MoH’s CHWs is still too new to handle this and one would have to look at joint efforts between IMC and the other consortium partners to mobilize the LMs for this. B. 2. The Community Component of IR-1 The community component of IR-1 started only recently and needs to be scaled up quickly. There is a need for more synergy between the outreach work led by FH-Tubaramure for IR-2 and the outreach work led by the MoH that is supported by IMC under IR-1. More of the trainings can be combined and to improve the collaboration between the CHWs and LMs, some form of incentive should be made available for the CHWs. C. Threats C.1. Shortage of Trained Health Workers  There is a shortage of nurses within the health centers: There should be six per health center, but in reality there are only two or at the most three. Opportunity: The MoH has established many new nursing schools and the new nursing schools in Cankuzo and Ruyigi will have their first graduates next year.  There is a high turnover of nurses within the health centers. As Cankuzo and Ruyigi Provinces are considered remote places, many nurses prefer to work elsewhere and the best nurses find it easy to get transferred. Opportunity: The MoH is decentralizing its hiring process. Nurses are now hired centrally and sent to the provinces but soon the provinces and districts will be hiring their own nurses.  There is a shortage of CHWs within the provinces: There should be three per colline, but in reality there are only two. This was already noted as a risk factor in the original proposal, and the MoH has pledged to train and install an extra CHW for every colline (= 268 new CHWs). C.2. Malaria  Malaria is the most prevalent disease in children under five, with 142,195 cases in Ruyigi and 100,891 cases in Cankuzo in 2011 (figures from BPS Ruyigi and BPS Cankuzo). Malaria is also the leading cause of death among children under five in these provinces.  Though the relationship between malaria and malnutrition is still controversial and the object of many studies, the nurses in charge of the inpatient stabilization centers indicated that most of the severely malnourished children they admitted had malaria. Furthermore the doctor at the Cankuzo hospital mentioned that he saw very high levels of parasitaemia among malnourished children with malaria. 26  In Burundi the national malaria control program promotes prevention by distributing insecticide treated mosquito-nets (ITN) through 1) mass campaigns every five years for all households in highly endemic areas and 2) provision of ITNs for pregnant women and young children through the health centers. There is no program or policy for the intermittent preventive treatment (IPT) of pregnant women and there are no programs or policies for vector control.  All pregnant women and all children age nine months (age for measles vaccination) are supposed to get an ITN at the health centers. The ITNs are provided by UNICEF but have been out of stock in most health centers for the past year. A country-wide distribution of ITNs from the Global Fund malaria program is not expected until the end of 2013 or the beginning of 2014.  Though the diagnosis and treatment of malaria is already part of the IMCI curriculum, the Tubaramure program recognizes the need to address this as a separate issue and will factor in an extra course on diagnosis and treatment of malaria in children under five, with a module on malaria in severely malnourished children. D. Opportunities  The program has established excellent relationships with the MoH at all levels, which makes it easy to advocate for adjustments and changes in the MoH’s policies and programs.  Though IR-1 has thus far focused mostly on the clinical aspects in the health centers, IMC has a wealth of experience in Burundi and in other countries of working with communities. They can use their experience in Tubaramure to explore community-based solutions for treatment of malnourished children.  If there are surpluses in CSB and vegetable oil in the program, one could consider using these to kick-start a community-based program for the treatment of malnourished children. IV. RECOMMENDATIONS AND SUGGESTIONS A. Recommendations A.1. M&E The Primary Health Care Expert, the team leader, the Tubaramure M&E Coordinator, the IMC M&E Specialist, and the Tubaramure National Coordinator for IMC looked at the current program indicators and came up with several amendments, which are shown in the table below (in bold and cursive): Table 8. Recommended changes in M&E indicators Current indicator: Proposed indicator: 27 Current indicator: Proposed indicator: 1.1. % of pregnant women completing package of prenatal visits 1.1. % of pregnant women completing 3 prenatal visits 1.2. % of mothers completing package of postnatal visits 1.2. % of mothers completing 2 postnatal visits 1.3. % of children 0-59 months attending growth monitoring at least once in a two-month period (as recorded on card) 1.3. % of children 0-36 months attending growth monitoring at least once in a two-month period (as recorded on card) 1.4. % of health providers (facilities/ CHW) accurately assessing a child using IMCI protocols 1.4. % of nurses accurately diagnosing and treating children under five 1.5. % increase of women registered for prenatal services by the sixth month of pregnancy 1.5. % of women registered for prenatal services by the sixth month of pregnancy 1.6. % of health facilities with 2 or more staff who completed in-service training in pre-post natal services 1.6. % of health facilities with 2 or more staff who completed in-service training in pre-post natal services 1.7. % of health facilities with two staff members trained in IMCI protocol through MOH's IMCI office 1.7. % of health facilities with two staff members trained in IMCI protocol through MOH's IMCI office 1.8. % of trained CHW in IMCI in target areas through MOH's IMCI office 1.8. % of collines with 2 or more CHWs trained in IMCI in target areas through MOH's IMCI office 1.9. Number of communes with at least one functioning community based growth monitoring center after year 3 (Target: 5 communes per province = 10) 1.9. Number of communes with at least one functioning community based growth monitoring center after year 3 (Target: 5 communes per province = 10) 1.10. % of health facilities with upgraded growth monitoring equipment. 1.10. % of health facilities with upgraded growth monitoring equipment. 1.11. % children referred to CMAM actually admitted for treatment 1.11. % children with malnutrition referred from the community who are enrolled in nutrition services 1.12. % of children recovered from acute malnutrition being followed by the CHWs or LM. 1.12. % of children recovered from acute malnutrition visited at home by the CHWs at least twice a month for three months. A full explanation for the changes was provided earlier in this chapter. A brief summary is provided below:  For Indicators 1.1 and 1.2: Take out the word ‘package’ and measure number of visits. Also, check the registers of the health centers regularly to see whether supplements are given and to monitor whether women are weighed during pregnancy, if they receive a mosquito net, and if they get their tetanus toxoid vaccination (which are also requirements in the national protocol).  For Indicator 1.3: Change the age to 0-36 months in indicator and denominator.  For Indicator 1.4: Measure nurses’ ability to diagnose and treat children under five correctly. Either perform a test (similar to the post-test used during the training modules) at semi￾annual intervals or look at the number of nurses who have correctly completed their five algorithms out of all the nurses in the 50 clinics. 28  For Indicator1.5: Use the number of women giving birth in the target area as denominator (instead of women registered at the health center for prenatal consultations.)  For Indicator1.8: Use the number of collines as denominator (instead of total CHWs in the target area.)  For Indicator1.11: Include referrals of children with moderate or mild malnutrition.  For Indicator1.12: Either take out the LM or find way to measure LMs’ performance within IR-1. Specify what ‘being followed’ means. A.2. Community Outreach Work  Continue to provide incentives for CHWs to go out into the community and visit more children in their homes.  Improve joint efforts between IMC and the other consortium partners to mobilize the LMs for follow-up of malnourished children.  Combine training to improve the collaboration between the CHWs and LMs. A.3. Management of Children with Malnutrition  The program should ensure that children with moderate or mild malnutrition are also identified, registered and followed-up in the community as was planned in the original proposal. B. Suggestions B.1. Community-based Solutions for Moderate Malnutrition  In the absence of functioning supplementary feeding services, the MoH and IMC could look into the possibilities (and feasibility) of community-based solutions for treatment of malnourished children, such as PD-Hearth or FARN, which is the Burundian equivalent. These options may be more sustainable, though are difficult to implement correctly and should be explored with caution.  IMC has a wealth of experience in Burundi and in other countries of working with communities. The organization can use the experience of Tubaramure to explore community￾based solutions for treatment of malnourished children.  If there are surpluses of CSB and vegetable oil in the program, one could consider using these to kick-start a community-based program for the treatment of malnourished children. B.2. Improving Nutrition Services  Consider providing blankets for the severely malnourished children who are hospitalized in the nutrition stabilization centers.  Consider using the CHW’s referral notes to register in the health centers and nutrition services whether the child is enrolled in Tubaramure or not. 29  Find means to assist the MoH with their stock outs of Plumpy’nut: Assist them with their stock management or think of alternatives for transport such as delivering the Plumpy’nut during joint support visits.  Consider the possibility for local production of RUTFs as a long-term strategy. B.3. Programmatic Issues  Consider having a vehicle in each of the provinces to facilitate supervision (instead of sharing one vehicle between the two provinces).  Propose to MoH that growth monitoring activities and immunization days be combined on the same day at the health centers.  Propose to MoH that women are ‘automatically captured’ for a postnatal check-up when they come to the health center for their baby’s BCG vaccination.  See whether the MoH’s IMCI office provides training through the nursing schools and whether they might need support from the Tubaramure program. INTERMEDIATE RESULT 2: HOUSEHOLDS PRACTICE APPROPRIATE HEALTH AND NUTRITION BEHAVIORS 2. Households practice appropriate health and nutrition behaviors. Technical Lead: FH 2.1: Households (HH) adopt Essential Nutrition Actions (ENA) 2.2: HHs adopt Essential Hygiene Actions (EHA) 2.3: HHs adopt prevention and management behaviors for maternal and childhood illnesses Food for the Hungry is the technical lead for this IR and is implementing the Care Group approach as a primary conduit for behavior change promotion. The Care Group Model is a community-based strategy for achieving widespread and lasting household- and community￾level behavior change. The Care Groups are comprised of volunteer LMs selected by their neighbors to conduct health promotion with women who are pregnant or have children 0-23 months of age. I. IR-2 ACTIVITIES A. Establishment of Care Groups of Leader Mothers The LMs were selected by their peers and each committed to carrying out the responsibilities presented by the Tubaramure team. The LMs are organized into CGs of 10-12 women per group and the THP conduct up to two training sessions per month for them as the BCC materials are 30 rolled out. The first BCC module, which was introduced to the LMs in early 2010, focuses on CG orientation and includes topics such as an introduction to Care Groups, teaching techniques and LM responsibilities. Each LM has approximately 10-12 women in her group. The key responsibilities for the LMs are to conduct training twice a month for the beneficiary mothers in their group and to carry out periodic home visits to support women and other family members who are adopting new behaviors. The three key themes they promote are Essential Nutrition Actions (ENAs), Essential Hygiene Actions (EHAs), and home-based prevention and management of illnesses. At the midterm there were 425 CGs (255 in Ruyigi and 170 in Cankuzo) with 4,866 LMs responsible for 44,903 beneficiaries. (NB: Throughout this report “beneficiaries” refers to the pregnant women and mothers of young children who participate in the Tubaramure program unless otherwise stated.) B. Development of BCC Materials FH-Tubaramure has devoted considerable effort toward the development of comprehensive, high-quality BCC materials for the LMs to use in their peer education activities. The materials include: - Five modules (illustrated flipcharts) with lessons for all aspects of maternal and child health as well as messages for general family well-being and the promotion of better community health (See Annex J for a complete list of lessons.) - An “Age-Specific Card”, which is actually a shorter, separate module synthesizing all the key maternal and child health messages organized according to the stage of pregnancy or the age of the child - A radio series with highly entertaining skits to reinforce the messages in the modules The BCC Technical Advisor at FH, who oversees the implementation of IR-2, and his colleagues have been careful to follow all the requisite steps to ensure that the materials are culturally appropriate, accurate, engaging and in line with the MoH protocols and international standards. These steps have included:  Formative research on several topics including a study on the local determinants of malnutrition carried out early in the first year; a number of barrier analyses on nutrition, sanitation and hygiene topics; and most recently (June 2011), a barrier analysis on the use of ITNs  Close collaboration with the MoH in the development of materials, especially with the Information, Education and Communication (IEC) Unit, the National Nutrition Program (PRONIANUT) and the National Reproductive Health Program  Thorough pre-testing with prospective audiences in the program area  Validation workshops with the MoH and other partners reviewing the materials before they are finalized and printed The main challenge in the development of the BCC materials has been adhering to the original production timetable. Ideally, all the materials for the beneficiaries would have been developed by now. But there were constraints that delayed production: the sheer number of steps involved 31 in developing quality, tested materials; locating a qualified artist for the illustrations; and finding a competent printing house that would respect the contractual requirements, especially the deadlines. In the case of the artist and the printing house, the work was eventually outsourced to a nearby country. As a result, only three of the five modules have been rolled out so far and most of the groups are only halfway through Module 3. On a positive note, FH-Tubaramure recognized early on the importance of beneficiaries hearing certain key messages as soon as possible and the Age-Specific Card was developed for use by December 2010. According to the BCC Technical Advisor the training schedule is designed so that all beneficiaries will have heard these messages before graduating. A significant investment has been made in the development of these materials, which are available at the Care Group website jointly maintained by FH and World Relief: http://www.caregroupinfo.org. Ideally, other organizations and agencies would be able to make use of them as well or to adapt some of them to their own needs; FH/Burundi may want to organize a sharing session with other NGOs, UN agencies and the GoB so that more people are aware of this valuable resource. C. Cascade Training Given the sheer number of beneficiaries and the relatively low ratio of trainers to trainees, cascade training is an effective way to reach everyone at least once with the messages. As each module is ready for dissemination, the THPs are trained in its use. They in turn train the CGs of Leader Mothers. The LMs then bring together the beneficiaries in their group and go through the lessons. In theory, the THPs are to meet twice a month with each CG and the LMs are also to hold group meetings with their beneficiaries twice a month. The modules are being disseminated but not as rapidly as planned for several reasons:  There have been production delays as described above.  The THPs work across all three IRs and have a number of other responsibilities in addition to the BCC activities.  Attendance at the second set of monthly meetings (THPs with LMs and the LMs with beneficiaries) is always lower. Sometimes the meeting is not held at all due to scheduling conflicts, competing priorities for the THPs or bad weather. As a result, some CGs cover only one lesson that month. The real issue here is that not all the beneficiaries will hear all the lessons in the five modules. Over 8,000 have graduated since November 2011 and unfortunately there are dropouts among graduating beneficiaries and LMs who have decided not to come to meetings since they are no longer receiving rations. (It is difficult to accurately estimate the dropout rate since both Tubaramure staff and LMs are reluctant to acknowledge that for many beneficiaries the rations were the main reason they participated. What is clear is that the dropout rate for LMs is much lower than for the general population of beneficiaries and this is a bright spot.) D. Complementary BCC Activities A number of other BCC activities were incorporated into the program design to reinforce the messages promoted in the modules. Some have been successful while others have not yet been implemented. 32 D. 1. Home Visits The LM are supposed to carry out regular home visits to the beneficiaries in their group to provide individual support both to the beneficiary and to other family members who are ready to try new behaviors. Home visits are a core element of the CG approach but unfortunately are not taking place systematically in Tubaramure. If a beneficiary is absent from the monthly meeting, she is more likely to be visited. Leader Mothers cite time constraints, especially during the agricultural season. This is understandable since the LMs are volunteers and are already expected to participate in two trainings per month with the THPs and to help organize the monthly food distribution. These two activities alone can take two to three days a month. Distance between households was another constraint cited as dwellings can be widely dispersed in rural Burundi. Some LMs also indicated that their husbands accuse them of neglecting domestic duties and income-generating opportunities because of their role as leader Mothers. With all this, the temptation for LMs is to conduct the monthly meeting or meetings and spend much less time on the home visits. This makes it difficult for beneficiaries to find the necessary encouragement and support to maintain new behaviors. It also reinforces the conception that transmitting information is sufficient; the emphasis on identifying barriers and finding solutions is diminished. In retrospect, perhaps the emphasis should have been on home visits to individual beneficiaries for disseminating the messages and not monthly meetings. This is not as “efficient” a method as cascade training but in the long run might result in more lasting changes. Another suggest is that where the beneficiaries are not all living close to the LM (as is the case in many areas of Burundi), she should have fewer women to follow. D.2. Hygiene and Sanitation Improvements One of the best reinforcements of the messages in the modules has been the improvements in sanitation. The THPs demonstrate how to construct drying racks, improved latrines, hand washing stations (Tippy Taps), garbage pits and other low-cost interventions to improve general hygiene and sanitation at the household level. These improvements have been a great success and are beginning to be widely copied by neighbors of beneficiaries. D.3. Radio Under the guidance of the BCC Technical Advisor, the program has designed a radio series to reinforce the key BCC messages. Before beginning the radio program, a survey was conducted to determine what stations are most popular, the best time for women to listen to the radio, and the format that would be most interesting. The result is a feuilleton or set of linked skits that are entertaining as well as informative. To date 36 skits have aired since December 2010 and people who hear them can recite them almost verbatim, showing that they are presented in a memorable way. The only downside to this cost-effective intervention – and it is a major downside - is that only a small number of women actually follow the series. During the individual and group interviews with the beneficiaries, they stated that either they don’t have radios or the feuilleton comes on at times when they are busy. The BCC Technical Advisor is aware of this problem and is exploring several possibilities:  Increase the number of times during the week that each station transmits the skits.  Increase the number of radio stations. 33  Give radios to LMs as an incentive.  Sponsor contests with prizes based on knowledge of the skits. It may also be useful to do a small sample to determine 1) if women really would have time to listen to the radio at other times of the day and 2) if men are hearing the messages and if so, what they think about them and whether they have made any changes in their homes. D.4. Other Support for Behavior Change One of the premises of the Tubaramure program design was that there would be widespread support for behavior change from a variety of actors. At the community level, this would include the CHWs, active and dynamic COSAs, and concerned local leaders, including the appointed and elected local authorities. The responsibility for ensuring this community-level involvement is shared across the entire program and each consortium member has a role to play. To some extent community support is evident. For instance, the THPs and other program staff maintain regular contact with the local leaders, especially the chefs de collines via the regular commune-level meetings that all chefs de collines are expected to attend. And IMC-Tubaramure has begun to work more directly with the CHWs, who have the potential to be a valuable resource for supporting behavior change. The original proposal states: “Monthly feedback sessions led by the LMs and health committees will help keep the community-at-large informed and vested.” The COSAs, however, have not been involved. But the issues with COSAs are not unique to Tubaramure. Community health initiatives are in their infancy in Burundi and while a community health policy document is available, it has not been operationalized. It may be useful for IR-1 and IR-2 program staff to find out what programs in the country have been successful in engaging COSAs and to determine what, if anything can be adapted for Tubaramure in the remaining two and a half years. At the health center level it was anticipated that the MoH personnel would provide both individual counseling and group sessions to promote behavior change. But given the shortage of staff and the work load for most nurses, this harmonization and reinforcement of the messages in the modules is not consistently happening. Men, especially the husbands of beneficiaries, are another potential resource for supporting behavior change that has not been fully realized. Men have been involved to some extent, especially in the construction of sanitation facilities and as replacements when their wives are unable to go to a food distribution. Some CGs have even selected a “Papa Tubaramure” who attends trainings and is expected to organize his peers for individual and group BCC activities. The men interviewed during the MTE seemed knowledgeable about the program but asked for some incentives similar to what the LMs have received: cloth, badges, a bag for carrying documents. They would also like their own set of BCC materials and it may be useful for FH￾Tubaramure to consider developing at least one module specifically for men to conduct peer education. It is true that some of the modules already developed have lessons targeted toward men but most men do not participate in these training sessions. 34 II. PROGRESS TOWARD PROGRAM OUTCOMES AND OUTPUTS The Public Health/Nutrition Evaluator was responsible for evaluating this IR and conducted both group and individual interviews with beneficiaries, Tubaramure staff, local authorities and program partners. Perhaps the most valuable source of information was the beneficiary mothers and the LMs. Over the course of the MTE she met with a total of 87 LM and 61 beneficiary mothers in eight collines and at two distribution sites. This section describes 1) progress made as a result of the BCC activities and 2) some of the issues that need to be addressed to sustain performance and reach the end of program targets. Overall, significant progress is being made toward achieving the three program outcomes as well as the three outputs for IR-2. FH-Tubaramure has designed and implemented interventions using current best practices and as a result, there are measurable increases in knowledge, increased utilization of health services, and noticeable changes in behaviors. The following success story also illustrates the synergy between IR-1 and IR-2: Through the BCC component FH-Tubaramure energetically promotes the use of health services. And through the IR-1 activities IMC-Tubaramure works with the MoH to improve the quality of those services. Both beneficiaries and health center staff noted an increased utilization of services for maternal and child health. And when the Public Health/Nutrition Evaluator asked beneficiaries and LMs if they had noticed any changes at their local health center since the program started, they readily noted a number of improvements. The five improvements cited most frequently during the interviews were the following:  A better welcome from the health center staff (almost always given as the first improvement)  Health center staff willing to receive them at any time, not just when it’s convenient  Health center staff encourage the women to have their children weighed and measured  Health center staff encourage delivery at the health center and coming to at least four prenatal consultations  When women deliver, the health center staff insists that they put the newborn to the breast immediately A. Performance Indicators for Measuring Outcomes Table 9. Tracking Outcome Indicators IR-2: Households practice appropriate health and nutrition behaviors. Baseline Target FY2011 Achieved FY2011 Target FY2012 Achieved Q2 FY 12 Target end of program Indicator 2.1. % of babies 0 to 5 months of age exclusively breast-fed in last 24 hours 69.4% 85% 92.6% 90% 91.4% 95% Indicator 2.2. % of babies 6-12 months of age receiving complementary foods (according to IYCF guidelines) 86.5% 95% NA NA NA 95% 35 plus breast milk in last 24 hours Indicator 2.3. % of children 0-24 months reported with diarrhea (3 or days of loose stools) within past two weeks 14.1% 5% 9.6% 5% 13.4% 5% A.1. Indicator 2.1: % of babies 0 to 5 months of age exclusively breast-fed in last 24 hours As Table 9 above shows, the program is already close to achieving the final target of 95% for Indicator 2.1. Although it is not possible to actually observe that mothers are doing what they report, it is clear that they recognize the importance of exclusive breastfeeding (EBF) and during the field interviews consistently cited this Essential Nutrition Action as one of the most critical for their children’s health. This is an impressive achievement given that the program is only at the midpoint; the challenge will be to maintain this high rate of EBF long-term and to promote it among other women who are not directly involved in Tubaramure. A.2. Indicator 2.2: % of babies 6-12 months of age receiving complementary foods (according to Infant and Young Child Feeding [IYCF] guidelines) plus breast milk in last 24 hours Activities for this indicator only started recently so progress hasn’t yet been formally measured. The semi-annual performance survey at the end of FY2012 (October) will provide a good idea of whether progress has been made in changing this behavior. As the program focuses more intensely on the activities associated with this behavior, it will be important to promote not only the message that a variety of foods are important for young children but also other IYCF recommendations such as: - Frequency of small meals - Consistency/digestibility of food, depending on age range - Sufficient quantity of food for age A.3. % of children 0-24 months reported with diarrhea (3 or more days of loose stools) within past two weeks Although the dramatic improvement in the adoption of Essential Hygiene Actions is one of the most striking findings of the MTE, the decline in diarrhea rates among children under two has not gone down significantly and in fact the result midway through FY2012 is actually higher than the FY2011 result. Since diarrhea rates vary seasonally, some of this increase over the past six months could be attributed to the fact that the mid-FY2012 measurement was taken during the rainy season when diarrhea rates are generally higher. But given the impact of repeated episodes of diarrhea on nutritional status and overall child health, it is important to determine what additional messages or actions need to be taken. FH-Tubaramure may want to work with their IMC colleagues to determine whether a limited study on the most effective ways to prevent diarrhea and manage it at home would be useful. In any case, Tubaramure will need to make a concerted effort between now and the end of the program to reach the target of 5%. B. Progress on Indicators for Measuring Outputs 36 Table 10. Tracking Output Indicators Indicator Baseline Target FY2011 Achieved FY2011 Target FY2012 Achieved Q2 of FY2012 Target end of Program Output 2.1: Households adopt Essential Nutrition Actions (ENA) Indicator 2.4. % of households observed carrying out four or more ENA actions at time of household visit NA 75% 87.1% 80% 88.9% 90% Output 2.2: Households adopt Essential Hygiene Actions (EHA) Indicator 2.5. % of households observed carrying out four or more EHA (essential hygiene actions) actions at time of household visit. NA 50% 57.5% 75% 66.5% 75% Output 2.3: HHs adopt prevention and management behaviors for maternal and childhood illnesses Indicator 2.6. % of households with children under 2 with a reserved package of ORS at time of household visit 4.6% 30% NA 35% NA 45% NA = Not available B.1. % of households observed carrying out four or more ENA actions at time of household visit The program has almost achieved its final target of 90%, an encouraging sign of the effectiveness of the BCC strategy. During the periodic surveys to monitor this indicator the M&E team looks for evidence of the following ENAs: EBF, child eating from four different food groups, immediate breastfeeding, use of iodized salt, GM during the past four months, and a woman eating more during her pregnancy. The results as of March 2012 show that households are quite likely to have iodized salt available and report that they practice optimal breastfeeding practices: Figure 2. Progress for ENAs 37 It should be noted, however, that not all these actions can necessarily be “observed” during a household visit. Some behaviors such as EBF, immediate breastfeeding, and food consumption are more likely to be reported, not observed. Another point in the wording of the indicator is that “ENA actions” is repetitive. The suggestion is to reword the indicator as follows:  % of households carrying out four or more ENAs at time of household visit (observed or reported) B.2. Indicator 2.5: % of households observed carrying out four or more EHA actions at time of household visit The Tubaramure program has made truly impressive strides in reaching targets for this indicator and the achievements in hygiene and sanitation represent one of the success stories at the midterm point. What is even more encouraging is that neighbors, inspired by the beneficiaries, are also adopting these practices. The following chart from the March 2012 performance report shows the EHAs that are the most successful: Figure 3. Progress for EHAs EBF 4 Food groups IBF Iodized salt GM Diet -pregnancy 38 As with Indicator 2.4 the term “EHA Actions” is repetitive. The suggestion is to delete the word “action” and re-word the indicator as follows:  % of households carrying out four or more EHAs at time of household visit (observed or reported) B.3. Indicator 2.6: % of households with children under2 with a reserved package of ORS at time of household visit The purpose of this output indicator is to determine whether households are taking measures to prevent maternal and child illnesses and whether they are implementing simple practices for appropriate home-based management of illnesses when they occur. This indicator is not being measured for several reasons including the fact that ORS packets are not for sale commercially. Following a discussion with the FH-Tubaramure team, the recommendation is to change this indicator to:  % of households with children under two who can state at least four of the six danger signs for childhood illness and at least two of the four danger signs for pregnant women Although there will be no baseline for the indicator, measuring it from now until the end of the program will provide some feedback on whether the messages on prevention and management of illnesses at the household level are being transmitted effectively. In spite of the fact that this output has not been measured, it was clear during discussions with the beneficiaries and LMs that there has been steady progress: Mothers could state most of the danger signs for young children and pregnant women and they reported that they take their children to the health center sooner when they are ill, a finding corroborated by health center staff who noted this improvement in caretaking. HW Latrine Clean yard Pit Platform Water purification ITN 39 As noted in the preceding chapter on Intermediate Result 1, malaria is a major threat to the population of these two provinces and is the leading cause of childhood morbidity and mortality. FH has recognized this and in June 2011 conducted a barrier analysis on use of ITNs. Renewing the emphasis on what can be done at the household level to prevent and manage this disease could make a big difference in health outcomes not just for children but for other family members as well. C. Issues to Address to Ensure Continued Success C.1. Work load of the Tubaramure Health Promoters One constraint to achieving all the planned BCC activities, including the scheduled rollout of each module in the CGs and the follow-up via home visits, is the work load of the THPs. This cadre has responsibilities within each IR and to each consortium member. They are also the primary interface between the local authorities and the program. None of those interviewed made a major issue of how busy they are and in response to concerns voiced earlier in the program, some adjustments in their workload have been made. For example, substitute THPs (suppléants) have been hired as full-time employees so that someone will be able to fill in if a Health Promoter is on leave. Staff and volunteers have also been added for the savings and credit activities (SILC or Savings and Internal Lending Communities), relieving the THPs of some of their responsibilities in this area. And while there are definite advantages to having the same group of agents involved in all three IRs, including food distribution, such an arrangement takes its toll in terms of quality, especially as the ratio of THPs to LMs and beneficiaries is low. A THP may have 10-12 Care Groups and as many as 115 LMs to train and support. The THPs are not always able to carry out home visits, to observe each LM conducting training, or to follow up on how well activities such as keyhole gardens and cooking demonstrations are being carried out. It is not unusual for them to fall behind on the rollout of BCC lessons due to the many demands placed on them. C.2. Incentives for Leader Mothers and Papas Tubaramure A perennial refrain from LMs (and other community volunteers in Burundi) is motivation meaning “What incentives will I receive for devoting my time and energy to these activities?” This is a fair question considering how busy Burundian women are and how much time is needed to be an effective LM. Tubaramure has already provided some incentives including the cloth the LMs proudly wear, but a concerted effort needs to be made to find other tangible and intangible rewards. Suggestions from those interviewed ranged from umbrellas to waterproof bags for their documents to exchange visits with other LMs in nearby communes or even in the adjoining province. At a minimum the LMs – and the men who are active as Papas Tubaramure – should be informed of the results of the MTE (including the ISTEEBU anthropometric survey) and publicly recognized for their contribution to the positive results. It may also be helpful for FH￾Tubaramure to survey a representative sample of LMs and men to determine what type of incentives, especially the more intangible ones, would encourage them. C.3. Dropouts The relatively high dropout rate among graduating beneficiaries is a concern and if it continues at the current pace, may negatively affect indicators between now and the end of the program in 2014. The dropout rate hints at an underlying problem – that many beneficiaries only 40 participated to receive the rations and do not see the many advantages of continuing in the program. III. STRENGTHS, WEAKNESSES, OPPORTUNITIES AND THREATS A. Strengths  The BCC materials are well-designed, culturally appropriate and comprehensive, with adequate pre-testing in the community and validation by outside experts.  The ASPIRE methodology is ideally suited to the program as it is an interactive communication strategy for raising knowledge levels of adults.  Relevance: This component is highly relevant to the overall Tubaramure goal of preventing malnutrition in children under 2. Where IR-1 works to improve health services and outcomes, this IR focuses on what individuals, households and communities themselves can do to prevent illness and to sustain improvements in health. The two IRs are highly complementary.  Effectiveness: The BCC strategy is quite effective in raising knowledge levels and promoting behavior change as demonstrated by measurable changes in good practices, especially for hygiene, breastfeeding, and the utilization of health services for pregnant women and young children.  Efficiency: The cascade training approach works (for the most part) but the ratio of LMs and THPs to beneficiaries is too low. It has also taken a long time to produce the modules and as a result the beneficiaries who graduate and choose not to continue attending BCC sessions will not benefit from all five modules.  Sustainability: It is likely that knowledge levels and certain practices (e.g., improved hygiene and sanitation facilities, optimal breastfeeding, utilization of health services and attendance at prenatal consultations) will remain high. Whether the LMs will continue their BCC activities in the long run will depend a great deal on what they themselves perceive as benefits from investing their time and energy in these volunteer activities. B. Weaknesses  The ratio of those primarily responsible for BCC activities (the THPs) is low compared to the number of LMs they train and support and the high number of beneficiaries.  Although one underlying objective of Tubaramure is to encourage the MoH to adopt the CG approach, few concrete activities have been undertaken in this regard other than including visits to CGs in the program for joint supervisions.  The program has not fully exploited the potential of men, especially husbands and fathers, to contribute to behavior change. C. Opportunities  The LMs have great potential to reinforce and complement the work of the CHWs. 41  The radio program appears to be well-designed, informative and entertaining and could reach a larger audience.  Other programs in Burundi use – or have used in the past – the CG approach and may be able to provide valuable lessons learned (e.g., Concern Worldwide and World Relief).  The BCC modules, including the Age-Specific Card, are valuable resources that could be more widely used outside the program.  Involving men more fully in BCC activities could strengthen the overall impact of IR-2. D. Threats  The relatively high dropout rate may jeopardize the program’s accomplishments and/or prevent Tubaramure from reaching its end of program targets. IV. RECOMMENDATIONS AND SUGGESTIONS A. Recommendations  Organize a workshop with Concern Worldwide, World Relief and other organizations that have had experience with CGs to share lessons learned and next steps.  Carry out a study with LMs and Papas Tubaramure to determine what incentives – tangible and intangible – are most likely to motivate them to remain involved and active. At a minimum the LMs – and the Papas Tubaramure – should be informed of the results of the MTE (including the ISTEEBU anthropometric survey) and publicly recognized for their contribution to the successes and achievements to date.  To better understand the reasons for the dropout rate among graduates, conduct a barrier analysis of beneficiaries and LMs who have graduated.  Reinforce the messages on malaria. Renewing the emphasis on what can be done at the household and community level to prevent and manage this disease could make a big difference in health outcomes not just for children but for other family members as well.  It may also be useful to do a small sample to determine 1) if women really would have time to listen to the radio at other times and 2) if men are hearing the messages and if so, what they think about them  It may be useful for IR-1 and IR-2 program staff to find out what programs in Burundi have been successful in engaging COSAs and to determine what, if anything can be adapted for Tubaramure in the remaining two and a half years. B. Suggestions 42  Find a way to share the BCC modules more widely with the MoH and other organizations that may be able to benefit from the investment the Tubaramure program has made in these materials.  Consider developing at least one module specifically for men to conduct peer education on responsible fatherhood and key ENA and EHA actions that men can support.  FH-Tubaramure may want to work with IMC colleagues to determine whether it would be useful to conduct a limited study on the most effective ways to prevent diarrhea and manage it at home.  Follow through with some of the ideas described above for maximizing the impact of the radio program. INTERMEDIATE RESULT 3: ELIGIBLE WOMEN AND CHILDREN HAVE INCREASED INTAKE OF NUTRIENT-RICH, DIVERSE FOODS 3. Eligible women and children have increased intake of nutrient-rich, diverse foods. Lead: CRS (in partnership with Caritas for commodity distribution) 3.1: FFP rations distributed to eligible women and children at community level 3.2: Mothers and children use FFP rations appropriately 3.3: HHs use appropriate local foods in addition to FFP ration CRS is the lead consortium member for this Intermediate Result, which focuses on one of the pillars of optimal nutritional status – food utilization. One of the most important components of this IR is the management of the commodity supply chain, a component that was thoroughly investigated during the midterm evaluation. Given the importance of the assessment of the supply chain management and the level of detail in the Commodity Supply Chain Specialist’s report, it has its own chapter, which directly follows this chapter. I. IR-3 ACTIVITIES For IR-3 there are three major outcomes, each with its respective set of activities: managing the commodity supply chain (covered in the next chapter); ensuring the proper utilization of the Title II rations; and promoting increased consumption of nutrient-rich local foods. This chapter focuses on the second and third outcomes. A. Ensuring the Proper Utilization of Title II Rations The program has a number of strategies in place to ensure that the Title II commodities 1) reach the right beneficiaries and 2) are prepared correctly. The Tubaramure team works closely with Burundian authorities at all levels - from the provincial governors to the chefs de collines - to ensure that the commodities are not sold or bartered. During the midterm evaluation authorities 43 described the actions they have taken in this regard and showed the team actual reports of businessmen arrested for trying to sell Title II commodities. To ensure that the CSB and vegetable oil are prepared correctly, Tubaramure developed and disseminated recipes in both Kirundi and French and periodically organizes cooking demonstrations, sometimes including local foods in season along with the CSB and vegetable oil. A great deal of effort was expended in the first year of the program to explain the rations: their purpose; who is eligible; what quantities are distributed for each category of beneficiary; and what the conditions are for continued eligibility. This effort has paid off:  When beneficiaries, family members and authorities were asked why rations are provided, the invariable answers were “to prevent malnutrition in young children” and “to improve children’s health”.  Everyone interviewed, including non-beneficiaries, knew the amount of CSB a particular category of beneficiary was to receive. However, non-beneficiaries were somewhat less accurate about the amount of vegetable oil per category.  The beneficiaries understood the conditions for continued eligibility. (See Annex K for the list of conditions.) What was less clearly understood by beneficiaries, authorities and even some Tubaramure staff was that the monthly rations included four rations for the rest of the household to “protect” the mother/child ration from being eaten by the whole family. The midterm evaluation clearly showed the importance of the protection ration. There was unanimous agreement among all those interviewed that the monthly mother/child ration would only last a few days if it was not accompanied by the protection ration. This was also the conclusion reached by the evaluation team. It is simply not acceptable or feasible for a mother not to share available food with other family members. In this respect, the protection ration definitely contributes to the overall program goal of preventing malnutrition in children under 2. But the protection ration does consume resources. On a monthly basis, pregnant women and mothers with children 0-6 months receive six kgs. of CSB and 0.6 kg. of vegetable oil as a direct ration and 12 kgs. of CSB and 1.2 kgs. of vegetable oil as a protection ration. The protection ration thus represents 66.7% of the entire monthly ration. The ratio is the same for the ration for children 6-24 months. From the start of the program until now, the total cost of the commodities shipped to Burundi was $18,769,869. When one applies the percentage (66.7%) for the protection ration to this total, the cost of the protection ration amounts to $12,519,502, twice the cost of the mother/child ration. There are obviously opportunity costs to providing the protection ration. The question remains: Could a program use the same amount of money for other activities (e.g. activities to increase the availability of and access to food through food production, income generation, direct cash transfers or other interventions) and still achieve the same reduction in malnutrition rates for the same time period? It may not be possible to achieve the same reductions in malnutrition within the same timeframe, but there is the possibility that the addition of a well-designed component to increase access to and availability of more diverse, nutrient-rich foods could contribute to longer￾term and perhaps longer-lasting improvements in nutritional status. B. Promoting Increased Consumption of Appropriate Local Foods 44 According to the original proposal, one of the underlying purposes of providing Title II rations was to “jump-start the effects of improved diets and to demonstrate how small changes in diet can yield big results”. This connection – that the consumption of nutrient-rich local foods can help to prevent malnutrition the same way rations do – is not understood by everyone, especially at the beneficiary level. B.1. Update on Activities An underlying objective for this outcome was to “improve household decision-making around access to and utilization of food” by giving women greater decision-making power over what the family eats. To accomplish this outcome, the program design called for the following activities: 1) As part of the formative research, the program will look at the role of fathers in the feeding of children. Based on the findings the project will develop a strategy for promoting men’s involvement in maternal and children nutrition. Update: This strategy has not yet been made operational. 2) Development of recipes incorporating local foods (including animal products) appropriate for pregnant and lactating women and children under two. Update: This activity was carried out and materials are available. 3) Raising farmers’ awareness of increased demand for nutritious foods for pregnant and lactating women and children under 2. Update: The plan is to meet with producers’ associations and with the women themselves, most of whom are farmers. 4) Pilot tests of local preparations that can be used by households once Title II ration distribution has ended. Update: The program has made steady progress to examine all the options for using local products as a replacement for the Title II commodities. The Technical Advisor for Food Utilization, previously employed at the Centre National de Technologie Agro￾Alimentaire (CNTA or National Center for Food Technology), is well-qualified to pursue this initiative. She is even examining what it would take to fortify such locally-produced foods with vitamins and minerals. The CRS Senior Technical Advisor for Nutrition, who was visiting Burundi during the midterm evaluation exercise, encouraged her to use available program resources to move this initiative forward. 5) CRS to provide seeds and small animals under its cost share to a very small population of the most highly vulnerable beneficiaries with extremely limited access to complementary foods. Update: CRS did provide some gardening seeds, fruit trees and poultry to certain LMs in 2012. These activities were evaluated separately by another consultant. However, interviewees indicated that there were a number of problems with this initiative, including in some cases poor quality of seeds, plants and poultry and the fact that not all LMs were included. 45 A related activity that has been successful is the construction of keyhole gardens, which allow women to have an easy-to-maintain kitchen garden right outside their door. These gardens are being replicated by non-beneficiaries and the THPs are much in demand to help people construct them. In a matter of months there were 119 gardens (end of April 2012) and more are planned. II. PROGRESS TOWARD PROGRAM OUTCOMES AND OUTPUTS Progress toward reaching the IR-3 program outcome and the three outputs has been mixed as the following tables and discussion show. For Output 1, a smoothly functioning commodity supply chain ensures that eligible women and children are receiving the rations as planned. The combined efforts of local authorities, the LMs and the Tubaramure staff, especially the THPs, ensure that the Title II rations are being used appropriately. As for Output 3, progress is being made on a number of fronts but it may be difficult to ensure that consumption of local foods increases significantly for reasons described below. A. Performance Indicator for Measuring the Program Outcome Table 11. Tracking Outcome Indicators IR-3: Eligible women and children have increased intake of nutrient-rich, diverse foods Baseline Target FY2011 Achieved FY2011 Target FY2012 Achieved Q2 FY 12 Target end of program Indicator 3.1. Increased average Household Dietary Diversity Score over life of project 4.6 -- -- 8 5.8 9 At this rate it may be difficult to reach the end of program target. Many beneficiaries state that they know a balanced diet is important for good health but they lack the means to achieve it either because they do not produce the necessary foods themselves or they do not have the resources to buy them. B. Progress on Indicators for Measuring Outputs Table 12. Tracking Output Indicators Indicator Baseline Target FY2011 Achieved FY2011 Target FY2012 Achieved Q2 FY2012 Target end of Program Output 3.1: Title II rations distributed to eligible women and children at community level Indicator 3.2. % of mother/child unit qualifying for food rations according to eligibility criteria of project. 0% 85% 54.7% 90% 66.4% 90% Output 3.2: Mothers and children use FFP rations appropriately 46 Indicator Baseline Target FY2011 Achieved FY2011 Target FY2012 Achieved Q2 FY2012 Target end of Program Indicator 3.3. % of mother/child unit recalling consumption of CSB/Oil within last 24 hours 6.1% 90% 89.9% 95% 84% 100% Indicator 3.4 % of households demonstrating remain supply of CSB/Oil based in accordance to expected CSB usage per month. 0% 80% 76.6% 90% 85.7% 95% Output 3.3: Households use appropriate local foods in addition to FFP ration Indicator 3.5 % of children 6 to 24 months consuming at least 4 food groups within last 24 hours 74.2% 85% 82.8% 90% 81% 95% B.1. Indicator 3.2: % of mother/child unit qualifying for food rations according to eligibility criteria of project. This indicator measures whether beneficiaries fulfill the conditions for receiving rations and the data as of March 2012 indicates that it may be difficult for the program to achieve the final target of 90%. In order to be eligible to receive rations, beneficiaries agree to a set of conditions that they will fulfill during their pregnancy and at different stages of their child’s life. (See Annex K.) The conditions for maintaining eligibility for rations are reasonable, especially as they were selected to help women optimize their health and the health of their young children. Indeed, in a number of cases the conditions consist of a beneficiary or household simply agreeing to carry out a practice such as adhering to IYCF guidelines. (The only conditions that do not currently apply are those referring to the activities on the Tubaramure Family Poster as the poster has not been developed.) Given that the conditions are not unreasonable, it is somewhat surprising that the percentage of beneficiaries fulfilling them is only 66.4%. There are a couple of possible explanations. First of all, there is no sanction for not fulfilling the conditions. Rations are never suspended when beneficiaries fail to carry out their part of the agreement. Indeed, with thousands of beneficiaries, it would be extremely difficult to monitor whether each beneficiary is fulfilling the conditions. Even if it were possible to monitor these data points, denying rations would likely have very negative consequences for the program and of course on the nutritional status of women and children. Second, when one examines the conditions for the time when the child is 6-9 months old, it is easy to see why the percentage falls to 35% during this period as the conditions require more effort to fulfill than a simple verbal commitment:  Postnatal visits completed (show card)  Child’s immunization record is current (show card)  Child shows satisfactory progress in growth (MUAC) or mother demonstrates capacity to recuperate the child B.2. Indicators 3.3 and 3.4 (to measure that rations are used appropriately) 47 In addition to the periodic monitoring by the M&E Unit, program staff also check the specially marked Tubaramure bucket during home visits to determine if the family is consuming the rations at the appropriate speed. As noted earlier, the local authorities also stress that it is not legal to sell the rations, that they are intended for family consumption. The program is on track to achieve this output. To be clearer, Indicator 3.4 should be reworded to say:  % of HH demonstrating that the remaining supply of CSB/Oil is in accordance with expected monthly CSB usage B.3. Indicator 3.5: % of children 6 to 24 months consuming at least 4 food groups within last 24 hours One BCC topic that everyone seemed to know well was the importance of a balanced diet. Beneficiaries and non-beneficiaries, men as well as women - all could list local products for the three main food categories. The program is well on its way to achieving this indicator, but whether the outcome – increased consumption of local food – will be achieved is another question. Whether the MTE evaluation team was meeting with the Governor of Cankuzo, program staff or beneficiaries, there was one constant theme: “It is not enough to know what we should be feeding our children; we also need to be able to increase our production, plant new varieties and/or purchase food that we can’t grow ourselves.” In short, almost everyone interviewed mentioned the need for “mésures d’accompagnement” (complementary activities) that would increase both the availability of food and access to it. Interviewees, including Tubaramure staff, had a long list of possible activities for increasing agricultural and animal production and/or generating income to buy food. In fact, one of the key assumptions of the program design is that “Food security continues to improve” and if a lack of local foods turned out to be a problem, the proposed solution was to link with other projects. The evaluation team agreed that in the context of a program designed to prevent malnutrition in areas such as Cankuzo and Ruyigi Provinces, it is important to ensure that there are either strong links to other programs or a component to 1) improve food production (e.g., agriculture and/or animal husbandry) and/or 2) increase household revenues (e.g., credit schemes and/or income-generating activities) so that families can buy more nutritious local foods. Improving health services and conducting BCC activities, however successful these interventions are, may not be sufficient in the long term to reduce malnutrition once the Title II ration distribution is finished. Although the midterm evaluation team concurred with the interviewees on this topic, there are some obstacles to acting on this finding: 1) Time: Is there enough time left to mount successful food production and/or income￾generating schemes before the end of the program? For income generation, SILC is a viable, relatively low-cost possibility. Access to credit allows families to engage in income generating activities (IGAs), to buy food, and to conserve their food stocks rather than selling them to meet unanticipated expenses. 2) Resources: Where would the resources come from for additional activities? Possibilities include examining the current budget to see if any cost centers can be realigned and/or finding additional resources either through FFP or another donor. But once again, time is a constraint for the latter suggestion. 48 3) Number of beneficiaries: Given the sheer number of beneficiaries in the program, it is likely that only a small proportion would be able to benefit from new activities. Would this have a sufficient impact to positively influence malnutrition rates? Would assisting some and not others create social and political friction? III. STRENGTHS, WEAKNESSES, OPPORTUNITIES AND THREATS A. Strengths  Well-run commodity supply chain ensures timely distribution of rations.  Activities such as BCC and cooking demonstrations have been successful in creating demand for local foods and a balanced diet.  Program works in close collaboration with the agricultural technicians at all levels and with nutritionists from PRONIANUT.  Strong working relationship established with CNTA and other institutions, facilitating the search for replacement foods for the Title II commodities.  Relevance: Both the use of rations and the promotion of increased consumption of local foods are highly relevant to the overall goal of preventing malnutrition in children under 2.  Effectiveness: The commodity supply chain component is highly effective. As for increasing the consumption of local foods, the program has been effective in raising awareness and creating demand but availability of and access to food remain problems in many areas of the program zone.  Efficiency: Rations are distributed efficiently and as cost-effectively as possible.  Sustainability: Whether lasting progress will be made on promoting the consumption of nutrient-rich, local food will depend on the programmatic and financial decisions that the consortium members and the donor make. B. Weaknesses  The issue of improving decision-making around food has not been addressed directly.  It may be difficult at this stage of the program to address the real concerns of beneficiaries who are requesting assistance with local food production and other means of improving access to better food. C. Opportunities  There is good potential for and keen interest in developing a substitute for the Title II rations.  Ongoing collaboration with agriculture and nutrition technicians opens the door to new food availability/food access opportunities. D. Threats 49  A major threat to promoting greater use of local foods remains food availability and access to the food that is available. IV. RECOMMENDATIONS AND SUGGESTIONS NB. A number of the recommendations and suggestions for IR-3 are contained in the following chapter on commodity management. A. Recommendations  Continue to work on a gender action plan that would involve men more fully in promoting maternal and child nutrition.  Determine the feasibility and potential effectiveness of adding activities to increase availability of and access to food to the program.  Continue to explore the possibility of developing a substitute for CSB and vegetable oil, using the local flour mixtures already being produced.  Continue to expand keyhole gardens and other agricultural and animal husbandry activities already initiated with the MAE.  Examine the current budget to see if cost centers can be adjusted to provide more resources for food production and/or SILC activities.  Complete the development of a plan to track women and children who have graduated from the ration distribution component. TITLE II FOOD COMMODITIES MANAGEMENT The overall objective of the midterm evaluation of the Title II food commodities component was to assess the relevance, the effectiveness and the efficiency of commodity management and distribution systems. The Commodity Supply Chain Expert had conducted an earlier assessment of Tubaramure in March 2011 and was quite familiar with the program and the commodities management set-up. FFP/USAID approved a total of 27,914 metric tons (MT) of food commodities - Corn Soy Blend (CSB) and vegetable oil - for distribution during the five-year life of the program. During the period November 2009 – April 2012 CRS/Burundi imported a total of 17,324 MT (15,897 MT of CSB and 1,427 MT of vegetable oil on behalf of the Tubaramure consortium. See Annex L for the most recent ledger. 50 CRS/Burundi’s principal partner in food distribution is Caritas with which it has long-standing ties. Both organizations have extensive experience in food and non-food distribution in Burundi. CRS has contracted with private third-party agencies for inland transport, clearing, handling and surveys. While there are a number of organizations and agencies involved, it is CRS’s responsibility to ensure that food management is done in compliance with the generally accepted food management principles as stipulated in Regulation 11 and by FFP/USAID. This chapter of the report is divided into two main sections: 1) a description of the system for moving food from the port in Dar-es-Salaam in Tanzania to the beneficiaries and 2) findings and recommendations from the field visits conducted during the midterm evaluation. I. THE FOOD LOGISTICS SYSTEM The Commodity and Monetization Unit (CMU) of the CRS office in Bujumbura is staffed with: a Commodity Manager, who is in charge of both monetization and food distribution; a National Commodity Manager who reports to the Commodity Manager; and a FoodLog Accountant. In the field there are two Warehouse Managers in Ruyigi and four End Use Checkers, two in each province. The M&E Officer responsible for monitoring the overall Tubaramure activities is also very involved with food management. For example he has designed a data base that helps the program staff monitor the list of participating mothers and their children with greater accuracy. Prior to this, the beneficiary list was manually updated and the field staff devoted a great deal of time to keeping it current and errors were not uncommon. This computerized application has proven to be an efficient and practical tool to track mothers and children on a monthly basis according to the type of rations they are to receive. CRS uses FoodLog, a commodity tracking system, to manage the warehouses, the distributions and the donor reporting. The regularly updated list of enrollees has been instrumental in preparing the food allocations according to the needs at the distribution sites. It should be noted that this teamwork between the M&E office and the CMU has been critical to ensure that the program does not serve false beneficiaries. A. Title II Food Movement from Dar-Es-Salaam to Burundi Burundi being a land-locked country, CRS uses the port of Dar-es-Salaam in Tanzania for the transit of both the distribution and monetization food commodities. CRS does not have a staff member stationed in Dar for the coordination of the port operations, but clearing and trucking of One of the Ruyigi warehouses is large enough to accommodate two operations at once. 51 the food shipments are carried out by SDV, a forwarding agent, which works in collaboration with POLUCON, an independent survey company. The latter has been contracted by CRS to survey the loading of the food commodities into trucks in Dar and the offloading at the two CRS warehouses in Ruyigi. In the first year of the program (2009), the main warehouse was located in Gitega. It was later determined that this location was too away from the program areas in Ruyigi and Cankuzo Provinces. Trucking the commodities from the Gitega warehouse to the distribution sites was not very efficient and was unnecessarily costly. Furthermore, frequent handlings of the bags of CSB or cartons of vegetable oil increase the potential for damages or losses. Renting warehouses in Ruyigi has proven to be cost-effective and efficient. The two warehouses in Ruyigi have an approximate capacity of over 4,000 MT and are much closer to the fixed sites (sites fixes) and mobile distribution sites (sites mobiles). Gitega, however, remains the central site where all goods, including the Title II commodities, are cleared when they enter Burundi thru the Dar-es￾Salaam corridor. A.1. Transit and Freight Forwarding 52 The evaluation team happened to be present at the Ruyigi warehouses during the offloading of the last shipments on May 30, 2012. It was an opportunity to meet with the SDV/Gitega agent who was overseeing the offloading operation. His comments on the transit process are quite noteworthy. He indicated that the port and border authorities in Dar are very professional with the processing of the paperwork. Contrary to what happens in many other countries on the continent, there are no hidden fees or unnecessary delays. After loading the trucks at the port in Dar, it takes three days to reach Gitega via Kobero, one of the many borders with Tanzania. In Gitega, the SDV/Gitega agent clears the goods with the customs and dispatches the shipments to Ruyigi. The Title II food commodities are tax-exonerated based on the agreement with the GoB. The way that the customs office now processes the paperwork has eliminated much of the frustrating red tape. Both Tanzania and Burundi have agreed on the types of documents and codes needed for clearing and forwarding goods. This makes the whole operation less cumbersome and quicker. The improved operating environment was confirmed by the three truckers we talked with on the warehouse premises. Sylvester Nyambela, one of the Tanzanian drivers, expressed his appreciation. “I don’t have to spend days or weeks at the borders anymore. More interestingly, we drivers don’t have to take the same route on our way back to Tanzania as we were required to do before. In the past, we had to go through Gitega again as proof that the cargo had been discharged. With the new arrangement between the Burundian and Tanzanian authorities, we are allowed to use an alternative road to go back home. Ruyigi is closer to Tanzania via the Ruyigi-Cankuzo road. The difference is about 300 km shorter. We go home more quickly to see our family and we save money on gas too”. The paradoxical effect of the expedited commodities clearing is that the trucks are ready to go more quickly. However, the loaded vehicles cannot always travel at the same time in a convoy as the arrival of several vehicles gluts the narrow roads near the Ruyigi warehouses. The evaluation team witnessed this situation first hand. According to the SDV agent, the police in Ruyigi have already warned the truckers not to block traffic or cause an accident when both motorists and pedestrians try to avoid the parked vehicles. Suggestion: We suggest that the SDV agent: 1) Dispatch the trucks from Gitega based on the pace of the offloading at the Ruyigi warehouse. Coordinate with the CRS warehouse staff in Ruyigi via phone, taking into account the distances. (NB. CRS is already working to improve the current dispatching system.) Cargo fully protected for inland transportation 53 2) In collaboration with CRS, ask the local authorities in Ruyigi to make a vacant plot available in the neighborhood where the trucks can be safely parked when they arrive in convoy from Gitega. A.2. Freight Transportation SDV/Dar-es-Salaam uses the services of BHANJI Transport to move the shipments from the port of Dar to CRS’s main warehouse in Ruyigi. On May 30, 2012 we witnessed the offloading of 406.36 MT of food commodities at the warehouse with no damages or losses according to the waybill. The majority of the cargo was containerized while the rest of the shipment in break bulk was well-protected by tarpaulins secured by tight straps. There are two explanations for the zero level loss at this stage of the food management: 1) the shipment comes containerized from the USA and 2) there is safe inland transportation of the break bulk cargo after de-stuffing some containers. As a result, all actions or strategies that have contributed to minimizing losses or damages have to be encouraged and maintained. The evaluator would like to remind CRS/USA and CRS/Burundi to always make sure that the request for “containerized commodities” is mentioned in the Call Forwards submitted to FFP/USDA. Since CRS/Burundi initiates the call forward process, it is advised not to omit the “containerization mention” in the “concurrence request” to the USAID mission in Burundi. A.3. Fleet of Vehicles and Inland Transportation It is reported that the transport company in Dar has a fleet of over 100 trucks. On May 30, 2012, fourteen trucks of the same make and logo arrived from Dar to offload CSB and vegetable oil commodities at the Ruyigi warehouses. They were in very good condition for inland transportation. Very often, transit or forwarding companies do not own their own fleet of vehicles and have to rely on the services of private individual truckers. As a result, it can be quite challenging to assemble the required number of vehicles needed for trucking commodities to their final destination on a timely basis. The current situation in which SDV deals with a company with a road-worthy fleet, available anytime when needed is a good illustration that CRS makes sure the commodities transportation is in good hands. The availability of sufficient road-worthy vehicles is a factor CRS should consider for future negotiations with a forwarding agency. Although the current arrangement seems to be functioning relatively smoothly, the Commodity Manager in Bujumbura indicated he has tendered the transit services for international bids and has already received some offers. He evoked some minor problems in CRS-SDV’s relationship. The current contract ends on June 30, 2012. Since poor transportation is not listed as the bone of contention, the evaluation team would like to recommend the following: Recommendations: 1) CRS or the selection committee should make sure that truck availability or a reasonable fleet ownership by the transit company is a determining factor for the selection of a freight forwarding company. 2) Furthermore, whatever the outcome of the bidding process might be, CRS/Burundi staff should travel to Dar regularly (every other shipment) to monitor the port operations and interact with all key parties (clearing and forwarding agent, surveyor, vessel agent, inland 54 transporter(s).) It is worth mentioning from the evaluation team’s experience that when the consignee is not around, freight forwarders tend to respond to pressure coming from other customers instead. This may lead to some delays that can jeopardize the smooth implementation of the program. For example, in May 2012 food distribution did not happen at some sites due to delays in inland transportation from Dar. A.4. Loading and Discharge Surveys POLUCON is an independent surveyor contracted by CRS for loading the trucks in Dar and offloading in Ruyigi. The CMU is very appreciative of POLUCON’S work as the staff is very responsive, very thorough in its analysis, and professional in the presentation of its reports. However, during the offloading of the commodities at the warehouse in Ruyigi on May 30, 2012, the POLUCON staff person designated for the discharge survey was apparently sick. His replacement was a friend who came without a sheet to tally the quantity or quality of the offloading. Fortunately, we noted that the containerized CSB commodities didn’t show any damages or losses against the inland waybill. (David Hay Smith FFP report of 21-26 Feb. 2012 reads: Polucon, who was monitoring the receipt of cargo, was not tallying the bags as they were off- loaded from the truck; this is not in line with procedures). As per Regulation 11, the survey report is a valid document in court of law and Surveyors are not allowed to sub-contract their services. In case of claims against port authorities, inland transporters, or overwhelming damages, survey reports are the only legal document the Cooperating Sponsor has to exhibit. Of course, CRS’s Commodity and Monetization Unit was not aware of the arrangement between the sick POLUCON agent and his friend to have the work done. Recommendation: CRS should send a notification to POLUCON to remind them of their role and responsibilities with reference to the provisions in Regulations 11. CRS has a legal responsibility in case of an audit. (NB. The evaluator brought the issue to the attention of the CRS Commodity Manager who promised to take action immediately.) 55 FIGURE 4 – SUMMARY OF FOOD MOVEMENTS LOADING AT PORT IN DAR-ES-SALAAM (TANZANIA) TO RUYIGI (BURUNDI) • Coordination with port authorities • Clearing procedures for trucks and quantity loaded. • Description & condition of the shipment i.e. sound/damaged/reconditioned, etc. • Survey Report (Polucon) Surveyor, etc. SDV - Transit Agent CRS (sometimes) INLAND TRANSPORTATION & ENTRY TO BURUNDI THRU GITEGA • Agreement with GoB for tax exoneration • Declaration of cargo transported (Quantity & Quality - Documentation) • Clearing by Customs Truckers from Dar (BHANJI Company) SDV, CRS/BUJ DELIVERIES TO MAIN WAREHOUSES IN RUYIGI DELIVERIES TO CARITAS WAREHOUSES BY LOCAL TRANSPORTERS DELIVERIES TO MOBILE DISTRIBUTION SITES BY CRS VEHICLES DISTRIBUTION TO BENEFICIARIES AT BOTH FIXED AND MOBILE SITES￾CARITAS/PARISH WAREHOUSE STAFF (Les Gestionnaires) 56 B. Food Distribution to Caritas Warehouses - Methodology Caritas is the contractual sub-recipient for CRS for the management and distribution of commodities to the distribution sites. Caritas in turn subcontracts with parishes of the Catholic Church, which are widely disseminated in the two provinces. The broad reach of the Catholic Church through the parishes facilitates access to the population and to secure warehouses. There are two levels of food allocations to Caritas: 1) from the Ruyigi main warehouses to fixed sites, and 2) from fixed sites to mobile sites. The CMU of CRS’s office in Bujumbura and the provincial offices in Cankuzo and Ruyigi are all responsible for monitoring food deliveries to distribution sites. The list of beneficiaries is updated every month after the distribution of the preceding month. The food allocation requests are forwarded to the National Commodity Manager in Bujumbura for verification and to the Chief of Party for approval. The M&E Officer checks the list against his data base and updates the list of eligible beneficiaries based on each beneficiary’s status (ration for pregnant/lactating woman or ration for child 6-24 months). The National Commodity Manager uses the updated list to prepare a monthly distribution plan for both the fixed and the mobile sites. This plan and the job order are forwarded to the CRS warehouse personnel in Ruyigi for information and action. B.1 First Level of Food Allocation to Caritas CRS/Bujumbura has contracted a private vehicle company (PROTAIS) that takes the food allocations from the warehouses in Ruyigi to the fixed sites where there are warehouses owned either by the Catholic Church or by parishioners. The food commodities are dispatched from the fixed sites to the mobile sites, which were established to reach more beneficiaries in remote communities in order to take the food distribution activities closer to the participating mothers. This distribution work is executed by Caritas, which works with the different parishes through contracts, the clauses of which are explicit enough to delineate the role and responsibilities of each party. The Caritas contract with the parishes holds them responsible for “the storage, the accountability and the security of the commodities”. Each parish is responsible for recruiting and paying local staff based on local salary scales. In some instances, the parish does not have a building to be used as a warehouse and has to rely on a parishioner who owns a facility. In such cases, the parishioner is sub-contracted by the parish and has full management responsibility over the food commodities including the food manager, night guards, food distributors, etc. Overall, things have worked well so far. But during the midterm evaluation, some of the workers (e.g., warehouse managers, security night guards, and food distributors) complained about 1) the salary level; 2) the lack of a salary increase for the last two years; and 3) some delays in the payment of the salaries. The late payment is sometimes due to the banking system, which does not have enough coverage to facilitate money transfers or deposits to some locations. Although CRS has subcontracted these distribution activities to Caritas and the two organizations have a solid working relationship based on many years of collaboration, CRS is ultimately accountable to FFP/USAID for commodity management and should provide adequate oversight without micromanaging its partner. Under the assumption that disgruntled workers may jeopardize the smooth running of the program or resort to some schemes, the evaluation team would like to recommend the following: Recommendations: 57 1) CRS and Caritas should talk with the parishes to determine a reasonable salary to be paid to the workers that does not conflict with the church’s principles. 2) Review a salary increase scheme (1% or 2%) or a flat yearly bonus. (NB: Salary increase or bonus might not be significant in monetary value but could be a psychological incentive.) 3) Work out strategies to transfer or deposit money to parishioner or parish accounts on a regular basis. B.2. Second Level of Food Allocation to Caritas The mobile sites do not have storage facilities and receive their food allocations the same day they have scheduled for their distribution activities. There is good collaboration between the two CRS offices in Ruyigi and Cankuzo and the provincial Caritas supervisors. CRS has two 3.5 MT-capacity trucks that are available to move food commodities from the parish warehouses to the mobile distribution sites. At times the CRS drivers have to make two trips to accommodate the tonnage needed for distributions. The distance per trip is generally between 20 and 25 km. To facilitate operations, the truck from either office is made available when needed to the other province. Consequently, the evaluation team determined that the two CRS vehicles are sufficient to move the food allocations around from fixed sites to mobile sites. It’s worth noting that the two trucks are ideal for the operation as their self-contained body needs no tarpaulins or straps during food transportation. One of the main characteristics of the monthly distributions is that they take place at the community level rather than through the health facility. This was a purposeful decision in order to dissociate the food from medical treatment. A THP is always available when Caritas carries out the distributions. There are two types of monthly rations, determined by the category of the participant: 1) pregnant woman (or mother of child 0-6 months) + protection ration for the rest of the household and 2) child 6-24 months + protection ration. B.3. Food Movements from the Warehouse to Distribution The evaluation team was able examine all the tools used and found them to be appropriate for a reliable Title II food management system:  The waybill (Bordereau de Transport) is a booklet with five sheets with sequential numbers in different colors, i.e., white for the Commodity and Monetization Unit, blue for Finance, yellow for Warehouse, green for Recipient Center and pink to be kept in the booklet.  The waybill page has five main sections: 1) nature of the transaction, 2) details of the cargo (commodities) approved by the CRS warehouse staff, 3) acknowledgment of the truckload by the driver, 4) receipt acknowledgement by the parish warehouse manager, and 5) observations by the warehouse manager with the driver signing off on any observations.  The other good practice on this waybill is the mention of the time of loading at the CRS/warehouse and the time of offloading at the destination. This is a very important way to monitor when a truckload of food has left the main warehouse and when it arrives at its final destination. It’s one of the mechanisms to prevent food diversions. 58  There is a good system in place to deduct the value of losses from the trucker’s invoices. The value of lost commodities is determined by the USAID formula in Regulation 11 which is C&F value + associated costs (inland transportation, clearing and other handling fees, etc.)  The contract between CRS and PROTAIS Company has explicit provisions to guarantee compliance with FFP’s principles. The Commodity and Monetization Unit reports that this clause of the contract was applied to PROTAIS Company when one of its trucks fell off a small bridge and part of its cargo sustained damages. The company was invoiced for the damaged commodities accordingly. Constraints: Paying the food handlers/laborers at the Ruyigi warehouse or at the distribution sites has been a challenge. With the current system, the Commodity and Monetization Unit in Bujumbura calculates the total amount to pay for loading and offloading the food commodities based on the approved distribution plan. A check is written in the name of the National Commodity Manager or someone else assigned by the program who travels to Ruyigi to cash the check at the bank in Ruyigi. He remits the money to the finance assistant at the Ruyigi office for deposit in the safe. Someone from the Cankuzo office comes to collect its share and the payment to the laborers is made when an End-use Checker or the Caritas supervisor is going to the site for a field visit. This person gives the total amount to the warehouse manager at the site who in turn looks for the laborers (or their leader) who had performed the job two or three weeks earlier. The laborers sign off on a document to acknowledge receipt of the money or the team leader signs on their behalf. The total amount of the last operation in April 2012 was 2,890,100 Burundi Francs (FBU) or approximately 2,042 USD of which 637,940 FBU (about 450 USD) was destined for Cankuzo. Not only is the system cumbersome but it is also risky. Moreover, some of the Caritas warehouse keepers do not seem to systematically keep documentation on the actual payment to the laborers. Recommendations: 1) To the extent feasible, CRS should open up a bank account in the two provincial offices for labor payment only. The payment should occur within the week that the service is performed. 2) Ask the warehouse keepers to file proof of payment to the laborers as they would with the other project documents. Each distribution site should always have proof of any payment made to a third party. It can be subject to audit by USAID. 3) Review the “Fiche de Paie des Manutentionnaires” (Pay Statement for Laborers) and include additional names of the persons involved in the transactions, i.e., CRS, Gestionnaire (Caritas), Supervisor (Caritas). (NB. Instead of the Tubaramure logo, you may want to use the consortium’s logo.) NB. The issue of how best to pay the laborers was raised with the Commodity Manager who indicated that another approach is being explored: the food handling fees will be taken on by the private transporter who will include them in his invoice. This approach has the merits of avoiding cash manipulation and of paying the laborers on the spot if the transporter has money upfront for these expenses before submitting his invoice. However, CRS and Caritas have to 59 make sure that the transporter is working through an existing group of laborers who have been constituted following a tacit agreement with the provincial governors and CRS. These laborers are mostly unemployed youth in the provinces who rely on this part-time job for their survival. B.4. Timing for Food Deliveries The timing for food deliveries is another constraint. Some Caritas warehouse managers would like to be informed ahead on the time of the delivery of their food allocation. They are not talking of an exact time, but would like a reasonable window. There are instances where food commodities have arrived after 17:00 and the warehouse declined to accept delivery until the next morning for security reasons. In other cases, the truck didn’t show up at all during the whole day although the warehouse staff had been told to expect the cargo in the morning. This issue was brought to the attention of the members of the consortium during the debriefing in Bujumbura and they offered the following recommendations: Recommendations: 1) Communicate the transportation/delivery plan through CRS/Caritas, Supervisors and warehouse staff. 2) Add provisions in the trucker’s contract stipulating fees to pay in case of unjustified long delays or if he fails to make it altogether. 3) Increase the number of contracted transporters. II. FINDINGS AND RECOMMENDATIONS FROM THE FIELD VISITS Of the 15 fixed sites and two mobile sites in Ruyigi and the eight fixed sites and two mobile sites in Cankuzo, the evaluation team visited seven fixed sites in Ruyigi and four fixed sites in Cankuzo and all four mobile sites. We chose to visit the sites during their scheduled activities to have a vivid appreciation of the work. The Commodity Supply Chain Specialist on this midterm evaluation team had the opportunity to visit most of the participating centers last year. By comparison, there is an undisputable improvement in the implementation of the activities and the overall management of the food component. The CRS and Caritas staff have to be commended for an impressive achievement in the field. Wherever we went, we saw the marks of the Commodity Manager, the National Commodity Manager, the End-use Checkers and the Caritas Supervisors, all of whom signed off on the stacking cards or warehouses ledgers. A. Findings at Food Distribution Sites A.1. Commonalities at Food Distribution Sites Contrary to the observations last year and to what was reported by FFP visitors in the first year of activity, there has been a remarkable improvement on many fronts: 60  All the sites visited (fixed and mobile) had a well-marked perimeter for distribution that was cordoned off.  Intruders were kept behind the rope and even mothers who have not yet been called in have to wait for their turn on the other side of the delimitation.  The sessions are orderly and well-organized. Food distributors acting as security guards are posted at the in and out points of the distribution perimeter. They maintain order and keep onlookers at bay.  There are new shelters built everywhere to protect the food distributors and the food against the sun/rain. This is a great improvement compared to last year where those who are responsible for checking beneficiary eligibility sat under the blazing sun. Some challenges still remain however: 1) No shelter/No shade: The vast majority of the beneficiaries who are waiting for their turn a short distance away from the distribution area have no roof over their heads. Rain or shine, they are exposed to the elements. The luckiest ones are those whose sites have some trees around under which they can sit. In that context, it is obvious that CSB cannot be distributed on a rainy day. The evaluation team talked with all the men at the distribution sites and they agreed to volunteer if asked to build a shelter for their wives and children. 2) No Toilets: The other challenge (reported in the past by FFP visitors) is the absence of toilets for the participating mothers and their children. Some distribution sites, which are not far away from the parish school, are allowed to use the school facilities. Some site managers maintain that they have their own latrines. Unfortunately, the evaluation team noted that these latrines do not reflect the hygiene and sanitation standards promoted in the Tubaramure program. There is work to do here. Recommendations: 1) CRS and Caritas should work with the community, the husbands and other volunteers to construct shelters to protect the women and their children against the elements. 2) CRS and Caritas should work with the husbands to construct latrines and simple hand washing stations under the guidance of the THPs, using the same principles as those taught to the women and the households in the communities. NB. The gender component in the program is a perfect strategy to get the men involved. There are so many provisions under the gender umbrella that can positively influence sustainability and promote the community’s participation in the project. The evaluation team’s Storage boxes made out of empty vegetable oil tins 61 suggestion for the construction is not to have anything fancy, but to use local materials (as shown in the pictures below) to make a small difference. The first picture is a ceiling made of empty cartons of vegetable oil: they first put a layer of tarpaulin on top and the second layer is made of straw. Everything lies on a strong interwoven wooden framework. The second picture is a ceiling made of flat open bags of CSB displayed on a wooden shaft. Since the interior layer of the CSB bag is plastic, a tarpaulin is not needed and the straw is put straight on top. These are examples of good practices to promote. A.2. Compliance with Standard Food Management Principles Overall, all the warehouses visited meet the criteria necessary to store Title II food commodities.  Buildings are adequate and respond to FFP’s norms of security i.e., strong walls, good roofing, good doors with security paddlocks, good windows, adequate location, etc.  Commodities stacked on pallets  Stack cards or tally cards visible on the piles  Space between piles and walls  Warehouse ledgers well-maintained  Figures of the ledgers, the tally cards and the waybill reconcile. The evaluation team randomly chose old and new food transactions in the book and reconciled the figures with those in the waybill, the stock cards and other distribution paperwork.  There are no improper erasures in the books. White-out is no longer used.  All food transactions either on the tally cards or the ledgers are authenticated by name and signature of the person who carries out the operation.  Clean warehouses and surroundings  Documentation filed in binders or sequentially piled up in boxes. At one site (Nyabitare), the warehouse manager has two storage cabinets made out of scrap metal from empty vegetable oil tins. The cabinet can be locked and keeps the documents away from the dust or other stains. This is a practice to be promoted at other distribution sites in Burundi and/or elsewhere. A.3. Some Challenges at Distribution Site Warehouses Some of the practices that need attention are not serious enough to require a full training or major reshufflement of the people dealing with food (CRS and partners included). The evaluation team suggests some coaching on the following issues. This endeavor can be carried out by the CRS Commodity and Monetization Unit staff, the End-use Checkers and/or the Caritas supervisors if the latter have received a refresher course from the Commodity and Monetization Unit: 1) Teach warehouse managers of fixed sites to set aside in the warehouse the food commodities to be distributed before instructing workers/distributors to take them out. The current practice of taking commodities out just by sizing up the beneficiaries present outside has two potential risks: First, in the process of moving the food out of the warehouse, a dishonest distributor can hide a bag or two in the bush nearby with the help of an accomplice while the warehouse manager is still inside. Second, relying on the count of the empty bags and tins at the end of the day to reconcile the tally cards and the main ledger is confusing and not viewed as good practice. 62 2) Rodents are still present here and there. CRS made a good move by providing all the warehouses with a box with rat poison in it. During our field visit, along with CRS and Caritas staff, we saw a couple of rats that didn’t die in the box or near the box, but away from the bait under the pallets. Hence, there is a risk that a rodent will walk on a bag or two before dying a couple of meters away, increasing the risk of contamination. 3) Some CSB bags may look alike but they are of different sizes coming from the USA. The measurement of the traditional bag is 16 x 4 x 33.5 while the other bag is 16 x 4 x 36. The 25 kg. weight is the same. The slight difference in the measurement makes stacking and physical inventories a bit challenging. The piles lean dangerously, making the physical count difficult. The CRS commodity management team has already alerted FFP to this problem more than once. FFP/Burundi needs to be informed. USDA/USAID have to talk with their bag suppliers. A.4. Waiting Time for Beneficiaries at Distribution Sites The waiting time for beneficiaries has been an issue raised by FFP visitors during their field visits. CRS and Caritas have been doing all that is strategically possible to reduce the distance between the beneficiaries and the distribution sites. They also do what they can to reduce the time spent by women and children at the centers, e.g., creation of mobile sites, invitation of the beneficiaries by groups and at different hours, etc. Some beneficiaries are lucky enough to live just 20 minutes away from the distribution sites while some women have to walk four hours one way and then spend another hour and a half or two to receive their food ration. They leave home around 6:00 AM and return between 20:00 and 21:00. Moreover, some women who are not affected by the distanc still have to wait two or three hours before being served. There are some groups of beneficiaries who have only themselves to blame. Even though they are told during the invitation to be there at 11:00, they choose to be there earlier. They come to the distribution center with their children or leave them home under the care of a young sibling. This issue was a topic of discussion during the one-day workshop following the debriefing to the consortium in Bujumbura. The working group had this to say: Recommendations: 1) Increase the different ways the invitations are given so that everyone is aware of the time and place for their group: e.g., parish/church, THPs, market place, village drinking place, etc. 2) Increase the number of distribution days. Suggestions for some of the issues described above: 1) CRS and Caritas will put in place a ledger where the draft calculation of the rations is recorded before taking the food out for distribution. (Please see the example of Nyabitare.) 2) As in any war, the first line of defense against Sample of trap with CRS/Malawi 63 rodents is prevention. Cover all the openings and all potential entry points for rodents. Weed the surroundings of the warehouses. Cover ventilation openings with mesh. 3) Use mechanical traps and make the bait more attractive. Use other devices that allow the rodent to get in the trap but unable to exit. (See CRS/Malawi). The bait inside has to be attractive and tasty too. 4) CRS should send another message to USAID/USDA about having one size bag from the suppliers. Ask FFP/Burundi to get involved. A.5. Suggestion for a new method for food distribution: Grouping vs. Scooping The current method of food distribution (referred to here as scooping) at the sites was agreed upon by both CRS and Caritas and has been in use since March 2011. Those collecting their rations are organized into two lines depending on the type of ration they are collecting. One line is for the pregnant and lactating women who receive 18 kgs. of CSB and 1.8 kgs. of vegetable oil and the other line is for the beneficiaries with children 6-24 months who collect15 kgs. of CSB and 1.5 kgs. of vegetable oil. The scooping method requires the presence of several workers: a. Two people to verify the eligible beneficiary list b. Four people to weigh out the rations (two for each line) c. Four beneficiary representatives who ensure that the ration provided is correct d. Two security guards who help direct the beneficiaries to the correct line and maintain order. They also prevent intruders from obtaining food rations. With this method it is estimated that one can serve 300 beneficiaries per day for fixed distribution sites from 8:00 AM to 15:00 PM and 450-500 beneficiaries for mobile distribution sites. Below is a diagram that shows what the current distribution site looks like: Figure 5. Current distribution set-up 64 Table 1 Table 2 CSB OIL OIL CSB Exit Entrance 15 Kgs & 1.5 Kgs Children Rations 18 kgs & 1.8 kgs Pregnant & Lactating Mothers Rations Some of the questions posed in the midterm evaluation TOR were:  Is the food distribution system design efficient?  Is the current distribution plan able to accommodate the increased numbers in a timely and efficient manner?  Are the recommended food rations size respected? With these questions in mind, the Commodity Supply Chain Specialist suggests that there may be a more efficient way to organize distributions, a way that involves the beneficiaries more, increases their participation, reduces the time they spend at the distribution site, and builds their capacity. The methodology being suggested will use all the parameters of the current distribution plan described above, i.e., delimited distribution area using ropes, scales, presence of warehouse managers, same documentation, etc. The main goal of the suggested approach is to organize the women and empower them with the distribution of the food commodities. The current methodology has nicely paved the way for this suggested new approach. The advantages of the current method will be maintained:  The groups at the distribution sites are managed by the LMs.  All the women interviewed during the midterm evaluation know their rations. They understand why some are entitled to 18 kg. of CSB and why others are receiving 15 kg.  The delimitation of the distribution area with rope creates an environment that keeps intruders at bay.  With very few exceptions, women have their eyes on the scales during the distribution. Here are the steps in the grouping method: 65 1. Two CGs can be combined the day of the distribution; they are still under the leadership of their respective LMs. Limiting the combined group to no more than twenty persons is a manageable size. 2. The LMs are given the distribution list of their CGs. First they verify everyone’s presence and affix their fingerprints. Cartons of oil are used as tables. You may want to add the verification of the health cards for growth monitoring and vaccinations. 3. The warehouse manager will instruct the laborers to take out to the group the number of bags and cartons they are entitled to according to her/his calculations. It is likely that the total amount in the bags and cartons will not correspond exactly to the rations to be distributed. The warehouse manager will inform the group 1) if there will be remaining CSB and vegetable oil and if so how much or 2) will inform them that he will complete any missing portion toward the end of the group’s distribution activity. 4. In previous experiences with the grouping method, all the participating women would have agreed previously on a container to be used as a gauge. This container will have been “technically” measured and tested in everyone’s presence. 5. With this methodology, six groups can distribute/receive food at the same time. If each group has no more than 20 beneficiaries, the whole operation (serving up to 120 people) generally takes less than an hour. 6. Distribution lists are given back to the warehouse manager for compilation. 7. During the process, THPs and warehouse managers circulate in their supervisory role, making sure that the BCC knowledge transferred to the LMs has actually passed down to the other mothers. It’s also the opportunity to check if there are some achievements in terms of hygiene and sanitation. 8. This approach is less expensive than any other method and has the advantage of empowering the beneficiaries and reducing the time spent at distribution. 9. Looking to the future, practice in using such a method could also help communities with other distributions. For example, if there is to be an ITN distribution, the CGs have had prior experience in organizing themselves for an equitable division of goods. Proposed next steps: This proposed new distribution system was one of the topics discussed in the small group work following the debriefing in Bujumbura on June 18, 2012. The group’s reaction was that the local USAID Mission has not endorsed this approach. This is not surprising because at the beginning, the food distributions were not always well-organized and introducing an innovation at that time did not make sense. However, both the Commodity Supply Chain Specialist and the local FFP Food Specialist have noted outstanding progress in the food distribution system. Distributions are carried out in an orderly manner, all beneficiaries know their rations, and men replacing their wives wait patiently behind “their” LMs. As the FFP Food Specialist noted in a recent report: “…FFP/Burundi noted extraordinary progress in food distribution management. FFP commends the hard work and creativity of program managers and partner collaboration. Sites are separated with rope, clear separation of distributions from waiting Gauges of 5 & 10 kg. for distribution in addition to the scales - CRS/Malawi, 66 beneficiaries, community members. Distributions were orderly and quiet/peaceful…..” (Audace Mpoziriniga’s report of May 3 – 5, 2011) Recommendations: 1) CRS and FFP/Burundi will get together to analyze the proposed approach. 2) If they concur, CRS/Caritas will implement a pilot in a couple of “strong” sites. A.6. Collaboration with Caritas CRS and Caritas have a long-standing relationship worldwide and Burundi is no exception. CRS has been working in Burundi since 1961. Caritas/Burundi has a remarkable national network including eight dioceses, which contain 140 parishes. Due to its presence at the grassroots level and due to the fact that the majority of the population is Catholic, the parishes have been the first channel of communication with the Tubaramure participants. The invitations to the food distribution sites are read during the church services on Sundays. LMs relay the message to other beneficiaries and the THPs do the rest. In Ruyigi and Cankuzo, there is one Caritas Supervisor assigned to the program activities in each province. Both conduct field visits to make sure things are being implemented according to the norms; they have a broader role in administration and activities monitoring and have participated in food management trainings offered by CRS. Consequently, they are qualified to do end-use checking as well. They do field visits either individually or jointly with their CRS counterparts and meet from time to time either in the field or at the office. Unfortunately, the joint visits are not systematic and there is no report written when the visits happen. Furthermore, the CRS End￾use Checkers send their field reports directly to the National Commodity Manager in Bujumbura. Some findings that necessitate the Caritas Supervisor’s input for correction may remain untouched until the Supervisor makes his own visit. Also, there are a couple of important documents like the “Rapport de Distribution des Vivres” and “La Fiche de Demande de Vivres” that the Supervisor simply forwards to his HQ in Bujumbura. The Supervisors say that they check the accuracy of the information before its transmission to the Caritas HQ, but the documents have no signatures to prove it. Recommendations: 1) Systematize the meetings with the CRS End-use Checkers at the office. 2) Share findings and set up a plan outlining who is responsible for addressing specific issues in the field and when. Women doing own food distribution (CRS/Malawi) 67 3) Systematize joint field visits and always write a quick report as proof of execution and collaboration. A.7. Physical Inventories at Ruyigi and Distribution Site Warehouses The End-use Checkers diligently carry out their monthly duties at the site level. The Commodity Supply Chain Specialist made some suggestions for the current “Fiche d’Inspection” and the “Fiche de Vérification des Agents de Contrôle” that both CRS and Caritas have agreed to review and to adopt. The passage of the Commodity Manager, the National Commodity Manager and the End-use Checkers is duly acknowledged by having them sign the tally cards and the ledgers. This is a sign of good practice. As for the two CRS warehouses in Ruyigi, the physical inventory takes place every month under the supervision of the CRS Internal Auditor based in Bujumbura. While the two warehouse managers always remain the resource persons, the rotation among the “neutral people” who oversee the inventory has always been the End-use Checkers from the two provinces. In fact, this is a good practice. However, as a routine, it is easy to determine who is next for the upcoming inventory. As a principle of better internal control, the evaluator suggests the following: Suggestion: Randomly select staff who are not involved with food commodities to participate in monthly physical inventories. Counting the bags or the cartons is easy and can be taught to any beginner in a minute. A week’s notice is enough to inform somebody that s/he will be a part of an inventory. If the Internal Auditor is already in Ruyigi, a two-hour notice is sufficient. A.8. Empty Containers Disposal of Containers. Cooperating sponsors may dispose of containers, other than containers provided by carriers, in which commodities are received in countries having approved Title II programs, by sale or exchange, or may distribute the containers free of charge to eligible food recipients for their personal use. If the containers are to be used commercially, the cooperating sponsor must arrange for the removal, obliteration, or cross out of the U.S. Government markings from the containers prior to such use. The above provision in Regulation 11 is self-explanatory. But during the field visits the evaluation team was not able to determine how the empty containers are handled by the site managers or the food distributors. The managers maintained that the empty containers are donated to participating women but the women stated that they never receive them. The evidence is the absence of a clear policy in place. Recommendations: 1) Both CRS and Caritas should work together to determine a policy on the disposal of empty containers. The policy should be widely communicated to the site managers. If the decision is to give them away to participating mothers or family members, set specific criteria as everybody wants the containers. 2) Encourage site managers and communities to use the containers as recycled materials to build shelters to accommodate the beneficiaries during food distribution. (See pictures below). 68 A.9. USAID Branding/Marking Compliance Wherever the evaluation team went, the USAID logo was prominently displayed on banners, stickers, signboards, cloth (worn by the LMs), visual aids, clothing and hats. The US origin of the food commodities and the other resources of the program is widely known. Of the beneficiaries interviewed, 99.9% knew the US origin of the commodities. The remaining 0.1% cannot be blamed for their answer when they responded that the commodities come from Ruyigi where the CRS warehouse is located. When we probed, we realized that even the remaining small number of people knows that the food resources are a gift from the USA. As a result, the evaluation team determined that 100% of the beneficiaries know that both CSB and vegetable oil come from the generosity of the American people. The beneficiaries say they learned this from the THPs. However, we suggest that the Caritas warehouse managers or the THPs take 15 seconds before each distribution to remind beneficiaries of the US origin of the food. The scenario will be like this: “Good morning, everybody; we are about to start our food distribution. Who can tell me where the commodities come from?” It can be anticipated that the reaction will be a chorus with the correct response. Roof with empty CSB bags - Branding & Marking! 69 Shelter with ceiling made from cardboard from vegetable oil cartons III. SUMMARY OF RECOMMENDATIONS FOR COMMODITY MANAGEMENT The commodity management system for the Tubaramure project has no threshold issues. It is evident that the key players in this component are very knowledgeable about Title II food commodities management principles and regulations. Some of the recommendations and suggestions in this report are already being carried out. With good coaching and closer monitoring of the warehouse managers and the field sites, CRS/Burundi will make its system a stellar model with good practices that others can replicate. A summary of the twelve most important recommendations for optimal commodity management is presented below: 1. CRS should send a notification to POLUCON to remind them of their role and responsibilities with reference to the provisions in Regulation 11. CRS has a legal responsibility in case of an audit. (NB. The evaluator brought the issue to the attention of the CRS Commodity Manager who promised to take action immediately.) 2. CRS and Caritas should talk with the parishes to determine a reasonable salary to be paid to the workers that does not conflict with the church’s principles. 70 3. Review a salary increase scheme (1% or 2%) or a flat yearly bonus. (NB: Salary increase or bonus might not be significant in monetary value but could be a psychological incentive.) 4. Work out strategies to transfer or deposit money to parishioner or parish accounts on a regular basis. 5. Ask the warehouse keepers to file proof of payment to the laborers as they would with the other project documents. Each distribution site should always have proof of any payment made to a third party. 6. Review the “Fiche de Paie des Manutentionnaires” (Pay Statement for Laborers) and include additional names of the persons involved in the transactions, i.e., CRS, Gestionnaire (Caritas), Supervisor (Caritas). (NB. Instead of the Tubaramure logo, you may want to use the consortium’s logo.) 7. Ccommunicate the transportation/delivery plan through CRS/Caritas, Supervisors and warehouse staff. 8. Increase the number of contracted transporters. 9. CRS and Caritas should work with the community, the husbands and other volunteers to construct shelters to protect the women and their children against the elements. 10. CRS and Caritas should work with the husbands to construct latrines and simple hand washing stations under the guidance of the THPs, using the same principles as those taught to the women and the households in the communities. 11. Increase the different ways the invitations are given so that everyone is aware of the time and place for their group: e.g., parish/church, THPs, market place, village drinking place, etc. 12. Increase the number of distribution days. CROSS-CUTTING THEMES The evaluation team reviewed a number of themes that concern all three Intermediate Results. The most important of them are described here. I. MANAGEMENT, COORDINATION AND COMMUNICATION The Tubaramure program is very well-managed, with effective coordination and communication systems in place and strong leadership. Consortium members stated that they appreciate the willingness of program management to take their concerns seriously and to take decisive, timely action to resolve issues. The four consortium members seem to work well together both in Bujumbura and in the field. The fact that team members from the four NGOs are located in the same office at the provincial level facilitates team-building and exchanges. In the case of 71 Cankuzo, the Provincial Coordinator even insisted that staff from different NGOs share individual offices rather than allocating offices to each NGO. To promote communication and problem-solving within the consortium, there is a monthly Technical Working Group meeting and a quarterly Program Coordination Unit meeting. Both are held regularly and well-documented. Program managers organize periodic joint visits to the field. To date these have included three visits by NGO country directors and several group visits of national technical advisors. Traveling together and observing the same activities fosters team building and problem solving. Program management maintains excellent working relationships with key GoB partners including the administration. The Governor of Cankuzo and the Principal Advisor for the Governor of Ruyigi acted as key informants to the MTE team; both were well-informed about Tubaramure activities, had visited the program on several occasions, and provided concrete recommendations for improving the program. Close ties were also evident between Tubaramure staff and the MoH at all levels. While many program proposals discuss joint supervision visits and regular coordination meetings with host country institutions, not all follow through with these initiatives. Tubaramure is the exception with the MoH at the national and provincial levels proposing topics for joint supervision visits and then preparing the reports. II. MONITORING AND EVALUTION Tubaramure has excellent monitoring and evaluation systems in place and all four consortium members prepare regular reports documenting progress, problems and trends. Having the quarterly reports from IMC, FH and Caritas helped the team understand the activities implemented by each NGO. The reports were comprehensive and well-written. Overall, the Tubaramure program is exceptionally well-documented, with reports available for all key meetings, initiatives, and external visits. The periodic Performance Reports prepared by the IMC and CRS M&E experts were invaluable in helping the MTE team understand the status of the outcomes and outputs for each Intermediate Result. The availability of the March 2012 for key indicators was especially helpful. As mentioned earlier, a recent addition to the M&E portfolio is the institution of periodic anthropometric surveys, to be conducted every six months by ISTEEBU. The first survey, conducted in May-June 2012, provided up-to-the minute data for the evaluation team, enabling them to discern net improvements in the nutritional status of young children since Tubaramure was implemented. Although the M&E system is one of the strong points of the program, there are some proposed improvements to be made. The evaluators worked with M&E teams from IMC and CRS and program staff to review 1) the indicators in the Indicator Performance Tracking Table (IPTT) and 2) the Performance Management Plan (PMP). Revisions were proposed for both documents and if the consortium concurs with these changes, it is recommended that the proposed revisions 72 be submitted to USAID/FFP as soon as possible, requesting donor concurrence for the changes. See Annex M for the revised PMP and Annex N for the revised IPTT. One area that needs improvement is sharing more widely Tubaramure’s successes and lessons learned. The M&E systems provide a wealth of information on progress made yet there is little evidence that the consortium members have disseminated information about these successes to a wider audience. III. GENDER The original Tubaramure proposal discusses gender in several places, focusing on 1) involving men in health and nutrition activities and 2) empowering women to have more decision-making authority where the family’s diet is concerned. Competing program priorities and the absence of a well-defined plan at start-up meant that the gender focus was often missing from the program and it is known more as a program for women than for men. Program management is beginning to remedy this situation, bringing in a gender consultant, organizing gender training for staff, and making tentative efforts to include men in trainings and peer education. The recent final evaluation report of the other MYAP implemented by CRS and IMC also contains a number of lessons learned that the consortium partners are reviewing in order to strengthen the gender focus of Tubaramure. For future programs, it may be helpful to 1) hire a full-time person responsible for gender from start-up and 2) design the program so that there is a cross-cutting gender objective. IV. TRACKING GRADUATES One way to determine the effectiveness of all three Intermediate Results is to track mothers and children who have graduated from the ration distribution component. The Public Health/Nutrition Evaluator attended a meeting of the Technical Advisor for Food Utilization, the Deputy Chief of Party, and the Senior Technical Advisor for Nutrition from CRS/Baltimore. Ideas were shared for developing a plan to track graduates. Some of the questions to consider in designing a follow-up study are:  Do the children who have graduated continue to maintain their nutritional status and overall health? If not, what explains the difference between children who do well and those who don’t? For example, does the family’s economic status play a role?  Are caretakers maintaining healthy practices: ENAs and EHAs? Utilization of health services for women and children? 73  If a woman becomes pregnant after graduation, does she register for prenatal consultations in her first trimester? Does she attend at least three prenatal consultations? Does she deliver at the health center? LESSONS LEARNED The following lessons learned fall into one of four categories:  Things that were done well from the beginning of the program:  Integrating Tubaramure into the MoH and using the MoH policies and protocols has been key to the success of IR-1. (IR-1)  Although there are opportunity costs, the protection ration destined for other household members is necessary to ensure that the mother-child unit receives adequate rations. (IR￾3)  Building in regular coordination meetings and joint visits (among program staff and with external partners) has fostered team spirit and reinforced effective working relationships.  Sound M&E systems, including comprehensive reporting, permit accurate tracking of progress, early identification of issues, and timely resolution of problems.  Timely adjustments in strategies or activities to address an issue identified during implementation:  Having a beneficiary data base is more efficient than maintaining manual records. It saves enormous amounts of staff time, reduces the margin for errors and makes it more difficult for people to cheat the system. (IR-3)  Deciding to hire additional THPs as substitutes (suppléants) was a timely adjustment, ensuring that activities continued on schedule even if a THP was on leave. (IR-2)  The relatively late start on addressing some of the gender issues will need to be remedied quickly if certain objectives are to be met (e.g., involving men more fully in family health and nutrition issues and promoting greater decision-making power for women, especially where decisions on resource allocation and family diet are concerned).  Issues that should be addressed now if possible: 74  The COSAs should be an integral part of the community-level strategy, supporting the LMs and CHWs and serving as an interface between the program, the MoH and the community. (IR-1, IR-2)  Care Groups of Leader Mothers often have a finite life (average of two years) so it is important to provide incentives for these groups to remain active. In Tubaramure the CRS SILC program and income-generating activities for CGs are proven ways to provide motivation for LMs to remain active and engaged. The same incentives might work for Papas Tubaramure as well. (IR-2)  The Tubaramure consortium should work in concert with partners to develop a concrete, detailed exit plan, outlining what achievements could be maintained, what activities might be continued, who will accept responsibility for these activities, and what actions need to take place from now to the end of the program for the exit strategy to succeed.  A realistic plan to track a sample of graduates (women and children) should be developed. (IR-3)  To fully achieve an Intermediate Result that focuses on improved utilization of food in the context of the Tubaramure program area, more activities to improve accessibility of and access to food (mésures d’accompagnement) should be considered. (IR-3)  To encourage program staff, partners and especially the volunteers such as LMs and CHWs, share the results of the qualitative MTE and the quantitative ISTEEBU anthropometric survey so that they can see how well they are doing.  If men are to be effective peer educators, they also need incentives, including perhaps a set of BCC materials directed at men. (IR-2)  To accurately evaluate the effectiveness of interventions, it is useful to have a system in place for tracking mothers and children who have graduated from the ration distribution activity. (All IRs)  Issues that the Tubaramure program may not be able to address (either because the opportunity has passed or the issue is beyond the control of program management) but that should be considered the next time a similar program is designed  If operations research (OR) is to be conducted, the protocol should be available during the program design phase in order to incorporate research activities into the action plan. Otherwise, delays in program implementation are likely to occur. (NB. Program managers estimate that up to eight months were lost while waiting for the IFPRI baseline study to be conducted.) 75  If the program managers had had the basic IFPRI protocol for the OR early on, including the fact that there would be four arms, they could have addressed some of the more controversial issues that arose (e.g., the fact that in the 15 control collines, there would be no food distribution) before they became major distractions.  If the OR has a control group, other program activities should not be included in this geographical area. In the case of Tubaramure, IR-1 activities take place in the health centers in the control collines.  If gender concerns are to be addressed well, it may be useful to have a cross-cutting Intermediate Result or at a minimum, a well-defined plan that is operational from the start of the program.  To ensure that the gender lens is maintained, assign the responsibility for maintaining a gender focus to someone with gender expertise.  For monitoring and evaluation IMC-Tubaramure uses health facility records for IR-1 and FH-Tubaramure uses random household surveys of beneficiaries for IR-2. Since the methodologies are different, it is difficult to determine to what extent the results for IR-1 include program beneficiaries. (M&E)  Agents responsible for BCC (in this case the THPs) shouldn’t have too heavy a work load. The ratio of THPs to LMs and beneficiaries should allow for adequate follow-up of activities, regular coaching, and trouble-shooting. (IR-2)  It is important to have the BCC materials ready as soon as the beneficiaries are identified. (IR-2)  The BCC strategy may have been too ambitious in proposing five modules as beneficiaries have multiple demands on their time as do the THPs and LMs. (IR-2)  When LMs rely primarily on group meetings to disseminate lessons, this can displace home visits and reduce the effectiveness of the BCC strategy. (IR-2)  When households are widely dispersed, consider lowering the ration of LMs to beneficiaries so that LMs can make regular home visits. (IR-2) SUMMARY OF PRINCIPAL RECOMMENDATIONS This report includes suggestions as well as major and minor recommendations. The evaluation team considers the following recommendations to be the most important at this point in the program: 76 1. Continue to improve the gender focus of the program. 2. In collaboration with Tubaramure’s principal partners, develop an exit strategy to promote sustainability; include at a minimum what achievements could be maintained, what activities might be continued, who will accept responsibility for these activities, and what steps need to be taken from now to the end of the program for the exit strategy to succeed. 3. Improve joint efforts between IMC and the other consortium partners to mobilize the LMs and CHWs for follow-up of malnourished children. 4. Combine training to improve the collaboration between the CHWs and LMs. 5. Ensure that children with moderate or mild malnutrition are also identified, registered and followed-up in the community as was planned in the original proposal. Look into the possibilities (and feasibility) of community-based solutions for treatment of malnourished children, such as PD-Hearth or FARN. 6. Consider the possibility for local production of RUTFs as a long-term strategy. 7. Propose to MoH that growth monitoring activities and immunization days be combined on the same day at the health centers. 8. Carry out a study with LMs and Papas Tubaramure to determine what incentives – tangible and intangible – are most likely to motivate them to remain involved and active. At a minimum the LMs – and the Papas Tubaramure – should be informed of the results of the midterm evaluation (including the ISTEEBU anthropometric survey) and publicly recognized for their contribution to the successes and achievements to date. 9. Complete the plan for tracking a sample of women and children who have graduated from the ration distribution activity. 10. To better understand the reasons for the dropout rate among graduates, conduct a barrier analysis of beneficiaries and LMs who have graduated and develop a plan to address barriers to continued participation. 11. Reinforce the messages on malaria. Renewing the emphasis on what can be done at the household and community level to prevent and manage this disease could make a significant difference in health outcomes not just for children but for other family members as well. 12. Carry out a study with LMs and Papas Tubaramure to determine what incentives – tangible and intangible – are most likely to motivate them to remain involved and active. 77 13. FH-Tubaramure could work with IMC colleagues to determine whether it would be useful to conduct a limited study on the most effective ways to prevent diarrhea and manage it at home. 14. Determine the feasibility and potential effectiveness of adding activities to increase availability 15. of and access to food to the program (e.g., agricultural and animal production and SILC.) Consult with consortium members and USAID/FFP about the possibility of realigning the current budget or increasing the monetization or 202 (e) levels. 16. Continue to explore the possibility of developing a local substitute for CSB and vegetable oil, using the flours already being produced by women entrepreneurs. 17. CRS and Caritas should talk with the parishes to determine a reasonable salary to be paid to the workers that does not conflict with the church’s principles. 18. Review a salary increase scheme (1% or 2%) or a flat yearly bonus. (NB: Salary increase or bonus might not be significant in monetary value but could be a psychological incentive.) 19. Work out strategies to transfer or deposit money to parishioner or parish accounts on a regular basis. 20. Ask the warehouse keepers to file proof of payment to the laborers as they would with the other project documents. Each distribution site should always have proof of any payment made to a third party. 21. Review the “Fiche de Paie des Manutentionnaires” (Pay Statement for Laborers) and include additional names of the persons involved in the transactions, i.e., CRS, Gestionnaire (Caritas), Supervisor (Caritas). (NB. Instead of the Tubaramure logo, you may want to use the consortium’s logo.) 22. Communicate the transportation/delivery plan through CRS/Caritas, Supervisors and warehouse staff. 23. Increase the number of contracted transporters. 24. CRS and Caritas should work with the community, the husbands and other volunteers to construct shelters to protect the women and their children against the elements. 25. CRS and Caritas should work with the husbands to construct latrines and simple hand washing stations under the guidance of the THPs, using the same principles as those taught to the women and the households. CRS Burundi Programme Tubaramure PM2A Terme de référence de l’expert du Ministère de la Santé Publique et de la lutte contre le SIDA à l’Evaluation à mi-parcours du programme PM2A-Tubaramure 1. Contexte Catholic Relief Services (CRS) en partenariat avec International Medical Corps (IMC), Food for the Hungry (FH) et la Caritas/Burundi, a obtenu de la part du Bureau « Food for Peace », de l’Agence Américaine pour le Développement (USAID) une subvention pour une période de cinq ans (2009-2014) en vue de prévenir la malnutrition chez les enfants de moins de deux ans dans les provinces burundaises de Cankuzo et de Ruyigi (toutes les collines). Plus précisément, le projet vise (i)- à améliorer la qualité des services prénataux et postnataux, (ii)- à contribuer à l’adoption de bons comportements et, (iii)- à la supplémentation nutritionnelle chez les femmes enceintes, allaitantes et les enfants de moins de deux ans, avec des vivres Title II, ainsi que les aliments locaux. Apres deux ans et demi d’exécution, le programme doit faire l’objet d’une évaluation à mi-parcours qui a été confiée à une équipe de trois (3) consultants internationaux. Cette évaluation qui doit durer un mois débutera le 24 Mai 2012. Le consortium, dans son souci de réaffirmer sa collaboration avec son partenaire principal qu’est le Ministère de la Santé et de la lutte contre le SIDA, a demandé et obtenu de la part de la Ministre de tutelle, la mise à disposition d’un expert qui participera à l’évaluation en tant qu’observateur. 2. Rôle de l’expert du Ministère de la Santé et de la lutte contre le SIDA : Faisant suite à la lettre Réf 630/280/CAB/2012 de Son Excellence Honorable Mme la Ministre de la Santé Publique et de la Lutte contre le Sida, l’expert du Ministère de la Santé publique et de la lutte contre le SIDA participera à l’évaluation à mi-parcours du programme PM2A- Tubaramure en tant qu’observateur et répondant d’appoint. Il servira aussi d’interface entre les consultants et le Ministère de la Santé publique et de la lutte contre le SIDA ainsi que ses démembrements au niveau province, commune, etc. L’expert exécutera entre autres tâches suivantes :  Accompagner l’équipe de l’évaluation à mi-parcours sur le terrain et assister aux différentes interviews que va conduire l’équipe des évaluateurs.  Faciliter les contacts entre les évaluateurs et les structures de santés dans les deux provinces  Apporter des réponses aux éventuelles questions que les évaluateurs viendront à poser au sujet de la politique sanitaire au Burundi  Fournir, à la demande des consultants, des documents sur la santé au Burundi. A défaut de pouvoir leur fournir les documents demandés, les orienter vers qui de droit. 3. Durée de la prestation La prestation de l’expert du Ministère de la Santé et de la Lutte contre le SIDA durera 10 jours (dans la période allant du 28 Mai au 9 Juin 2012) et sera constituée essentiellement de réunions et de la visite de terrain en compagnie des consultants. 4. Documents de référence :  Plan de l’évaluation à mi-parcours, incluant les termes de référence des évaluateurs  Document du projet  Rapports du projet 5. Prise en charge : Le CRS Burundi prendra en charge tous les frais en rapport avec cette mission. CRS Burundi Prevention of Malnutrition in children under 2 years of Age (PM2A￾Tubaramure-MYAP 2009-2014) Mid-term Evaluation Plan USAID\FFP Title II – PM2A-Tubaramure MYAP Implemented by Cooperative Sponsors (CRS in consortium with IMC, FH and Caritas/Burundi in Burundi (2009-2014) No- AID-FFP-09-00004-00 February, 2012 PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 2 1. Introduction/Background information Catholic Relief Services/Burundi (CRS/BI) is the lead agency for a five-year USAID-financed Multi-Year Assistance Program (MYAP) entitled Tubaramure (“Let’s Help Them Grow” in Kirundi) with sub-awards to International Medical Corps (IMC), Food for the Hungry (FH) and CARITAS Burundi. The program uses a “preventing malnutrition of children under the age of two” approach (PM2A) which was first identified by IFPRI et al.1 as more effective in reducing malnutrition than historically favored remedial methods. The objectives of the Tubaramure program are to improve the health and nutritional status of pregnant and lactating women and children less than two years of age and to strengthen the quality and delivery of health care services. Program activities include: improving the capacity of health facilities/staff and community health workers to deliver high quality general health and nutrition services, promoting household and community level behaviors that prevent maternal and childhood illnesses and support good nutrition using CARE Groups as the delivery mechanism, and improving the caloric and nutrient intake of mother and children under two through culinary demonstrations and monthly delivery of Title II commodities. Approximately 51,075 mother-child pairs are targeted over the five years (July 2009 to October 2014) of the $45 million grant. The program at mid-point of execution had reached 46,000 mother-child pairs. Integrated into the Burundi Tubaramure program is a research program conducted by the International Food Policy Research Institute (IFPRI) with funding from USAID through the Food and Nutrition Technical Assistance II (FANTA-2) and FANTA-2 Bridge projects. Over the life of the project, IFPRI will conduct a series of studies to assess the impact and cost effectiveness of Tubaramure on child nutritional status, as well as to evaluate the differential and absolute impact of varying the duration of receiving food rations. 1.1. Objectives of the Evaluation The overall objective of the PM2A midterm evaluation is to assess achievements against targets, to determine progress against the strategic objective and identify the project’s strengths and weaknesses from inception to mid-term; and to provide recommendations for improving project implementation. 1 The International Food Policy Research Institute (IFPRI), in collaboration with World Vision￾Haiti, Cornell University, and the Food and Nutrition Technical Assistance Project (FANTA), provided the first programmatic evidence that the blanket targeting of a food assisted Maternal and Child Health and Nutrition program to all children 6-24 months old (preventive approach) was more effective in reducing the prevalence of stunting, wasting, and being underweight than the traditional recuperative approach based on targeting underweight children (weight-for-age Z-score (WAZ) < -2) under five years of age (Ruel et al. 2008). The Food and Nutrition Technical Assistance II Project (FANTA-2), managed by the Academy for Educational Development (AED), commissioned a new study to refine and strengthen the preventive approach, now called the “Preventing Malnutrition in Children Under Two Years of Age Approach” (PM2A). Burundi and Guatemala were selected to implement the PM2A, and IFPRI and FANTA-2 were tasked with designing and carrying out a rigorous evaluation of the impact and cost-effectiveness of the PM2A in these two contexts. PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 3 Objectives are as follows: • Assess successes achieved, constraints encountered in implementing major activities (outputs) and actions taken to overcome them in relation with associated performance indicators outlined in the Performance Indicators Tracking Table (IPTT); qualitatively assess these results and their effects and document lessons learned. • Review the overall and operational strategy of program implementation with regard to achieving results and determine if proposed interventions are sufficient to reach the desired outcomes. • Determine if the current management design and roles and responsibilities of the prime and sub￾awardees are adequate. • Assess the effectiveness of program commodity management. • Evaluate the effects of capacity building on communities and partners (care groups, government services, partner NGOs, communes, etc.) with respect to health/nutrition, risk prevention and management. • Assess the quality of partnership between provincial health service providers and consortium members. • Review the quality of program implementation with regard to expected reductions in vulnerability of target groups; particularly children aged 0 to 2 and child-bearing women in the program zone of activity. • Determine the effectiveness and efficacy of the M&E systems in place. • Determine if the Tubaramure program design supports IFPRI’s research efforts; • Assess how the Ministry of Health and other stakeholders have been influenced by the Care Group model. • Evaluate the program’s impact on child care practices at the mid-term and if that impact is sustainable. • For each specific objective, make recommendations as needed to improve performance. 1.2. Description of the PM2A-Tubaramure program The PM2A-Tubaramure program is summarized through the following result framework: SO: Malnutrition in children under 2 is prevented IMC IR-1 1: Women and Children 0-59 months access quality nutrition and health services 1.1: Pregnant and lactating women access pre and postnatal care services 1.2: Implementation of National IMCI plan is supported 1.3: Health Facility Growth Monitoring (GM) services comply with national protocols 1.4: SAM is detected and referred for treatment FH IR-2 2: Households practice appropriate 2.1: Households Essential Nutrition Actions (ENA) 2.2: Households adopt Essential Hygiene Actions (EHA) 2.3: Households adopt prevention and management behaviors for maternal and PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 4 health and nutrition behaviors childhood illnesses CRS￾Caritas IR-3 3: Eligible women and children have increased intake of diverse foods 3.1: FFP rations distributed to eligible women and children 3.2: Mothers and children use FFP rations appropriately 3.3: HHs use appropriate local foods in addition to FFP ration In collaboration with the MoH/Burundi (PRONIANUT, Provincial Departments of Health), FANTA2/IFPRI and the populations of Cankuzo and Ruyigi All community-level activities are implemented through CRS Tubaramure Health Promoters (THP). THPs report to the Head of the Provincial Office and are responsible for: training Lead Mothers (LM) through Care Groups on Behavior Change Communication (BCC) activities; for collaborating with Community Health Workers (CHW) on referrals; and working with Community Health Centers (CHC) to create a feedback mechanism by which the community can be informed of households’ progress toward the reduction of malnutrition. Technical advisors from each consortium partner (IMC-General Health, FH-BCC, and CRS-Food Utilization) are responsible for: designing curricula in collaboration with Ministry of Health (MOH) when relevant; implementing trainings; providing technical support to THPs to ensure that targets are met; and making adjustments to approaches or messages when barriers are identified. This core group of field staff is supervised and supported by management and administrative staff at provincial and national (Bujumbura) levels, and work directly with MOH and community structures. 1.3. Achievements as of September 2011: In addition to start-up activities (acquisition of vehicles, equipments and materials, opening of provincial offices, training of key staff, etc), the program achieved the following by the end of FY11: enrollment of a cumulated total of 46,000 mother-child pairs; nutrition equipment support to a network of 50 Heath centers and 11 hospitals, training of 39 health promoters, 24 nurses, 535 community Health workers, 4,920 Leads Mothers over 420 CARE groups, production of 24 behavior change communications episodes which have been broadcast over 40 times, distribution of Title-II commodities (a cumulative total of 9,000 MT of CSB and vegetable oil) in over 27 distribution sites. 1.4. Geographic Coverage Tubaramure targets 51,075 mother-child units from pregnancy to age 24 months in Cankuzo and Ruyigi provinces. The program covers a total of 268 collines/villages in the 12 communes of the two provinces. 1.5. Key Partners: CRS is the consortium lead and is responsible for commodity management and assures overall program /compliance responsibilities. In collaboration with Caritas Burundi, CRS executes IR 3: Eligible women and children have increased intake of diverse food. Activities include program and commodity management, provision of food rations to beneficiaries and training on food diversity. PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 5 International Medical Corps (IMC) is the implementing partner for IR 1: Women and children under 5 access quality nutrition and health services. IMC is responsible for the provision of equipment to health centers and capacity building of health workers. Food for the Hungry (FH) is the implementing partner for IR 2: Households practice and appropriate health and nutrition behaviors. FH trains beneficiaries using the Care Group model on key household behaviors; Essential Health Actions (EHA) and Essential Nutrition Actions (ENA). 1.6. Major Implementation Challenges Key challenges to date are: The consortium faced numerous challenges in implementing a “first-ever” PM2A MYAP. Delays in launching the research portion of the program impacted the enrollment schedule, timeline and duration of grant. The exit strategy of fostering ownership of the program by government and community stakeholders is on-going. The consortium is exploring and experimenting with new approaches to low-cost food fortification in Burundi that can replace TITLE II commodities by the project’s end. 2. Team Composition 2.1. Roles and responsibilities of consultants: • One international consultant with Public Health & Nutrition expertise to evaluate IR3 & 2 of the program; S/he will serve as the team leader • One international consultant with Primary Health Care and Community Health expertise to evaluate IR1 • One international consultant with expertise in logistics and supply chain management to evaluate IR3. Experience with Title II programs required. CRS will recruit the international team on behalf of the Consortium. The international consultant with strong experience in Public Health & Nutrition will serve as the team leader of the evaluation team. He/she will be familiar with the PM2A approach and MYAP evaluations, possess strong leadership abilities and be knowledgeable on issues related, monitoring and evaluation, program management. The main task of the international consultant will be to lead the team and assure quality of the work throughout the evaluation process. S/he will give due attention during the phases of development of data collection tools, data analysis, report writing and other relevant deliverables by the local consulting firm. The Primary Health Care/Community Health specialists will be responsible for the evaluation of IR1 activities. The consultant will evaluate the quality of the nutrition and health care services that are provided in the network of the primary health care centers and hospitals. A review of the trainings conducted and the skill levels of the trained community health workers, primary health care staff and Tubaramure Health Promoters (THP) will also be carried out. The international logistics and supply chain management consultant will have experience in Title II commodity distribution and management. S/he will demonstrate abilities in assessing the relevance, the effectiveness and the efficiency of commodity management and distribution systems in order to identify constraints, document lessons learned and provide recommendations for future programming.\ PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 6 Prior to the qualitative phase of the midterm evaluation, quantitative data such as birth weight, height and weight of children under 5 and Household Diet diversity will be collected to assess progress made by the PM2A approach children’s growth after two years 2.2. PM2A Mid-Term Evaluation Staffing/organization Oversight Committee: Provide additional guidelines to the consulting team, make final review of the MTE, and also make strategic and managerial decisions per the recommendation of the Technical Team. Technical Team: The main task of the team is to develop the MTE and lead the process of bid preparations, selection and recruitment of the international and local consultants. The Technical Team is organized by drawing representatives from: CRS/ Burundi, IMC/Burundi; FH; Caritas/Burundi; and USAID and Ministry of Health (observers). The technical team also provides administrative and technical supervision of the consultants’ work. International Consultants: A three-member team (Public Health Nutrition, Primary Health Care Specialist, Logistic & Supply Chain expert) will be headed by a Team Leader (the Public Health Nutritionist). The Team Leader will work closely with the Technical Team and is responsible for the overall evaluation process and output. Specific tasks are described in the attached individual Terms of Reference. CRS Technical Coordinators: Technical coordination is provided by CRS staff, generally the CRS’ M&E specialist and the Tubaramure Chief of Party, who ensure coordination and linkage between field operations and consultants in each evaluation area. The coordinators liaise with the consultants with respect to the activity in the field. The Technical Coordinators communicate with the Team Leader. The best way to represent the relationship between the Technical Coordinators and the Team Leader is one of consultation and facilitation. CRS will serve as the focal point for communicating with the Team Leader on all matters related to general administration and logistics. 3. Role and Responsibility of Consortium 3.1. Lead agency (CRS) - Facilitate coordination throughout the evaluation cycle - Coordinate recruitment of all consultants - Organize joint meetings between Consortium members and midterm evaluation team 3.2. Consortium members (CMs) CRS and Consortium members (CM) will have the responsibility of facilitating the evaluation in their respective operational areas. CMs will assign Monitoring and Evaluation officers to support the evaluation team, provide timely responses and facilitate field work throughout the evaluation process. Consortium members led by CRS have the final approval of the team’s methodology. In addition, CRS and Consortium members will provide the team with all relevant documents needed for the consultancy. CRS will also provide support to the evaluation team by making arrangements for all logistics: in￾country accommodation, hotel reservations, transport, and contact with program staff, partners, Ministry of Health and local stakeholders. PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 7 The evaluation team members are expected to bring their own laptops/computers, and any appropriate software necessary for the required final report. CRS will provide access to printers, copiers, and all communication facilities (internet, local telephone). 4. Team and Individual Team Member Mid-term evaluation plans 4.1. Key Evaluation Areas The Mid-term evaluation required to carry out the detailed assessment of the PM2A in terms of the basic evaluation areas: - Relevance: Investigate in detail the extent to which the objectives of the PM2A are consistent with the needs of the beneficiaries, the recipients, the host country and donor. - Effectiveness: Look into whether or not the proposed development strategy achieves its long term goals. Verify whether the program meet targets. - Efficiency: Examine how economic inputs (resources, expertise, time, etc.) are converted into outputs. Conduct a comparison of the value (not necessarily monetary) of the development strategy and the resources allocated to achieve outputs. - Sustainability: Assess the likelihood that the positive effects of the PM2A (such as assets, skills, facilities or improved services) will persist for an extended period after the end of the program (financial assistance). The above mentioned four key evaluation areas are used as a lens that should be taken into account when answering the following key evaluation questions; detailed questions related to technical areas are listed in various terms of reference (see annexes): 4.2. General mid-term evaluation questions Design, Implementation and Achievements:  How effective is the program at reaching the vulnerable population? What could be done to improve targeting?  What interventions have been more or less successful in meeting targets?  Which interventions are most critical and/or effective in achieving project objectives and intermediate results?  What are the factors that hinder/assist the effective implementation of the program?  What improvements can be made to the design to improve results?  What improvements can be made in the implementation of the program in order to improve results?  Is the program well integrated in the local government’s strategy and priorities? Are there steps that could be taken to improve the integration as well as food security impacts through greater integration?  How effective is the program at reaching women? What could be done to improve women's participation?  What is the quality and frequency of community workshops organized by PM2A￾TUBARAMURE animators and other actors to support gender? Capacity strengthening:  Is the technical field staff well trained and supervised?  What technical areas, if any, need strengthening? PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 8  Is the program effectively developing the capacity of counterparts and/or partners? If not how could the design or implementation be altered to improve capacity strengthening?  Is the program effectively enabling, or developing the capacity of beneficiaries? If not how could the design or implementation be altered to improve capacity strengthening? Sustainability:  Are the outcomes related to adoption of better practices sustainable, i.e. are participants likely to continue after the project ends?  Which outcomes are likely or unlikely to be sustainable, and why?  What can be done to increase the sustainability?  Will it be possible in the remaining years of the program to hand off responsibility to a local entity? If not, what additional actions need to be taken?  Are activities appropriate and are they efficient in helping achieve results?  Are there any lessons learned? What recommendations on activities if taken into account will contribute to improving performance? Monitoring and Evaluation/Reporting:  Are M&E data collected adequately and reported regularly and in a timely fashion?  Are M&E data and anecdotal information used for management purposes?  How can M&E data and anecdotal information be better used in program management decisions?  Does the technical staff use M&E data and anecdotal information to conduct their work and assess progress? Is the information being used to manage the program?  How can they use it more effectively? 4.3. Technical Sector Questions Nutrition  Are nutritional services carried out according to standard?  Do targeted beneficiaries receive appropriate health and nutritional messages?  To what extent are infant feeding practices being practiced?  To what extent are Behavior, Communication and Change (BCC) activities being effective?  Are health and nutrition BCC materials developed and tested appropriately? Are they tailored to the user, actionable, accurate and linked to growth promotion messages (where growth monitoring is being implemented)?  Is there change in community practice regarding child care and management due to PM2A program interventions?  Are the program eligibility and graduation criteria appropriate, given the objectives and assumptions of the program? If not, how should it be modified? Primary Health Care  Assess the achievements the health component of the program (IR 1);  Evaluate the relevance, effectiveness and efficiency health equipment provided by the program to the target health centers;  Identify constraints in the implementation of the health component activities;  Document lessons learned and provide recommendations for future programming,  To what extent has the quality of the nutrition and health services provided in the health care centers improved?  Has the training provided to health care staff been appropriate for their education/skill level?  Has the skill level of the health care staff improved from training? PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 9 Commodity Management  Are food distributions program carried out according to standard?  Are correct procedures and best practices used in receiving, distributing and storing food commodities?  Is the food distribution system design efficient?  Are there good procedures to monitor food distributions and in a timely manner?  Are the recommended food rations respected? What are the weaknesses of the distribution scheme and what are the recommendations to address them?  Does the current system protect against “false” beneficiaries from receiving commodities?  Are beneficiaries well-targeted for food rations?  Do beneficiaries understand the quantities of food they should receive?  Do targeted beneficiaries receive appropriate food rations?  Are commodities being managed appropriately? To what extent? 5. Methodology for the mid-term evaluation 5.1. Methodology The evaluation will focus on qualitative rather than quantitative methodologies and will use an iterative process for information analysis. Data will be analyzed continually throughout the collection period. An evaluation action plan and tools will be developed and adapted before the start of field work; this will maintain a degree of flexibility in order to respond to questions and issues that arise during the process. The evaluation team will propose a methodology to CRS (Oversight Committee) for validation prior to field work. Quantitative data such as birth weight, height and weight of children under 5 and HH Diet diversity will be collected prior to the qualitative phase of the midterm evaluation. The proposed evaluation methods will include but not limited to the following:  Document Review and Quality Assurance: gathering and review of pertinent documentation such as field trips reports, annual progress reports etc. In addition an evaluation will determine compliance with set standards or criteria.  In-Depth Individual Interviews with project/program managers, technical staff etc.  Key Informant Interviews with partners and authorities.  Focus Group Interview with people who share similar experiences or characteristics; allows group members to discuss opinions with others.  Observation; site visits and discussion with beneficiaries to gain additional information  Periodic and Final Debriefings in Bujumbura to inform CRS and partners and FFP on progress 5.2. Protocols for the mid-term evaluation The team leader will organize the collection of the qualitative information through focus group and individual interviews. The team will design the sample of communities to be visited for the data collection. The evaluation team will be in charge of the data collection, compilation, transcription and analysis. The evaluation report should include the following sections: Title page with date, and logos Executive summary Introduction PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 10 Objective of evaluation Brief description of the program Detail analysis of findings by technical sector Accomplishments and constraints, progress towards objectives/IRs, program quality (management, M&E, commodity, program sustainability, relationship to current and future USAID SOs), integration of PM2A components. Summary of findings by technical sector and regions Cross cutting issues Lessons learned Recommendations by technical sector Annexes Composition of consultant team, Tools and Methods, List of sites visited, List of key informants, References, Indicator performance tracking tables (IPTT), List of acronyms etc. PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 11 6. Time and Deliverables The mid-term evaluation will be implemented under the supervision of the Team Leader. Though tentative dates are shown below, evaluation activities will commence only when the mid-term evaluation Team Leader is recruited. Summary of timetable and deliverables are as follows: Task/deliverables Timeline/deadline Responsible body Primary Health Care Specialist Supply Chain/ commodi ty Expert Public Health Expert/Te am Leader Review documents and draft evaluation plan, methodology and tools 3 days prior to arrival (March 28) X X X Arrival in Burundi 1 day (April 02) X X X Debriefing and meet with key PM2A-TUBARAMURE staff to review, finalize, methodology and tools with evaluation Team 2 days (April 04- 05) X X X Field interview/data collection and discussion with community leaders, consortium members and local government authorities (4 day in each province) 8 days (April 05-13) X X X Sectoral data analysis and synthesis by team Leader 2,5 days (April 13- 17) X X X Lead workshop with PM2A￾TUBARAMURE team to present/discuss preliminary findings 0,5 day (April 18) X Submission of draft reports prior to departure from Burundi 3 days (April 21) X De-briefing with FFP/USAID X X X Final reports due 5 working days after receipt of comments X Total Consultancy 25 days 7. Illustrative list of Reference documents  Burundi PM2A-TUBARAMURE narrative Proposal and relevant Appendices (Performance Monitoring plan, and Detailed implementation plan)  IPTT (Indicator Performance Tracking Table)  PMP (Performance Management Plan)  Baseline study assessments/reports  Annual Result reports (FY10 and FY11)  Quarterly reports (from CRS, IMC and FH)  M&E quarterly reports (3 reports for FY11) PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 12  M&E annual reports (for FY10 and FY11)  DQA reports  Field trip reports PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 13 Annex 1 Terms of Reference for the Team Leader/Public Health Nutritionist Midterm Evaluation CRS Burundi PM2A-TUBARAMURE 2009-2014 Introduction/Background information Catholic Relief Services/Burundi (CRS/BI), in partnership with International Medical Corps (IMC), Food for the Hungry (FH) and CARITAS Burundi are implementing a five-year USAID-financed Multi-Year Assistance Program (MYAP) entitled Tubaramure (“Let’s Help Them Grow” in Kirundi) in Ruyigi and Cankuzo provinces of Burundi. The overall objective of the program is to “prevent malnutrition in children under the age of two” by enrolling all pregnant women and mothers of children younger than 6 months in Ruyigi and Cankuzo provinces and then providing them with a package of services until the child reaches 24 months. The benefits include: 1) access to quality health and nutrition services; 2) education on key health, hygiene and nutrition practices through the CARE Group approach; and 3) improved caloric and nutrient intake through culinary demonstrations and monthly delivery of Title II commodities. The PM2A-TUBARAMURE program is pursuing the following strategic objective: Strategic Objective: Malnutrition in children under 2 years of age is prevented The anticipated results are: 1: Women and Children 0-59 months access quality nutrition and health services 2: Households practice appropriate health and nutrition behaviors 3: Eligible women and children have increased intake of diverse foods Objectives of the Evaluation Overall Objective The objective of the midterm evaluation of PM2A is to objectively assess progress towards achieving the strategic objectives above, particularly in the areas of health care, behavior change, nutrition, and commodity management. Specific Objectives for the Team Leader /Nutritionist:  Lead the team and ensure quality of the work throughout the evaluation process;  Give due attention during all phases of development of data collection tools, data analysis, report writing and other relevant deliverables by the team members on behalf of the CRS and its partners;  Be responsible for leading sector specialists;  Be responsible for reviewing and/or finalizing the deliverables by the other team members  Assessing the achievements under IR2/3;  Evaluate the relevance, effectiveness and efficiency of Nutrition interventions;  Analyze the effectiveness of coordination with the government and other actors, identify constraints (logistics and operations in general),  Document lessons learned and provide recommendations for future programming  Be responsible for achieving assignment objectives as well as periodic briefings and presentations.  Be responsible for the final evaluation report. Composition of the Evaluation Team PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 14 CRS will recruit the international team on behalf of the Consortium. The international consultant with strong experience in Public Health & Nutrition will serve as the team leader of the evaluation team. He/she will be familiar with the PM2A approach and MYAP evaluations, possess strong leadership abilities and be knowledgeable on issues related, monitoring and evaluation, program management. The main task of the international consultant will be to lead the team and assure quality of the work throughout the evaluation process. S/he will give due attention during the phases of development of data collection tools, data analysis, report writing and other relevant deliverables by the local consulting firm. The Primary Health Care/Community Health specialists will be responsible for the evaluation of IR1 activities. The consultant will evaluate the quality of the nutrition and health care services that are provided in the network of the primary health care centers and hospitals. A review of the trainings conducted and the skill levels of the trained community health workers, primary health care staff and Tubaramure Health Promoters (THP) will also be carried out. The international logistics and supply chain management consultant will have experience in Title II commodity distribution and management. S/he will demonstrate abilities in assessing the relevance, the effectiveness and the efficiency of commodity management and distribution systems in order to identify constraints, document lessons learned and provide recommendations for future programming. Evaluation Questions General questions for the evaluation team: Design, Implementation and Achievements:  How effective is the program at reaching the vulnerable population? What could be done to improve the targeting?  What interventions have been more or less successful in meeting targets?  Which interventions are most critical and/or effective in achieving project objectives and intermediate results?  What are the factors that hinder/assist the effective implementation of the program?  What improvements can be made to the design to improve results?  What improvements can be made in the implementation of the program in order to improve results?  Is the program well integrated in the local government’s strategy and priorities? Are there steps that could be taken to improve the integration as well as food security impacts through greater integration?  How effective is the program at reaching women? What could be done to improve women's participation?  Assess the quality and frequency of community workshops organized by PM2A￾TUBARAMURE animators and other actors for the gender activity. Capacity strengthening:  Is the technical field staff well trained and supervised? What areas, if any, need strengthening?  Is the program effectively developing the capacity of counterparts and/or partners? If not how could the design or implementation be altered to improve capacity strengthening?  Is the program effectively enabling, or developing the capacity of beneficiaries? If not how could the design or implementation be altered to improve capacity strengthening? Sustainability: PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 15  Are the outcomes related to adoption of better practices sustainable, i.e. participants are likely to continue after the project ends? Which outcomes are likely or unlikely to be sustainable, and why? What can be done to increase the sustainability?  Will it be possible in the remaining years of the program to hand off responsibility to a local entity? If not, what additional efforts need to be undertaken?  Assess appropriateness of activities conducted, and make recommendations (not just how the implementation of approved activities could be improved, but whether different activities might be more appropriate). Monitoring and Evaluation/Reporting:  Are M&E data collected and reported regularly and in a timely fashion?  Are M&E data and anecdotal information used for management purposes? Can M&E data and anecdotal information be better used for program management?  Does the technical staff use M&E data and anecdotal information to conduct their work and assess progress? How can they use it more effectively? Health and nutrition The evaluation will help the program gather specific answers to key issues such as:  Are nutritional services carried out according to standard?  Do targeted beneficiaries receive appropriate health and nutritional messages?  To what extent are infant feeding practices being practiced?  To what extent are Behavior, Communication and Change (BCC) activities being effective?  Are the program eligibility and graduation criteria appropriate, given the objectives and assumptions of the program? If not, how should it be modified? Methodology (See proposed methodology in the Midterm Evaluation Plan) Timeline The Mid-term evaluation will be carried out from March 28, to April 21, 2012 in the two provinces covered by the program. Deliverables (See proposed methodology in the Midterm Evaluation Plan) Minimum Qualification for the Public Health Nutritionist/Team Leader:  Masters in Public Health Nutrition;  Experience in the evaluation of Health and Nutrition programs;  Familiarity with PM2A and USAID Title II programs;  Previous experience as a Team Leader in an evaluation  Strong English writing skills for the final report (see detailed ToR attached).  Proficiency in both English and French (speaking, reading, and writing)  Ability to work in a team  Excellent communication skills  Willingness to travel to remote areas  Capable of working under pressure How to Apply: Consultants interested in this assignment should send the following information to CRS Burundi: • Brief cover letter highlighting relevant experience and skills, as well as confirming availability for 25-30 days (March-28 to April 22), 2012) time frame; • Curriculum Vitae; PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 16 • Written proposal (in English) for not less than two pages and not more than 5 pages describing the proposed methodology and actions for completing the mid-term evaluation; • A writing sample in English from a previous consultancy. • Three professional references with phone numbers and/or email addresses; • One page budget indicating daily fee and other related consultancy costs. The above materials should be sent by email to: BI_RMC@global.crs.org PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 17 Terms of Reference for the Commodity Supply Chain Consultant Midterm Evaluation CRS Burundi PM2A-TUBARAMURE 2009-2014 Introduction/Background information Catholic Relief Services/Burundi (CRS/BI), in partnership with International Medical Corps (IMC), Food for the Hungry (FH) and CARITAS Burundi are implementing a five-year USAID-financed Multi-Year Assistance Program (MYAP) entitled Tubaramure (“Let’s Help Them Grow” in Kirundi) in Ruyigi and Cankuzo provinces of Burundi. The overall objective of the program is to “prevent malnutrition in children under the age of two” by enrolling all pregnant women and mothers of children younger than 6 months in Ruyigi and Cankuzo provinces and providing then with a package of services until the child reaches 24 months. The benefits include: 1) access to quality health care services at area health care centers; 2) education on key health, hygiene and nutrition practices through a CARE Group approach; and 3) improved caloric and nutrient intake through culinary demonstrations and monthly delivery of Title II commodities. Approximately 51,075 mother-child pairs are targeted over the five years (July 2009 to October 2014) of the $45 million grant. The PM2A-TUBARAMURE program is pursuing the following strategic objective: Strategic Objective: Malnutrition in children under 2 years of age is prevented The anticipated results are: 1: Women and Children 0-59 months access quality nutrition and health services 2: Households practice appropriate health and nutrition behaviors 3: Eligible women and children have increased intake of diverse foods Objectives of the Evaluation Overall Objective The objective of the midterm evaluation of PM2A is to objectively assess progress towards achieving the strategic objectives above, particularly in the areas of health/nutrition and commodity management. This evaluation is also an opportunity to clarify the strategic direction for the remainder of PM2A. Specific Objectives:  Assess the achievements in the commodity and food distribution component,  Evaluate the relevance, effectiveness and efficiency of commodity management and food distribution.  Identify constraints (logistics and operations in general),  Document lessons learned and provide recommendations for future programming, . Composition of the evaluation team The Burundi PM2A-TUBARAMURE mid-term evaluation team will be composed of the following:  One international consultant with Public Health & Nutrition expertise to evaluate IR3 & 2 of the program; S/he will serve as the team leader  One international consultant with Primary Health Care and Community Health expertise to evaluate IR1 PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 18  One international consultant with expertise in logistics and supply chain management to evaluate IR3. Experience with Title II programs required. Evaluation Questions The Commodity Supply Chain Consultant will evaluate the following questions:  Are commodities being managed appropriately? To what extent?  Are food distributions program carried out according to standard?  Are correct procedures and best practices used in receiving, storing and distributing food commodities?  CRS’s transportation system to partner warehouses and mobile distribution sites: Does CRS’s current logistic systems of trucks and warehouses have sufficient capacity to transport all needed commodities on time, as the program reached its maximum number of beneficiaries (over 46,000)?  Is the food distribution system design efficient?  Are implemented procedures efficient for monitoring food distributions and in a timely manner?  CARITAS warehousing: • Are correct procedures and best practices used in receiving, storing and distributing food commodities? • Is there a sufficient number of staff working in the partner warehouses to ensure a separation of duties (for inventory & distribution)? • Are all documents associated with inventory stocks (waybills, spoiled food, etc.) in warehouses maintained correctly and up to date?  CARITAS Food distribution: • Is the food distribution system design efficient? • How long on average are women waiting to receive rations? • Is there adequate sun/rain protection for women as well as sanitation facilities? • Is the current distribution plan able to accommodate the increased numbers in a timely and efficient manner? • Is it a need for additional staff, extra distribution days? Does the current system protect against “false” beneficiaries from receiving commodities? If yes, what mechanisms can be put in place? • Are the recommended food rations size respected? • Do beneficiaries know the quantity of CSB and oil they are to receive? If not, how can the communication of this information be improved? • Is there adequate security during the food distribution? • What are the weaknesses of the distribution scheme and what are the recommendations to address them?  USAID branding/marking compliance: Is it adequate? How can it be improved?  Is CRS using good procedures for monitoring CARITAS food distributions and in a timely manner?  Is CRS providing adequate supervision and support to CARITAS for both their commodity warehousing and distribution? Is post-distribution monitoring including verification of distribution documentation taking place? In addition, the Commodity Management expert will:  Analyze logistics and commodity information flow to the programmatic unit/COP  Review training plans and practices, assessment of training needs Methodology(See proposed methodology in the Midterm Evaluation Plan) PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 19 Timeline The Mid-term evaluation will be carried out from March 28, 2012 to April, 28, 2012 in the provinces covered by the program. Deliverables(See proposed methodology in the Midterm Evaluation Plan) Minimum Qualification for Commodity Supply Chain Consultant  Master’s degree in finances, microfinance, agro economy, management or other relevant diploma.  Five to ten years experience dealing with commodity management and safety net interventions  Relevant knowledge of the safety net interventions systems and networks in Burundi;  Substantial experience in commodity program evaluations (quantitative and qualitative methods)  Bilingual English- French (the evaluation report will be written in English)  Ability to work in a team  Excellent communication skills  Willingness to travel to remote areas  Capable of working under time pressure How to Apply: Consultants interested in this assignment should send the following information to CRS Burundi: • Brief cover letter highlighting relevant experience and skills, as well as confirming availability for 25-30 days (March-28 to April 22), 2012) time frame; • Curriculum Vitae; • Written proposal (in English) for not less than two pages and not more than 5 pages describing the proposed methodology and actions for completing the mid-term evaluation; • A writing sample in English from a previous consultancy. • Three professional references with phone numbers and/or email addresses; • One page budget indicating daily fee and other related consultancy costs. The above materials should be sent by email to: BI_RMC@global.crs.org PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 20 Terms of Reference for the Primary Health Care Specialist Midterm Evaluation CRS Burundi PM2A-TUBARAMURE 2009-2014 Introduction/Background information Catholic Relief Services/Burundi (CRS/BI), in partnership with International Medical Corps (IMC), Food for the Hungry (FH) and CARITAS Burundi are implementing a five-year USAID-financed Multi-Year Assistance Program (MYAP) entitled Tubaramure (“Let’s Help Them Grow” in Kirundi) in Ruyigi and Cankuzo provinces of Burundi. The overall objective of the program is to “prevent malnutrition in children under the age of two” by enrolling all pregnant women and mothers of children younger than 6 months in Ruyigi and Cankuzo provinces and providing then with a package of services until the child reaches 24 months. The benefits include: 1) access to quality health care services at area health care centers; 2) education on key health, hygiene and nutrition practices through a CARE Group approach; and 3) improved caloric and nutrient intake through culinary demonstrations and monthly delivery of Title II commodities. Approximately 51,075 mother-child pairs are targeted over the five years (July 2009 to October 2014) of the $45 million grant. The PM2A-TUBARAMURE program is pursuing the following strategic objective: Strategic Objective: Malnutrition in children under 2 years of age is prevented The anticipated results are: 1: Women and Children 0-59 months access quality nutrition and health services 2: Households practice appropriate health and nutrition behaviors 3: Eligible women and children have increased intake of diverse foods Objectives of the Evaluation Overall Objective The objective of the midterm evaluation of PM2A is to objectively assess progress towards achieving the strategic objectives above, particularly in the areas of health/nutrition and commodity management. This evaluation is also an opportunity to clarify the strategic direction for the remainder of PM2A. Specific Objectives for the Primary Health Care Specialist:  Assess the achievements the health component of the program (IR 1);  Evaluate the relevance, effectiveness and efficiency health equipment provided by the program to the target health centers;  Identify constraints in the implementation of the health component activities;  Document lessons learned and provide recommendations for future programming,  To what extent has the quality of the nutrition and health services provided in the health care centers improved?  Has the training provided to health care staff been appropriate for their education/skill level?  Has the skill level of the health care staff improved from training?  Has the access and the use of health and nutrition services improved for beneficiaries communities?  Are beneficiaries satisfied with the quality of services provided (comparison????)? PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 21 Composition of the evaluation team The Burundi PM2A-TUBARAMURE mid-term evaluation team will be composed of the following: • One international consultant with Public Health & Nutrition expertise to evaluate IR3 & 2 of the program; S/he will serve as the team leader • One international consultant with Primary Health Care and Community Health expertise to evaluate IR1 • One international consultant with expertise in logistics and supply chain management to evaluate IR3. Experience with Title II programs required. Evaluation Questions for the Primary Health Care Specialist:  Are the health component activities implemented according the DIP (Detailed Implementing plan)? What are obstacles and the delays observed? What can be done to catch up the delays?  Does the health component fit with the Burundian government’s strategy and priorities? Are there steps that could be taken to improve the integration?  What interventions have been more or less successful in meeting targets?  Are all heath center targeted by the program equipped by the program as scheduled and according to the protocol?  Are the equipments appropriate to reach the expected results?  Are the equipments really used by health agents in the health center? Are the trained staffs able to properly use the equipment?  Are the equipments used in an efficient and sustainable way?  Are the equipments useful to the health centers? Do the persons in charge of the health centers consider the equipments as a relevant support?  Do the staffs using the equipments understand the objective of the overall program?  Do the health centers staff and the community health workers understand the contents of the training modules?  Do the health centers staff and community health workers use the training knowledge? What can be done to improve their use of the training knowledge?  Are the training modules appropriate? How can they be improved?  Do the targeted health center staffs who are not involve in the trainings understand program?  Given that IMC has no control on health center staff, what strategy can IMC put in place in order the reach the program results?  How is the collaboration between the program and the Ministry of health (at provincial and national levels)? What can be done to improve the collaboration? Methodology (See proposed methodology in the Midterm Evaluation Plan) Timeline The Mid-term evaluation will be carried out from March 12, 2012 to April, 12, 2012 in the provinces covered by the program. Deliverables (Methodology (See proposed methodology in the Midterm Evaluation Plan) Minimum Qualification required for the Primary Health Care Specialist:  Diploma in Medicine and Public Health, or in Public Health and Nursing;  Experience in the evaluation of Primary Health Care programs;  Minimum of ten years work experience in Primary Health Care, and or Community Health programs;  Familiarity with PM2A and or USAID Title II programs; PM2A-TUBARAMURE 2009-2014 Mid-Term Evaluation: Scope of Work Page 22  Strong English writing skills for the final report  Ability to work in a team  Good communication skills  Willingness to travel to remote areas  Capable of working under pressure; How to Apply: Consultants interested in this assignment should send the following information to CRS Burundi: • Brief cover letter highlighting relevant experience and skills, as well as confirming availability for 25-30 days (March-28 to April 22), 2012) time frame; • Curriculum Vitae; • Written proposal (in English) for not less than two pages and not more than 5 pages describing the proposed methodology and actions for completing the mid-term evaluation; • A writing sample in English from a previous consultancy. • Three professional references with phone numbers and/or email addresses; • One page budget indicating daily fee and other related consultancy costs. The above materials should be sent by email to: BI_RMC@global.crs.org 1 ANNEX D: Summary Responses to the Evaluation Questions The Terms of Reference for the Midterm Evaluation included several sets of questions: a set of general questions on the program and sets of questions for each IR. Below are the summary responses to some of those questions. Most of this information is also included in the report itself. A. Responses from PHC Expert, Dr. Vera Bensmann General Evaluation Questions Design, Implementation and Achievements: 1. How effective is the program at reaching the vulnerable population? Good: Pregnant women, mothers and children are coming to the health centers. 2. What could be done to improve targeting? Scale up work with CHWs. Combine immunization with other services: i.e. Second Postnatal Consultation at same time as child’s first vaccination (BCG); Growth Monitoring in first year during routine immunization days. 3. What interventions have been more or less successful in meeting targets? Most interventions have been successful in meeting targets. For some the indicators had been changed, but when we revert to the original indicators, targets were met. 4. Which interventions are most critical and/or effective in achieving project objectives and intermediate results? The training and the supportive supervision: The training is not only a means to improve the knowledge and skills of health personnel; it also works very well on their motivation. The supportive supervision has been good to gauge the knowledge retained after training, and has served to make changes in the training program. 5. What are the factors that hinder/assist the effective implementation of the program? Factors that hinder the implementation:  Staff shortages and high turnover of staff at health centers.  The high incidence of malaria among children under five, can contribute to malnutrition in this group. 2 Factors that assist the implementation:  The program is well embedded in the MoH’s policies and structures: MoH has ownership of all training (trainers are of MoH and so are the modules and protocols).  Good relations with the MoH at all levels 6. What improvements can be made to the design to improve results? Start community component earlier, and train MoH’s CHWs simultaneously with program’s nutrition health promoters (PSN). 7. What improvements can be made in the implementation of the program in order to improve results? Staff shortages and high turnover of staff at health centers: - This is out of the program’s hands, but the MoH is working on this: 1) The new nursing schools in Cankuzo and Ruyigi will have their first graduates next year; and 2) the MoH is decentralizing its hiring process for health center staff, which means nurses will no longer be hired centrally and sent to places they don’t want to go, but instead they will be hired at the district level, and will stay there without transfers to other provinces. High incidence of malaria: - Under IR-1 an extra training module will be scheduled in for the appropriate diagnosis and treatment of malaria in children and pregnant women. Furthermore, the lack of distribution of ITNs to pregnant women will be recorded in the prenatal consultations, to serve as a tool for advocacy. - Growth Monitoring at the health centers is done on different days than routine immunizations. It would be good to combine these activities for children in their first year, so every child gets weighed before he/she is vaccinated. - All women who deliver at the health center should receive a postnatal consultation upon exit. And all women who take their baby for their first vaccination (BCG immediately after birth) should receive a postnatal consultation. NB. All women should be encouraged to deliver at the health center. However, those who deliver at home should come to the health center for a postnatal check of mother and baby. 8. Is the program well integrated in the local government’s strategy and priorities? Yes. 9. Are there steps that could be taken to improve the integration as well as food security impacts through greater integration? IR-1 is well integrated into the ministry of health, though food security lies more within the domain of the ministries of agriculture and social affairs. Each component of the program has its 3 own links and ties to the respective government systems, and the four NGOs within the consortium meet regularly to improve the integration of the different components of the program. 10. How effective is the program at reaching women? Very effective: Women are accessing health centers in large numbers. 11. What could be done to improve women's participation? Health Centers already focused on women and children. The question would be how to improve men’s participation. Capacity strengthening: 1. Is the technical field staff well trained and supervised? Yes. Two nurses per health center have received additional training from the MoH through the program. All modules were taught by trainers within the MoH, and were part of the national training curriculum. IMC conduct its own monthly supervision, as well as a joint supervision, together with the MoH. 2. What technical areas, if any, need strengthening? As malaria is still the most prevalent illness and the number one cause of death among children under five in both provinces, an extra module on diagnosis and treatment of malaria will be added to the training schedule. 3. Is the program effectively developing the capacity of counterparts and/or partners? Yes. And IMC has been very good at detecting weaknesses in the program, and amending these. For example, they found the quality of supervision by the MoH supervisors needed to be improved, and organized an extra training for all supervisors and their superiors (i.e. district supervisors, médecins chefs des districts, and médecins chefs des provinces). 4. If not how could the design or implementation be altered to improve capacity strengthening? NA 5. Is the program effectively enabling, or developing the capacity of beneficiaries? Within IR-1 this is a weak point, as training of CHWs has only started recently. The CHW are volunteers within the MoH public health program. Most CHWs started working in 2003, and according to the national protocol there should be 3 CHWs per colline. In reality however, there are 2 CHWs per colline. 4 6. If not how could the design or implementation be altered to improve capacity strengthening? If possible, it would be recommended in future programs to start the training trajectory of CHWs in parallel with that of the clinical health care workers. Also, a considerable amount of nutritional health promoters (PSN) were trained by the program in IR-2; it would be good to include the MoH’s CHWs in these trainings in the future. Sustainability: 1. Are the outcomes related to adoption of better practices sustainable, i.e. are participants likely to continue after the project ends? One would think so. The ingredients are put in place to make the health centers more accessible for women and children: Free healthcare, well-trained staff, availability of equipment etc. However, ultimately the sustainability depends on how the MoH is able to sustain this. i.e. do the provinces have a sufficient budget for their health care? 2. Which outcomes are likely or unlikely to be sustainable, and why? At the moment the prenatal and postnatal consultations are a requirement for women to remain enrolled in the Tubaramure program. Attendance is good: 94% of pregnant women complete 3 or more prenatal visits, and 68% of the new mothers come for two postnatal consultations. In the coming year one can see whether these rates are sustained, as there will be no new enrollments into the Tubaramure program. The sustainability of the outpatient nutrition treatment centers depends on the availability of Plumpy’nut©. We have already seen the supplementary feeding services for moderately malnourished children end because food was no longer available, and the same could happen with the services for the severely malnourished if food is not secured. 3. What can be done to increase the sustainability? An exit strategy needs to be developed together with the MoH and government authorities in the two provinces. 4. Will it be possible in the remaining years of the program to hand off responsibility to a local entity? Yes. The activities for IR-1 are already done under the responsibility of the MoH. However, IMC provides a lot of support and facilitation in terms of fuel for supervision, the venue, lodging and per diems for training, etc. IMC proposes to gradually decrease its financial inputs, while the MoH assumes more of these costs towards the end of the program. The provincial health authorities need to consider these costs in their annual budgets. 5 5. If not, what additional actions need to be taken? NA 6. Are activities appropriate and are they efficient in helping achieve results? Yes. Malnutrition in children under five is caused partially by low food intake, and partially by disease. By improving the treatment of illnesses one can expect a decrease in malnutrition. 7. Are there any lessons learned? - Integrating Tubaramure into the MOH and using the MOH policies and protocols has been key to the success of IR-1. - Although PM2A emphasizes prevention of malnutrition, a malnutrition treatment program should also be included if not already in place. - A community component (with training of CHWs) should commence at the same time as the clinical component. 8. What recommendations on activities if taken into account will contribute to improving performance? - Scale up training and supervision of CHWs, and provide incentives for CHWs to perform home-visits and follow-up of children at risk. - Combine Growth Monitoring with routine immunization: i.e. weigh all children before they are vaccinated. - Combine the first vaccination of the child (BCG) with a postnatal consultation of the mother. Monitoring and Evaluation/Reporting: 1. Are M&E data collected adequately and reported regularly and in a timely fashion? Yes. Data on consultations are collected from all health and nutrition centers on a monthly base, and compiled centrally for IR-1 by their M&E specialist. In addition, a performance survey has been conducted every 3 months among a random sample of beneficiaries and health centers, for all Intermediate Results. An anthropometric study of a random sample of children will be conducted every 6 months. The Burundian institute of statistics -ISTEEBU- is currently implementing the first of these studies. 2. Are M&E data and anecdotal information used for management purposes? Yes. IMC conducts monthly supervision visits together with the MoH, which are followed by quarterly coordination meetings in which proceedings and results are discussed. 6 3. How can M&E data and anecdotal information be better used in program management decisions? At the moment it is not possible to see in the data from the health centers whether a pregnant woman or a child is enrolled in the Tubaramure program or not. i.e. At the health centers one cannot see which mothers and children are receiving food rations and are part of a care group. If at the health center one could distinguish between those in the program and those not, it would be easier to attribute results, and to provide feedback for the community component of IR-2. 4. Does the technical staff use M&E data and anecdotal information to conduct their work and assess progress? Yes within IR-1 5. Is the information being used to manage the program? Yes within IR-1 6. How can they use it more effectively? See point 3 Sector-Specific Questions 1. Are the health component activities implemented according the DIP (Detailed Implementing plan)? Yes. 2. What are obstacles and the delays observed? The commencement of the whole PM2A program was delayed because of the late baseline by IFPRI for the study groups. In general the training is well on schedule, though the program had to wait for approval of the MoH for some of the modules (on pre- and postnatal consultations). 3.What can be done to catch up the delays? The approval caused a delay of 2 months, which has already been caught up. 4. Does the health component fit with the Burundian government’s strategy and priorities? Yes. 5. Are there steps that could be taken to improve the integration? 7 None of the district health directors (médecins chefs des districts) has been trained in the integrated management of childhood illnesses (IMCI), and they do not always know how to supervise the implementation of IMCI. Though this is an expensive and time-consuming training (the original module of 11 days has already been compressed into 6 days, and cannot be reduced further - according to the IMC site managers), it would be worth considering training on IMCI for these district authorities to assure the sustainability of this component. 6. What interventions have been more or less successful in meeting targets? All interventions have been successful in meeting the original targets. During the course of the program the indicators for pre- and postnatal consultations had been changed, though it would be better to keep the original indicators: 1) The original indicator for prenatal consultations was: % of women realizing a minimum of 3 prenatal consultations, with a target of 98%. By the end of FY2011 the program had achieved 94%, which is good. However, in the intermediate progress reports the indicator had been changed to the % of women realizing a minimum of 4 prenatal consultations. This with the same target of 98%, though only 61.4% was achieved. A suggestion was made to lower the target, but it is better to keep the target high (i.e. at 98%) and retain the original indicator of 3 visits. Most of the women are registered for prenatal services in the second trimester of their pregnancy, which leaves sufficient time to complete 3 visits. If one should insist on 4 visits, efforts should be made to capture women in their first trimester. 2) The original indicator for postnatal consultations was: % of women realizing a minimum of 2 postnatal consultations, with a target of 50% for the end of FY2011 and 75% for the end of the program. By the end of FY2011 the program had achieved 68%, which exceeds the target for the year. However, in the intermediate progress reports the indicator had been changed to the % of women realizing a minimum of 3 postnatal consultations. This with the same target of 50%, though only 8.4% was achieved. A suggestion was made to lower the target, but it is better to keep the target of 75% and retain the original indicator of 2 visits, because women who have no complications during the first visits do not really need to come back for a third visit. Most women do not see the need for a postnatal consultation if they don’t have any complaints. However, almost all these women come to the clinic with their babies after birth to receive the first immunization (BCG), and it would be a good moment to capture these women then for their postnatal check-up. 7. Are all heath center targeted by the program equipped by the program as scheduled and according to the protocol? 8 Yes, even though new health centers have been added after the beginning of the program. 8. Are the equipments appropriate to reach the expected results? Yes. All equipment that was donated (both consumables and non-consumables) is listed in the Burundian standard requirements for the inventory of a health center. 9. Are the equipments really used by health agents in the health center? Yes. 10. Are the trained staffs able to properly use the equipment? Yes. Both nurses and clinical assistants (aides soignants) were trained to use the scales and measuring boards. Otherwise no sophisticated equipment was given. 11. Are the equipments used in an efficient and sustainable way? Yes, except for some otoscopes that were delivered without batteries, and some spoons for the supplementary feeding. 12. Are the equipments useful to the health centers? Yes, they form part of the essential inventory list. 13. Do the persons in charge of the health centers consider the equipments as a relevant support? Yes, they are basic necessities. 14. Do the staffs using the equipments understand the objective of the overall program? Yes, though not everyone understands the study component with the various controlled groups. 15. Do the health centers staff and the community health workers understand the contents of the training modules? Yes. All modules use a pre-test and post-test to gauge their knowledge before and after the training. And during the supportive supervision the same topics are often repeated. It would be interesting however, to repeat the post-test a couple of months after the training, to see whether they have retained the knowledge. Simple and concise messages were used for the training of the CHWs, and it was interesting to see that CHWs from different areas used the same hand motions to explain how one should cool a child with fever. 16. Do the health centers staff and community health workers use the training knowledge? 9 Yes, the nurses seem very competent at diagnosing and treating children with the most common illnesses such as malaria, acute respiratory infections and diarrhea. However, not all nurses understand how to use the algorithms and questionnaires that are provided with the IMCI module. The CHWs have thus far only received very basic training, but they have become very active in referring malnourished children to the centers. 17. What can be done to improve their use of the training knowledge? As long as the nurses diagnose and treat children correctly, one can wonder whether the IMCI algorithms and questionnaires are really necessary. For the moment the MoH supervisors have decided that the nurses only need to complete all the paperwork (of algorithms and questionnaires) for 5 children per day. This is to get used to the forms and understand the reasoning within the algorithms. For the CHWs it is important to find some kind of incentive to continue to go out into the community and visit more children in their homes. The CHWs have been doing outreach work since 2003 for numerous health programmes; they receive a Fanta and some money for transport during the training modules, but otherwise they work for free. The CHWs themselves suggested they would like to have their own visual aids (pictorials) to take into the communities and homes, instead of borrowing -the large and heavy- image box (boîte d’images) from the health center. Furthermore they suggested they should receive T-shirts and/or caps to make them recognizable; boots, to reach the muddy areas; bicycles to get from the colline to the health center; and they wanted more training... 18. Are the training modules appropriate? Yes, the program uses training modules that have been tested, validated and accredited by the MoH. 19. How can they be improved? At the moment the modules do not need to be improved, but more people need to be trained. Two people were trained in each health center, in accordance with the detailed implementation plan. However, 10 new health centers have been added to the two provinces, and many of the trained nurses have been transferred to other provinces. 20. Do the targeted health center staffs who are not involve in the trainings understand program? Yes, most of them do, as they are included in the supervision visits. 21. Given that IMC has no control on health center staff, what strategy can IMC put in place in order the reach the program results? 10 IMC has no control over health center staff, but conducts its monthly visits together with the district health directors and/or supervisors, who do have control over health center staff. Furthermore IMC organizes quarterly coordination meetings with health authorities and health center committees (COSAs). 22. How is the collaboration between the program and the Ministry of health (at provincial and national levels)? Very good! Some of the IMC staff and the MoH authorities were in medical school together, which facilitates communication. 23. What can be done to improve the collaboration? Collaboration already very good. _____________________________________________________________________________ B. Responses from Commodity Supply Chain Management Specialist, Ange Tingbo Design, Implementation and Achievements: 1. How effective is the program at reaching the vulnerable population? What could be done to improve targeting? 2. What interventions have been more or less successful in meeting targets? 3. Which interventions are most critical and/or effective in achieving project objectives and intermediate results? 4. What are the factors that hinder/assist the effective implementation of the program (i.e., the food distribution? Positive factors: a) Need of food assistance in impoverished provinces of Ruyigi and Cankuzo. b) Acceptance and collaboration of GoB and local authorities c) Good network of Caritas in communities i.e. communication, warehouse facilities, human resources, etc. d) Organization in Care Groups Negative factors (Hindrance) a) Imported food commodities (!?!?) b) Distance between distribution sites and some communities c) Slow involvement of males in the program 5. What improvements can be made to the design to improve results in food distribution? 11 a) Involvement of males in activities at the startup of the project b) Inclusion of an agriculture component for local foods availability c) More cooking demonstrations using local foods and development of recipes 6. What improvements can be made in the implementation of the program in order to improve food distribution? a) Involve men in the construction of shades and latrines at the distribution sites b) Reduce the waiting time of beneficiaries at sites i.e. increase the number of distribution days c) Review scooping vs. grouping as system of distribution d) Caritas supervisors and CRS’ EUCs should draft a plan after every warehouse inspection on how the findings will be addressed with: deadline, person/entity responsible. e) Define a clear policy for Empty Containers (to avoid possible social conflict) 7. Is the program well-integrated in the local government’s strategy and priorities? Are there steps that could be taken to improve the integration as well as food security impacts through greater integration? 8. How effective is the program at reaching women? What could be done to improve women's participation? 9. What is the quality and frequency of community workshops organized by PM2ATUBARAMURE animators and other actors to support gender? Monitoring and Evaluation/Reporting (specifically for the commodity activities) 1. Are M&E data collected adequately and reported regularly and in a timely fashion? YES 2. Are M&E data and anecdotal information used for management purposes? YES. The M&E database (with the computer application) has been instrumental in updating the list of beneficiaries according to their categories with accuracy. This is one of the strategies to avoid “false beneficiaries”. Yes, anecdotal information has been used for management purpose: the months of pregnancy and new prenatal protocols have been updated when it was rumored and reported that women sell/buy urine to fake pregnancies. 3. How can M&E data and anecdotal information be better used in program management decisions? See above 4. Does the technical staff use M&E data and anecdotal information to conduct their work and assess progress? Is the information being used to manage the program? YES. See above. There is a teamwork between field technicians who do things manually and they compare with what is generated by the computer. 5. How can they use it more effectively? The PSN’s have to be more effective on the ground during the food distribution days. There are many cases that can be solved on the spot and some need a follow-up i.e., mother with a malnourished child allegedly referred by the health center to PM2A while the project is no longer enrolling; mother who reports that her card along with the Tubaramure containers have been destroyed by a fire in her house; husband who has kept all the beneficiary’s attributes to get foods while he is a 12 newly divorcee; husbands who come to receive foods and sell them out before reaching home, etc. 1 ANNEX E: LIST OF KEY INFORMANTS Organization Name & Title CRS/Bujumbura  Debbie Schomberg, Country Director  Salvator, GSO and Security Officer CRS/US  Mary Herrigan, Senior Technical Advisor/Nutrition IMC/Bujumbura  Dr. Dayan, Country Director  Dr. Isu Mazambi, Program Manager  Dr. Isaac Kumushi, Program Coordinator  Gerard Mbonimpa, M&E and Data Specialist FH/Bujumbura  Philip Mato, Program Director  Dr. Dieudonné Bikorimana, BCC Advisor Caritas/Bujumbura  Abbé Térence Ntitangirageza, Secretary-General of Caritas  Rémy Kibinakanwa, Project Manager Tubaramure Personnel in Bujumbura  Dr. Bajay Raphael, Chief of Party  Dr. Jeanne d’Arc Ntiranyibagira, Deputy Chief of Party  Joseph Iboudo, M&E Officer  Melkamu Dereb, Commodity Manager  Eric Nimubona, National Commodity Manager  Régine Pacis Nihoreho, National Technical Advisor for Food and Nutrition  Thaddée Niyonzima, M&E Officer  Chrispin Mwizero, M&E Officer Tubaramure Team in Ruyigi  Guerrier Iryabavyeyi, Provincial Coordinator  Ezéchiel Kabwebwe, Provincial Nut. Advisor  Jean-Claude Ciza, Admin.& Finance Assistant  Nahoyo Nepo, End Use Checker  Gratien Nyenaha, End Use Checker  Marie-Claire Nzeyimana, Chef Magasinière, Ruyigi  Janvier Misago, Assistant Magasinier, Ruyigi  Oumar Itangishaka, Caritas Supervisor, Ruyigi  Emilienne Kaneza, THP  Gaspard Ndacikiriwe, THP  Pierre Ndayiziga, THP  Gaspard Ndacikiriwe, THP  Gilberte Mbazumutima, THP  Dr. Hervé Kaptchouang, IMC Site Manager  Samuel Niyonkuru, Nutrition Advisor  Eliane Kanyange, SFP/GM Advisor  Jacqueline Ndikumagenge, Care Group Supervisor  Jean Bosco Rusatira, Care Group Supervisor 2  Janvière Ndayishimiye, Care Group Supervisor  Ezéchiel Kabwebwe, Provincial Technical Advisor for Food and Nutrition Tubaramure Team in Cankuzo  Sévérien Sikobagira, Provincial Coordinator  Edmond Twagirayesu, Provincial Food Advisor  Anaclet Dominique Nduwayo, End Use Checker  Evelyne Nvhire, End Use Checker  Severin Gashwigiri, Caritas Supervisor Cankuzo  Dr. Jean Paul Bukombe Cubaka, IMC Site Manager  Donatien Twagirayezu, Health and Nutrition Supervisor  Ignace Simali Dunyonge, Nutrition Supervisor  Mambo Medard, IMC Finance, Admin and Logistics Assistant  Alexis Havyarimana, Growth Monitoring Advisor  Gerard Biregeya, Care Group Supervisor  Désiré Njiji, Care Group Supervisor  Edmond Twagirayezu, Provincial Technical Advisor for Food and Nutrition  Sylver Buduguru, THP  Rose Akimana, THP  Saidi Ntakibi, THP  Pie Mbonabuca, THP SDV (Commodities)  Alain Haringandji, Agent SDV/Gitega  Sylvester Nyembela, Driver Bhanji Transport, Dar-es-Salaam Caritas Field Warehouse Staff  Athanase Barandereka, Rusengo  Tharcien Gahungu, Ruyigi  Léopold Bigirimana, Buthezi  Sr. Marie, Nyabitare  Evariste Ndenzako, Gisuru  Erasme Ntahimpera, Gisura/Mobile  Charles Kavakure, food distributor  Ephrem Nzeyimana, Biyorma  Ephride Niyokwizera, Nyabitsinda  Jean Barizera, Kinyaya  Leonidas Kizoya, Munzenze  Therese Butoyi, Muremera  Gaudence Nimbona, Camazi  Elizabeth Ndayikumana, Muyaga  Claver Rugace, Cendajuru Ministry of Health and CHWs  Dr. Térence Hatorimana, Expert assigned to evaluation team  Dr. Dismas Bigirindavyi, Director of Cankuzo Hospital  Révocat Kamwenubusa, Head of Nutrition Center, Cankuzo 3 Hospital  Dr. Isidore Ntiharirzwa, Director of Ruyigi Hospital  Dr. Dismas Mukokereza, Provincial Director, Ruyigi  Cécile Dusabe, Provincial Health Information Systems, Ruyigi  Pontien Irambona, Provincial Focal Point for Nutrition, Ruyigi  Dr. Pierre Sinarinzi, District Director, Cankuzo  Dr. Christophe Gahungu, District Director, Ruyigi  Dr. Marc Nzambimana, District Director, Kinyinya  Dr. Stany Bigirimana, District Director, Murore  Sylvain Ciza, Murore District Supervisor  Japhet Ndayisenga, Public Health Supervisor, Cankuzo Provincial Health Department  Hermenegilde Ndengutse, Provincial Focal Point for Nutrition, Cankuzo  Delphine Niyimpa, Head Nurse, Muriza Health Center  Spéciose Pendo, Deputy (Nurse A3), Muriza Health Center  Jean Ntorogo, TPS, Muriza Health Center  Léocadie Ntawuhorahiriwe, CHW/ Muriza Health Center  Régine Ndirarika, CHW/ Muriza Health Center  Sophie Sabimbona, CHW/ Muriza Health Center  Guido Mbabawe, CHW/ Muriza Health Center  Léa Minani, CHW/ Muriza Health Center  Jeanine Nininahazwe, CHW/ Muriza Health Center  Ernest Niyokwizera, Nurse A3, Rusengo Health Center  Juvénal Bucumi, Head Nurse (A3), Mubavu Health Center  Gisèle Senkware, Deputy (Nurse A3), Mubavu Health Center  Isaac Amini Ndiragora, Deputy (Nurse A3), Kigamba Health Center  Isidore Hatungimana, Deputy (Nurse A3), Nyarurambi Health Center  Nicole Niyonsavye, TPS, Cankuzo Health Center  Dancile Ntirampeba, TPS  Albéric Rurahenye, Head Nurse (A2), Camazi Health Center  Didas Ndayikeza, Deputy (Nurse A3), Camazi Health Center  Gérard Baricako, TPS, Camazi Health Center  Jean Bosco Nkurunziza, Head Nurse (A2), Nyuro Health Center  James Hacimana, Deputy(Nurse A3), Nyuro Health Center  Patricie Ndagijimana, Head Nurse (A3), Dutwe Health Center  Fidès Ahiboneye, Nurse Administrator (A2), Dutwe Health Center  Périne Niyongabo, CHW, Nyuro Health Center  Jean Muyanda, CHW, Nyuro Health Center  Vincent Mukunganya, CHW/Gisagara  Abraham Marira, CHW/Gisagara 4  Pascasie Bangirinama, , CHW/Gisagara  Chrispin Gatunzi, , CHW/Gisagara  Stéphanie Muteragiranwa, CHW, Cankuzo Health Center  Sévérin Busando, CHW/Muhimbure  Denise Minani, CHW/Muhimbure  GoB Administration & Local Leaders  Jean Berchmans Niragira, Governor of Cankuzo Province  Principal Advisor to the Governor of Ruyigi Province  Chef de Colline, Rutimbora  Pauline Hatungimana, Chef de Colline in Gisuru  Stany Ndomvyi, Chef de Colline  Pierre Claver Bitungwa, Chef de Colline  Chrispine Gatunzi, Chef de Colline, Rubabara  Salvator Ciza, Chef de Colline, Gerero  Joseph Ndirazana, Chef de Colline, Kirambi  Léopard Nzeye, Chef de Colline, Kagoma  Damien Rucinca, Chef de Colline, Gitwenge USAID/FFP in Burundi  Stanley Stalla, Food for Peace Officer  Audace Mpoziriniga, Food Security Specialist  Jim Anderson, USAID/Burundi Director 1 ANNEX F: LIST OF PRINCIPAL DOCUMENTS CONSULTED Tubaramure Documents, including IFPRI and FANTA Resources 1. Proposal, including appendices 2. PM2A Snapshot (September 2011 summary document) 3. Baseline survey report 4. IFPRI Research Protocol 5. FANTA – PM2A – Burundi Baseline (May 2012) 6. ARR Narrative for FY2011 7. DIP Table FY2010 (from PREP) and DIP table 2011 (from ARR) 8. IPTT FY2011 9. PREP 2010 10. Rapport sur les Performances du Programme (2 Annual Reports - FY2010 and FY2011 and 3 trimestriel reports for FY 2011) 11. 7 “Trigger Indicator” reports 12. Quarterly reports from IMC 13. Quarterly reports from Food for the Hungry 14. Lancet article on the Haiti study that launched the PM2A concept 15. Enquête Anthropometrique en Provinces Cankuzo et Ruyigi: Evaluation du Programme PM2A – Tubaramure (ISTEEBU June 2012) 16. TRM-PM2A Revised Version (FANTA – November 2010) 17. CRS/Burundi: Food Commodity Management Systems (Ange Tingbo – April 2011) USAID/FFP Documents 1. Burundi Health Sector Assessment (November 2009) 2. MYAP 2009 Proposal Guidelines 3. FFP Trip Report (Bob Drapcho – Jan/Feb 2011) 4. FFP Trip Report (David Hay Smith) 5. BEST Analysis for Burundi (January 2012) Other Resources 1. Local Determinants of Malnutrition (Thomas Davis et al) 2. Plan National de Développement Sanitaire: 2011 – 2015 (Ministry of Health/Burundi) # Description (attach support documentation) Quantity # Description (attach support documentation) Quantity 1 Thermometer (medical) 27 23 Measuring tape 49 2 Stethoscope 16 24 Pot 10 liters 90 3 Otoscopes 27 25 Pot 5 liters 64 4 vaginal speculum kit 6 26 Plastic cups 270 5 Cuvette for vaginal speculum 12 27 Spoons de 5 CC 270 6 Blood pressure meter 23 28 Plates 540 7 Stethoscope (obstetrical Pinard) 10 29 Measuring spoons 90 8 Baby scale 20 30 Jerry-can 20 L 54 9 Balance Salter 25kg 35 31 Bucket de 10 L 27 10 Basin 6 32 Bucket 20 L 54 11 Delivery Kit 6 33 Basin plastic 10 L 54 12 Set of tambours small and medium 20 34 Measuring cans 2 liters 90 13 Instrument dish 20 35 Whisks 6 14 covered container (transport of placenta) 18 36 Stand for Balance Salter 27 15 Delivery bed 6 37 Brooms 27 16 Neonatal table 16 38 Files 42 17 plastic mats 54 39 Dustbins 1 18 Labor beds 44 40 GM charts 13500 19 Boots 81 41 registration charts 2500 20 Plastic glasses (eye) 27 42 Registers CPN 8 21 Masque (Packet of 100) 39 43 Register SPC 30 22 Sterile gloves (Packet of 100) 54 44 Posters 27 ANNEX J: LIST OF LESSONS IN BCC MODULES Module1: Title: Care Group Orientation; Leader Mother Flipchart; Module 1 of 5 Table of Contents Lesson 1: Introduction to Care Groups Lesson 2: Teaching Techniques Lesson 3: Leader Mother Responsibilities Lesson 4: Watching for the Day when Infants Do Not Die Lesson 5: All People are Created with Value Lesson 6: I Can Change Module2: Title: Essential Nutrition, Hygiene, and Care Actions during Pregnancy; Leader Mother Flipchart; Module 2 of 5 Table of Contents Lesson 1: Antenatal Care and Medical center Births Lesson 2: Maternal Nutrition Lesson 3: Micronutrients for a Healthy Birth and Delivery (Iron and Iodine) Lesson 4: Hand washing with Soap Lesson 5: Creation of Household Hand Washing Stations Lesson 6: Preventing Malaria in Pregnancy Lesson 7: Preparing for Birth Lesson 8: Immediate Breastfeeding (avoidance of Pre-lacteal Feeds) and use of colostrum Lesson 9: Newborn Care Practices Module 3: Title: Essential Nutrition, Hygiene, and Care during Infancy; Leader Mother Flipchart; Module 3 of 5 Table of Contents Lesson 1: Importance of Maternal Postpartum Care Lesson 2: Exclusive Breastfeeding: Benefits, Frequency and HIV Lesson 3: Exclusive Breastfeeding: Hydration, Emptying the Breast and Complementary Feeding Lesson 4: Danger Signs during Childhood Illness Lesson 5: Overcoming Breastfeeding Problems: Poor Position, Engorgement and Breast Lumps Lesson 6: Clinical Services and Growth Monitoring and Promotion Lesson 7: Men's Involvement in Breastfeeding and Child Care Lesson 8: Child Spacing Lesson 9: Improved Water Source, Water Purification and Storage Lesson 10: Proper Disposal of Feces Lesson 11: Malaria Transmission and Prevention (including ITNs) Lesson 12: Malaria: Danger Signs, Treatment and Home Care Module 4: Title: Essential Nutrition, Hygiene and Care during Early Childhood; Leader Mother Flipchart; Module 4 of 5 Table of Contents Lesson 1: Child Feeding: 6-8 months Lesson 2: Child Feeding: 9-11 months Lesson 3: Child Feeding: 12-23 months Lesson 4: Porridge Recipes: CSB and Local Foods Lesson 5: Vitamin A: Foods and Supplements Lesson 6: Worms and Deworming Lesson 7: Preparing, Cooking and Storing Foods Module 5: Title: Management of childhood Infections; Leader Mother Flipchart; Module 5 of 5 Table of Contents Lesson 1: Dehydration Signs and Dangers Lesson 2: Prevention of Dehydration with ORT Lesson 3: Proper Feeding of Sick Children Lesson 4: Dysentery and Persistent Diarrhea Lesson 5: Pneumonia Prevention Lesson 6: Kitchen Gardens: Pumpkin ANNEX K: LIST OF ELIGIBILITY CONDITIONS FOR BENEFICIARIES Critical Child Development Period Criteria for receiving ration card Pregnancy (at least 3 months pregnant)  Registered for pre-natal services (demonstrated by show of health card) to start receiving food aid  Agrees to complete 4 pre-natal visits  Household agrees to practice at least 6 selected activities on the Tubaramure Family Poster Lactation  Completed 4 pre-natal visits (show card)  Registered for postnatal services (show card)  Baby has been registered for growth monitoring  Household agrees to practice (or has completed) at least 6 selected activities on the Tubaramure Family Poster 6 to 9 months old  Post natal visits completed (show card)  Child’s immunization record is current (show card)  Child shows satisfactory progress in growth (MUAC) or mother demonstrates capacity to recuperate the child  Household agrees to introduce some animal products, vegetables and fruit as part of IYCF 10-12 months  Child’s immunization record is current (show card)  Child shows satisfactory progress in growth (MUAC) or mother demonstrates capacity to recuperate the child  Household agrees to continue IYCF practices  Household agrees to practice (or has completed) at least six selected activities on the Tubaramure Family Poster 13-18 months 18 to 24 months (23rd month is final month for receiving food) Award certificate of completion at 2 nd birthday  Household agrees to continue to practice good health behaviors and to increase the number of practices adopted  HH is willing to share experiences with others in the community. CRS BURUNDI MASTER SHIPPING LEDGER AS OF JUNE 30, 2012 PL# VESSEL FY ATA PORT COMMODITY QTY in UNITS NET WEIGHT (KG) 09BI1552 MAERSK JAMBI v.S005 09 22-Nov-09 DAR CSB 55,433 1,385,825.00 09BI1580 MSC DAMIA v. S032 09 10-Dec-09 DAR CSB 720 18,000.00 09BI1580 MSC DAMIA v. S032 09 10-Dec-09 DAR CSB 35,270 881,750.00 09BI1580 MSC DAMIA v. S032 09 10-Dec-09 DAR VEGO 7,718 170,143.31 09BI1581 MSC CARLA v S074 09 22-Nov-09 DAR CSB 87,698 2,192,450.00 09BI1581 MSC CARLA v S074 09 22-Nov-09 DAR VEGO 15,407 339,647.32 09BI1581 MSC CARLA v S074 09 22-Nov-09 DAR CSB 23,040 576,000.00 10BI1799 Frisia Rostockv.S005 10 25-Nov-10 DAR CSB 32,041 801,025.00 10BI1799 Frisia Rostockv.S005 10 25-Nov-10 DAR CSB 7,188 179,700.00 10BI1800 Safmarine Ngami v.S0 10 25-Nov-10 DAR CSB 2,800 70,000.00 10BI1828 Maersk carolina v.10 10 04-Oct-10 DAR VEGO 2,266 49,953.97 10BI1830 MSC Damla v.S039 10 13-Nov-10 DAR CSB 2,846 71,150.00 10BI1838 M/V Advantage v.64 10 26-Jan-11 DAR CSB 19,539 488,475.00 10BI1838 M/V Advantage v.64 10 26-Jan-11 DAR VEGO 2,161 47,639.25 10BI1845 Gemini v.S006 10 22-Jan-11 DAR CSB 10,444 261,100.00 10BI1846 Frisia Rostockv.S006 10 22-Jan-11 DAR CSB 10,020 250,500.00 10BI1847 MSC Carla v.S080 10 13-Nov-10 DAR CSB 15,120 378,000.00 10BI1848 Gemini v.S005 10 19-Nov-10 DAR CSB 40,320 1,008,000.00 11BI1856 Liberty Grace v.47 11 14-May-11 DAR CSB 99,523 2,488,075.00 11BI1856 Liberty Grace v.47 11 14-May-11 DAR VEGO 14,516 320,005.22 11BI1883 MSC Jenny v.S018 11 11-Oct-11 DAR CSB 20,105 502,625.00 11BI1889 Noble Star v.53 11 07-Nov-11 DAR CSB 66,013 1,650,325.00 11BI1921 Independence v.S005 11 11-Oct-11 DAR VEGO 13,598 299,767.91 11BI1927 Nanjing Dragon vS004 11 01-Nov-11 DAR CSB 59,091 1,477,275.00 11BI1944 Northern Jamboree v. 11 01-Nov-11 DAR CSB 720 18,000.00 12BI2054 Maersk Drummond v.12 12 13-May-12 DAR CSB 14,245 356,125.00 12BI2055 WILL RICKMERS 12 13-May-12 DAR CSB 20,016 500,400.00 12BI2055 WILL RICKMERS 12 20-Apr-12 DAR CSB 13,680 342,000.00 12BI2060 Maersk Missouri v.12 12 19-Apr-12 DAR VEGO 9,073 200,013.00 17,323,969.98 CSB: 15,897 MT + Vegoil: 1,427 MT = 17,324 MT