FINAL PREVENTION IS BETTER THAN CURE. Final Report of the Evaluation: Prevention or Cure? Comparing Preventive and Recuperative Approaches to Targeting Maternal and Child Health and Nutrition Programs in Rural Haiti International Food Policy Research Institute - Cornell University – World Vision Haiti Team Written by: Purnima Menon, Cornell University Marie T. Ruel, International Food Policy Research Institute (IFPRI) With contributions from (in alphabetical order) Mary Arimond, IFPRI Jean-Pierre Habicht, Cornell University Bekele Hankebo, WV-Haiti Cornelia Loechl, International Potato Center John Maluccio, Middlebury College Mduduzi Mbuya, Cornell University Lesly Michaud, WV-Haiti Gretel Pelto, Cornell University Submitted to: The Food and Nutrition Technical Assistance (FANTA) Project July 31, 2007 FINAL PREVENTION IS BETTER THAN CURE. Final Report of the Evaluation: Prevention or Cure? Comparing Preventive and Recuperative Approaches to Targeting Maternal and Child Health and Nutrition Programs in Rural Haiti International Food Policy Research Institute - Cornell University - WV-Haiti Team Written by: Purnima Menon, Cornell University Marie T. Ruel, International Food Policy Research Institute (IFPRI) With contributions from (in alphabetical order) Mary Arimond, IFPRI Jean-Pierre Habicht, Cornell University Bekele Hankebo, World Vision-Haiti (WV-Haiti) Cornelia Loechl, International Potato Center (CIP) John Maluccio, Middlebury College Mduduzi Mbuya, Cornell University Lesly Michaud, WV-Haiti Gretel Pelto, Cornell University Submitted to: The Food and Nutrition Technical Assistance (FANTA) Project July 31, 2007 This report was made possible through the generous support of the American people through the support of the Office of Health, Infectious Disease and Nutrition, Bureau for Global Health, and the Office of Food for Peace, Bureau for Democracy, Conflict and Humanitarian Assistance, United States Agency for International Development (USAID), under terms of Cooperative Agreement No. HRN-A-00-98-00046-00, through the Food and Nutrition Technical Assistance (FANTA) project operated by the Academy for Educational Development (AED). The contents are the responsibility of AED and do not necessarily reflect the views of the USAID or the United States Government. Financial support for this research is also provided by the Government of Germany, WV-Haiti, and the World Food Programme. RECOMMENDED CITATION: Menon, P., and M. T. Ruel, with (in alphabetical order) M. Arimond, J.-P. Habicht, B. Hankebo, C. Loechl, J. Maluccio, M. N. Mbuya, and G. Pelto. Prevention Is Better than Cure. Final Report of the Evaluation: Prevention or Cure? Comparing Preventive and Recuperative Approaches to Targeting Maternal and Child Health and Nutrition Programs in Rural Haiti. Submitted to the Food and Nutrition Technical Assistance Project, Academy for Educational Development, Washington, D.C. July 2007. i ACKNOWLEDGMENTS This three-year evaluation has been the culmination of the efforts and commitment of numerous people and organizations. We begin by expressing our gratitude to key individuals who played critical roles in this evaluation, and then acknowledge all the other individuals and groups who have contributed to the success of this study. First, we acknowledge the critical support from Gilles Bergeron at the Food and Nutrition Technical Assistance (FANTA) Project, Academy for Educational Development (AED), who conceived of this study and saw it through from the early stages to the very end. In addition to his roles at FANTA for coordinating this work, Gilles has made substantial intellectual contributions to the evaluation throughout the study. In addition, he was instrumental in connecting the IFPRI-Cornell evaluation group with numerous other individuals and organizations in Haiti, all of which have strengthened this study. Second, we are grateful to the senior management team at WV-Haiti, particularly Wesley Charles, Bekele Hankebo, and Lesly Michaud. All of them were involved in this study from its inception. Their willingness to engage with the research team at all stages of this evaluation, to fund the fieldwork for the study, as well as to invest in the development of the preventive approach are all indicative of a commitment to improving nutrition among Haitian children. In addition, their commitment provides an example of an agency that aims to improve staff performance and program effectiveness by developing and supporting a motivational work context for staff at all levels. Third, we thank Eunyong Chung of USAID/Washington for her continued support and engagement with this evaluation and for the critical insights and inputs she provided at various junctures. We also are grateful to Carell Laurent, who was based at the Food for Peace Office at USAID/Port-Au-Prince at the inception of this study. Carell galvanized support for this study at USAID/ Port-Au-Prince and among other stakeholders in Haiti and was a driving force in seeing this study materialize in Haiti. Florence Cadet – also from USAID/Port-au-Prince – also contributed significantly and helped ensure continued financial and logistical support from USAID/Port-au-Prince. Finally, we are grateful for the many contributions of all those who were involved with this study, whether in the form of intellectual, financial, in-kind, logistical, and personal support. Here, we acknowledge these many forms of contribution by naming those people and organizations who we engaged with through the course of this evaluation. Any omissions are sincerely regretted. ™ FANTA: Gilles Bergeron, Bruce Cogill, Anne Swindale, Megan Deitchler ™ WV-Haiti: Lesly Michaud, Bekele Hankebo, Wesley Charles, Frank Williams, John Dorman, Jean-Marie Boisrond, Rony Destine, Rulisa Kimenyi, Makayi Chinyam, Kassa Selfago, Nixon Desir, Emmanuel Tinord, Rafael Cambronero, Cedrelle Jean Louis, Harry Francois, Lionel Isaac, Maude St. Victor ii ™ WV-Haiti Regional office at Hinche: Widy Laroque, Jean Louis Pierre, Dorlean Delince, Wagnes Desir, Ruth Calvaire, Marie-Carmel Joseph, Charlot Willelmyne, Delide Jean Baptiste, Elisabeth Elysée, Jean Robert Francoeur, Wisler Nicolas, Nuclair Guillaume, as well as all nurses, health promoters, assistant health promoters, and food monitors at the regional office ™ USAID/Port-au-Prince: Florence Cadet, Michael Kerst, Carell Laurent ™ Ministry of Health (Haiti): Dr. Francesca Joceline Marhone Pierre, Head of the National Coordination Office of the Program on Nutrition ™ Haiti-based research staff: Elisabeth Metellus, Arsene Ferrus, Remy Lafalaise, Pierre Lenz Dominique, Remy Lafalaise, Mathieu Honoré, Louise Dumay, Josianne Loredan, Slande Celestre, Yves Francois-Pierre, as well the survey fieldworkers and team chauffeurs ™ Collaborators in Haiti: Edouine Francois (Freedom from Hunger), DidacArts, Paul Thomas, Paul Brea, and Michelle Cayemittes (Insitut Haitien de l’Enfance,), Colette Vilgrain (IntellConsult), Agathes Pellerin (CRS), Sonia Stines (CRS), Jude-Marie Banatte (CRS), Marjolein Moreaux (Save the Children, US), Mireille Sylvain (CARE), Bette Gebrian (Haitian Health Foundation) ™ International Food Policy Research Institute (IFPRI): Wahid Quabili, Ali Subandoro, Jay Willis, Suneetha Kadiyala, Ginette Mignot, Lynette Aspillera, Celine Castillo-Macy ™ Cornell University: Kate Dickin, Edward Frongillo, Francoise Vermeylen, Amanda Zongrone In closing, we acknowledge the communities, families, and women who were interviewed over the course of this study, from the formative research for the development of the intervention to the large surveys that provided the impact data. Our research teams were always welcomed by these people of the Central Plateau of Haiti, and for this we express our sincere thanks. iii ACRONYMS ADP Area Development Program AED Academy for Educational Development AHPs Assistant Health Promoters ARI Acute Respiratory Infection BCC Behavior Change Communication BCG Bacille Calmette Guerin (vaccine) BF Breastfeeding BMI Body Mass Index CBR Cost-benefit ratio CF Complementary Feeding CER Cost-effectiveness ratio CIF Cost, Insurance, and Freight (prices) CRSP Collaborative Research Support Program CS Cooperating Sponsors CSB Corn-Soy Blend CTS Commodity Tracking System CU Cornell University DAP Development Activity Program DD Dietary Diversity DTP Diphtheria, Tetanus, and Pertussis (vaccine) EBF Exclusive Breastfeeding EMMUS Enquête Mortalité Morbidité et Utilisation des Services FANTA Food and Nutrition Technical Assistance FDPs Food Distribution Points FES Focused Ethnographic Study FFH Freedom from Hunger FOB Free on Board (prices) iv FY Fiscal Year GDP Gross Domestic Product GMP Growth Monitoring and Promotion HAZ Height-for-Age Z-score HH Household HPs Health Promoters IHE Institut Haïtien de l’Enfance IFPRI International Food Policy Research Institute LAC Latin American and the Caribbean LAM Lactational Amenorrhea Method LSMS Living Standards Measurement Survey MCs Mothers’ Clubs MCHN Maternal and Child Health and Nutrition OR Operations Research ORS Oral Rehydration Salts PVOs Private Voluntary Organizations RPs Rally Posts SD Standard Deviation SFB Soy-Fortified Bulgur SSS Salt-Sugar Solution TIPS Trials of Improved Practices USAID United States Agency for International Development USD U.S. dollar WAZ Weight-for-Age Z-score WHZ Weight-for-Height Z-score WSB Wheat-Soy Blend WV WV-Haiti v TABLE OF CONTENTS ACKNOWLEDGMENTS ............................................................................................................... i ACRONYMS.................................................................................................................................iii EXECUTIVE SUMMARY ........................................................................................................ xvii 1. INTRODUCTION AND SCIENTIFIC RATIONALE FOR THE STUDY ............................. 1 1.1 Haiti: The Poorest Country in the Western Hemisphere ................................................ 2 1.2 Title II-MCHN Programs................................................................................................. 2 1.3 Rationale for a Preventive Approach to Reducing Undernutrition.................................. 3 1.3.1 The recuperative versus the preventive approach............................................ 3 1.3.2 Biological rationale for targeting food supplementation to children under 24 months ............................................................................................ 4 1.3.3 Behavioral rationale for combining food supplementation and behavior change communication strategies.................................................................. 7 1.3.4 Conclusions.................................................................................................... 10 1.4 Organization of the Report............................................................................................ 11 2. OBJECTIVES AND DESIGN OF THE EVALUATION....................................................... 13 2.1 Introduction................................................................................................................... 13 2.2 Objectives of the Evaluation......................................................................................... 13 2.2.1 Additional objectives ..................................................................................... 13 2.3 Design of the Overall Evaluation.................................................................................. 14 2.3.1 Program theory and pathways of influence ................................................... 14 2.3.2 Designing evaluation activities based on program theory ............................. 17 2.4 Project Activities........................................................................................................... 19 2.4.1 Evaluation activities........................................................................................ 21 2.5 Conclusions................................................................................................................... 26 3. WV-HAITI’S MCHN PROGRAM ......................................................................................... 27 3.1 Introduction................................................................................................................... 27 3.2 Services Provided.......................................................................................................... 27 3.3 Program Service Delivery Points.................................................................................. 29 3.3.1 Rally Post services ......................................................................................... 29 3.3.2 Mothers’ Club services .................................................................................. 30 3.3.3 Food Distribution Point (FDP) services......................................................... 31 3.3.4 Pre-and postnatal consultations...................................................................... 32 3.3.5 Home visits .................................................................................................... 32 3.4 Program Organization/Staffing..................................................................................... 32 3.5 Conclusions................................................................................................................... 34 vi 4. GETTING IT RIGHT FROM THE START: DESIGNING THE PREVENTIVE AND RECUPERATIVE APPROACHES.................................................................................. 35 4.1 Introduction................................................................................................................... 35 4.2 The Food Assistance Component ................................................................................. 35 4.3 The BCC Component.................................................................................................... 36 4.3.1 Research phase............................................................................................... 36 4.3.2 Development phase........................................................................................ 39 4.4 The Implementation Plan for the Preventive and Recuperative Program Approaches .................................................................................................... 44 4.4.1 At Rally Posts (RPs) ...................................................................................... 44 4.4.2 At the Mothers’ Clubs (MCs) ........................................................................ 46 4.4.3 Food Distribution Points (FDP)..................................................................... 49 4.5 Conclusions................................................................................................................... 49 5. DELIVERING IT WELL: THE OPERATIONS RESEARCH PROCESS ........................... 51 5.1 Introduction................................................................................................................... 51 5.1.1 Integration of operations research into implementation and evaluation processes...................................................................................................... 51 5.2 Objectives ..................................................................................................................... 53 5.3 Methods......................................................................................................................... 54 5.3.1 Program context and stakeholders ................................................................. 54 5.3.2 Data collection methods................................................................................. 54 5.3.3 Consultative workshop................................................................................... 56 5.4 Results........................................................................................................................... 57 5.4.1 First round of operations research (OR1) ...................................................... 57 5.4.2 Outcomes of consultative workshop.............................................................. 61 5.4.3 Second round of operations research (OR2).................................................. 63 5.5 Conclusions................................................................................................................... 73 6. PROGRAM PARTICIPATION AND HOUSEHOLD AND CAREGIVER CHARACTERISTICS .............................................................................. 75 6.1 Introduction................................................................................................................... 75 6.2 Data and Analysis ......................................................................................................... 75 6.2.1 Data and variables.......................................................................................... 75 6.2.2 Analysis.......................................................................................................... 76 6.3 Results........................................................................................................................... 76 6.3.1 Program participation..................................................................................... 76 6.3.2 Household and caregiver characteristics........................................................ 82 6.4 Summary of Results...................................................................................................... 86 vii 7. NUTRITIONAL IMPACT OF THE PREVENTIVE APPROACH COMPARED TO THE RECUPERATIVE APPROACH ............................................................................. 87 7.1 Introduction................................................................................................................... 87 7.2 Objective of the Impact Evaluation .............................................................................. 87 7.3 Intervention and Comparison Groups........................................................................... 87 7.4 Methods......................................................................................................................... 88 7.4.1 Evaluation design........................................................................................... 88 7.4.2 Data analysis .................................................................................................. 88 7.5 Results........................................................................................................................... 89 7.5.1 Baseline characteristics.................................................................................. 89 7.5.2 Intervention impact: Results from probability design analysis..................... 90 7.5.3 Results of plausibility analysis....................................................................... 91 7.5.4 Differences in morbidity between the two program groups .......................... 96 7.6 Summary of Results...................................................................................................... 98 7.6.1 Probability results .......................................................................................... 98 7.6.2 Plausibility results.......................................................................................... 99 7.7 Conclusions................................................................................................................... 99 8. IMPACT OF THE PROGRAM ON KNOWLEDGE AND PRACTICES ........................... 101 8.1 Introduction................................................................................................................. 101 8.2 Data and Variable Creation......................................................................................... 101 8.2.1 Maternal knowledge..................................................................................... 101 8.2.2 Trial and adoption of key recommended practices...................................... 104 8.2.3 Child-feeding and care practices.................................................................. 105 8.2.4 Other care practices: Preventive care, hygiene practices, and care during illness.............................................................................................. 108 8.3 Analysis....................................................................................................................... 109 8.4 Results......................................................................................................................... 109 8.4.1 Impact of the program on maternal knowledge ........................................... 109 8.4.2 Trial and adoption of key recommended practices...................................... 114 8.4.3 Impact of intervention on infant and young child-feeding and care practices ............................................................................................. 121 8.4.4 Impact of interventions on preventive care, care during illness, and hygiene................................................................................................ 133 8.4.5 Comparisons between participants and nonparticipants on child￾feeding and care practices.......................................................................... 136 8.5 Conclusions................................................................................................................. 141 9. IMPACT OF THE PROGRAM ON FOOD SECURITY ..................................................... 143 9.1 Introduction................................................................................................................. 143 9.2 Data and Analysis ....................................................................................................... 143 9.2.1 Data and variables........................................................................................ 143 9.2.2 Analysis........................................................................................................ 144 viii 9.3 Results......................................................................................................................... 145 9.3.1 Impact of the Program on Household Food Insecurity................................ 145 9.4 Conclusions................................................................................................................. 151 10. INDIRECT BENEFITS OF THE PROGRAM.................................................................... 153 10.1 Introduction............................................................................................................... 153 10.2 Data and Analysis ..................................................................................................... 153 10.2.1 Data and variables...................................................................................... 153 10.2.2 Analysis...................................................................................................... 155 10.3 Results....................................................................................................................... 156 10.3.1 Impact of the program on household and respondent asset ownership ..... 156 10.3.2 Indirect impact of the program on women’s well-being............................ 158 10.4 Conclusions............................................................................................................... 162 11. COST-EFFECTIVENESS OF THE PREVENTIVE APPROACH RELATIVE TO THE RECUPERATIVE APPROACH .................................................................................... 163 11.1 Introduction............................................................................................................... 163 11.2 Methodology............................................................................................................. 164 11.2.1 The incremental cost-effectiveness between the two program approaches................................................................................... 164 11.2.2 Measuring the costs.................................................................................... 164 11.3 Data: Costs in the Program Areas............................................................................ 169 11.3.1 Direct and outside program costs in the study areas.................................. 169 11.3.2 Opportunity costs in the study areas .......................................................... 174 11.4 Cost-Effectiveness of Preventive versus Recuperative............................................. 175 11.4.1 Incremental cost-effectiveness estimates................................................... 175 11.4.2 Interpreting the incremental cost-effectiveness estimates ......................... 177 11.5 Conclusions............................................................................................................... 178 12. KEY FINDINGS, AND PROGRAM AND RESEARCH IMPLICATIONS ..................... 181 12.1 Relative Impact and Cost-Effectiveness ................................................................... 181 12.1.1 Is the preventive approach more effective than the recuperative approach at reducing childhood undernutrition? ....................................... 181 12.1.2 What is the relative cost-effectiveness of the preventive and recuperative approaches?........................................................................... 182 12.2 Limitations of the Study............................................................................................ 184 12.3 Pathways of Impact................................................................................................... 185 12.3.1 Factors pertaining to program design ........................................................ 185 12.3.2 Factors pertaining to program delivery...................................................... 187 12.3.3 Factors pertaining to program access and utilization................................. 189 12.3.4 Conclusions on pathways of impact .......................................................... 195 12.4 Implications for Programs, Policies, and Future Research....................................... 196 12.4.1 Implications for programs and policies...................................................... 196 12.4.2 Research implications ................................................................................ 198 ix 12.5 Lessons for Program Evaluations ............................................................................. 199 12.6 Conclusions............................................................................................................... 200 REFERENCES ........................................................................................................................... 201 Annex 2.1 List of household survey modules............................................................................ 209 Annex 3.1 Organizational structure of the Health component of the program.......................... 211 Annex 3.2 Organizational structure of the Commodities component of the program............... 213 Annex 4.1 Infant and child feeding practices in Haiti compared to best practices, and constraints and opportunities for behavior change in Central Plateau.............................................. 215 Annex 4.2 Identification of programmatic options to address the constraints to infant feeding, and to support facilitating factors ................................................................................... 219 Annex 4.3 BCC strategy matrix for a BCC program to prevent malnutrition among children between 0-24 months ...................................................................................................... 223 Annex 4.4 Schedules of learning sessions and topics at Mothers’ Clubs.................................. 227 Annex 5.1 Summary matrix from Consultative Workshop, February 2003.............................. 231 Annex 5.2 Mothers’ Clubs observation checklist items ............................................................ 237 Annex 5.3 Questions for staff related to motivation.................................................................. 239 Annex 5.4 Questions for staff related to supervision................................................................. 241 Annex 8.1 Scoring of questions in Infant and Young Child Feeding Knowledge Test............. 243 Annex 8.2 Scoring of questions in General Nutrition and Health knowledge Test (responses that were considered appropriate)................................................................. 245 Annex 8.3a Guiding principles for complementary feeding of the breastfed child................... 249 Annex 8.3b Guiding Principles for feeding non-breastfed children 6-24 months of age.......... 251 Annex 8.4 Awareness, trial, and adoption of key recommended practices, by program group and participation (ever)......................................................................................... 255 Annex 8.5 Percent of infants fed according to recommendations during early postnatal period, by intervention area and by participation ........................................................... 257 Annex 8.6 Age of introduction of selected liquids and foods, by intervention area and by participation ............................................................................................................... 259 x Annex.8.7 Percent of children who were fed at least the minimum recommended number of times, and mean number of meals and snacks, by age group, current program participation, and breastfeeding status ............................................................. 261 Annex 8.8 Percent of children who consumed selected food groups in the previous 24 hours, by intervention area and participation............................................................. 263 Annex 8.9 Percent of children who consumed selected animal source foods in the previous 24 hours, by intervention area and participation.............................................. 265 Annex 8.10 Frequency of consumption of nutrient-rich foods, including recipes promoted in mothers’ clubs, by intervention area and participation (mother ever participated) .................................................................................................................... 267 Annex 8.11 Frequency of consumption of nutrient-rich foods, including recipes promoted in mothers’ clubs, by intervention area and current participation (children 6-42 months) ................................................................................................................... 269 Annex 8.12 Mean dietary diversity, by age group, intervention area, and participation (children 6-42 months).................................................................................................... 271 Annex 8.13 Receipt of vitamin and mineral supplements, by index children (12-41 months) and their mothers .............................................................................................. 273 Annex 8.14 Feeding during and after diarrhea: Index children................................................ 275 Annex 8.15 Immunization status, by intervention group and by participation (among children 12-41 months of age) ........................................................................................ 277 Annex 9.1 Scoring of variables for the food insecurity scale.................................................... 279 Annex 10.1 Scoring of women’s well-being scales................................................................... 281 Annex 12.1 Summary of evaluation results............................................................................... 283 LIST OF TABLES 2.1 Summary of study activities................................................................................................... 20 3.1 Services provided by the WV MCHN program in Central Plateau....................................... 28 3.2 Composition of direct and indirect food rations, per beneficiary category ........................... 31 4.1 Package of interventions provided in the recuperative and preventive approaches .............. 45 5.1 Data collection methods used in the OR process................................................................... 55 xi 5.2 Types of information communicated to mothers during growth monitoring at Rally Posts, by year, by program area, and by research method ............................................................... 66 5.3 Types of advice given to mothers at Rally Posts in 2004, by child’s progress...................... 67 5.4 Advice given at Rally Posts in 2004, by child’s progress and program area......................... 67 5.5 Quality of information and facilitation at Mothers’ Clubs, by year and by program area .... 68 5.6 Summary of program changes in WV-Haiti MCHN program, and differences between preventive and recuperative program areas in 2004 ......................................................... 69 6.1 Program participation at the household level (includes participation by all children in the household, as well as pregnant/lactating women)....................................................... 77 6.2 Enrollment of index children in the food assistance program, by program group ................ 78 6.3 Timing of exposure for index children, by program group ................................................... 78 6.4 Receipt of food assistance among index children, by child age and program participation........................................................................................................ 79 6.5 Participation in Mothers’ Clubs and use of Rally Post services ............................................ 80 6.6 RP participation, by child age and program group ................................................................ 81 6.7 Program uptake and targeting, by program group ................................................................. 81 6.8 Household characteristics, by program and by program participation (ever participated).... 83 6.9 Caregiver/respondent characteristics, by program group and by program participation (ever participated) ............................................................................................................. 85 7.1 Key differences between recuperative and preventive approach intervention packages....... 87 7.2 Comparison of child, maternal, and household characteristics between program groups at baseline.......................................................................................................................... 90 7.3 Mean anthropometric outcomes at final survey, by program group...................................... 91 8.1 Guiding principles and recall data collected on recommended practices............................ 106 8.2 Maternal nutrition and health knowledge, by program group ............................................. 111 8.3 Maternal nutrition and health knowledge, by program group and participation (ever participated) ........................................................................................................... 112 8.4 Program impact on maternal nutrition and health knowledge (regression analysis)........... 113 xii 8.5 Differences in trial of recommended practices between preventive and recuperative groups and reasons for non-trial of practices, by program exposure.............................. 115 8.6 Child-feeding and care practices, by program group and time of survey............................ 122 8.7 Preventive care, care during illness, and hygiene, by program group and time of survey, for index children 12 to 42 months of age ...................................................................... 134 8.8 Summary of findings on behavior change outcomes........................................................... 137 9.1 Household food insecurity, by program and time of survey................................................ 145 9.2 Selected household food insecurity experiences, by program and time of survey .............. 146 9.3 All household food insecurity measures at final survey, by participation (ever and current) ..................................................................................................................... 146 9.4 Program impact on household food insecurity experiences at final survey (regression analysis)........................................................................................................ 148 9.5 Program impact on months of inadequate household food provisioning (MIHFP) at final survey (regression analysis).................................................................................... 149 9.6 Program impact on severity of months of inadequate household food provisioning (MIHFP) at final survey.................................................................................................. 149 10.1 Household and women’s asset ownership, by program group, participation, and time of the survey.................................................................................................................... 157 10.2 Respondent mental well-being, by program group, participation, and time of survey...... 159 10.3 Program impact on self-rated health (random effects regression)..................................... 160 10.4 Program impact on mental stress scale (presence/absence of symptoms) (random effects regression) ........................................................................................................... 161 11.1 Variables and data sources for incremental cost-effectiveness analysis............................ 165 11.2 Number of program beneficiary-months in Central Plateau and study areas .................... 170 11.3 Number of program child beneficiaries per month in the Central Plateau and study areas (FY 2005)............................................................................................................... 172 11.4 Direct program and outside program costs in the study areas ($000)................................ 173 11.5 Direct program and outside program costs per beneficiary-month in the study areas ($)................................................................................................................. 174 11.6 Incremental cost-effectiveness........................................................................................... 176 xiii LIST OF FIGURES 1.1 Mean weight-for-height (WHZ), weight-for-age (WAZ), and height-for-age (HAZ) of rural children in Haiti (EMMUS-III 2001) ......................................................................... 5 2.1 Program impact pathways...................................................................................................... 16 2.2 An “engaged” model of evaluation and technical assistance................................................. 18 3.1 Flow of activities at the Rally Post ........................................................................................ 30 4.1 Steps of the design process of the BCC component .............................................................. 37 5.1 An “engaged” model of evaluation and program development............................................. 52 7.1 Prevalence of undernutrition among children 12-41 months at baseline (groups combined) and end line, by program group...................................................................... 92 7.2 Mean anthropometric outcomes by child age and program group at final survey................. 93 7.3 Prevalence of stunting, underweight and wasting by child age and program group at final survey........................................................................................................................ 93 7.4 Weight-for-age Z-scores of index children (12-41 mo) and their younger siblings (0-11), by age and program group at final survey ............................................................ 95 7.5 Percentage of children who had fever in the past two weeks, by age and program group (final survey)........................................................................................................... 96 7.6 Percentage of children with cold/cough in the past two weeks, by age and program group (final survey)........................................................................................................... 97 7.7 Percentage of children with diarrhea in the past two weeks, by age and program group (final survey)..................................................................................................................... 97 7.8 Percentage of children with fast breathing in the past two weeks, by age and program group (final survey)........................................................................................................... 98 8.1a From awareness to adoption: Leaving expressed breast milk behind when going out..... 116 8.1b From awareness to adoption: Adding breast milk to gruel............................................... 116 8.1c From awareness to adoption: Adding an egg to child’s gruel........................................... 117 8.1d From awareness to adoption: Preparing gruel with beans and nuts.................................. 117 8.1e From awareness to adoption: Feeding enriched gruel in the evening............................... 118 xiv 8.1f From awareness to adoption: Preparing mashed plantain with pumpkin.......................... 118 8.1g From awareness to adoption: Feeding an extra meal after illness .................................... 119 8.2 Percent of mothers who reported optimal early feeding practices during first day of life, by program group and baseline and final survey—Index child............................... 124 8.3 Percent of mothers who reported optimal early feeding practices during first day of life, by program group and baseline and final survey—Younger siblings ..................... 125 8.4 Comparing baseline and final sample exclusive breastfeeding in past 24 hours for infants < 6 months: Kaplan-Meier survival functions (preventive/recuperative groups combined)............................................................................................................ 125 8.5 Maternal recall of timing of introduction of liquids and foods, at baseline and final survey (preventive/recuperative groups combined ......................................................... 127 8.6 Percent of breastfed children having received complementary foods minimum recommended times yesterday, by age, program group, and time of survey.................. 129 8.7 Percent of non-breastfed index children fed complementary foods a minimum recommended times, by age, program group, and time of survey.................................. 130 8.8 Percent of index children who consumed selected food groups in previous 24 hours, by program group, and baseline and final survey........................................................... 131 9.1 Household food insecurity experiences score, by program group and current participation at final survey ............................................................................................ 147 9.2 Prevalence of severe food insecurity (based on terciles of the household food insecurity experiences score), by program group and current participation at final survey...................................................................................................................... 147 9.3 Months of inadequate household food provisioning (MIHFP), by program group and current participating at final survey ................................................................................ 148 9.4 Proportion of households with insufficient food in each month of the calendar year, by program group (derived from the months of inadequate household food provisioning (MIHFP) measure)..................................................................................... 150 12.1 Program impact pathways.................................................................................................. 186 xv LIST OF BOXES 3.1 Five service delivery venues for the WV-Haiti food-assisted MCHN program.................... 27 4.1 Example from decision matrix to organize formative research results on exclusive breastfeeding................................................................................................ 39 4.2 Example from program-planning matrix to address constraints to exclusive breastfeeding..................................................................................................................... 40 4.3 Example of BCC strategy planning matrix on expression of breast milk.............................. 40 4.4 Example of message adaptation for expression of breast milk.............................................. 42 4.5 Example of learning session on expression of breast milk.................................................... 42 4.6 Mothers’ Club learning sessions............................................................................................ 48 5.1 Example of prioritization matrix used at the consultative workshop .................................... 57 8.1 Knowledge test questions on topics covered at MCs and RPs ............................................ 103 8.2 From awareness to trial and adoption: Key practices included in the survey..................... 105 xvi xvii EXECUTIVE SUMMARY Background Food-assisted maternal and child health and nutrition (MCHN) programs have traditionally targeted children less than 5 years of age who were identified as undernourished through growth monitoring activities. Scientific evidence, however, shows that children under two are more at risk of becoming undernourished, and more responsive to nutrition interventions than older children. Research findings consistently show that the earlier and the longer food supplementation is provided before the child reaches two years of age, the greater the benefits not only on growth in early life, but also on long-term physical, cognitive, and reproductive performance. Thus, investing in the first two years of life provides benefits way beyond childhood and is an essential element of development strategies and human capital formation interventions. Although the benefits of intervening early in life are unequivocal, nutrition-focused programs globally have been hesitant to adopt the universal targeting of children under two because of the lack of evidence of the effectiveness of the approach in large-scale programs. In order to fill this knowledge gap, researchers from the International Food Policy Research Institute and Cornell University undertook a study commissioned by the Food and Nutrition Technical Assistance (FANTA) Project at the Academy for Educational Development (AED) to compare the effectiveness and cost-effectiveness of a “preventive” approach, which targets all children under two, with the traditional “recuperative” approach, which targets children under five years of age once they have become undernourished. The project was carried out in collaboration with World Vision-Haiti (WV-Haiti) in the context of their PL 480 Title II food￾assisted MCHN program in the Central Plateau. Study Objectives The main objective of the study was to compare, under programmatic conditions, the impact on childhood undernutrition of targeting food assistance and behavior change communication (BCC) in the context of a Title II MCHN program to all children 6-24 months of age (preventive approach), versus to undernourished children less than 5 years of age (recuperative approach). We also compared the cost and cost-effectiveness of the two program approaches, as well as their impact on a variety of intermediate outcomes, such as food security, maternal nutrition knowledge, and infant feeding practices. Intervention Packages The preventive and recuperative packages of nutrition services were designed based on the best available biological evidence about the efficacy of supplementation as well as formative research to develop the BCC strategy that would accompany a monthly food assistance ration. The composition and size of the monthly food ration was identical for both program approaches, but the programs differed in the following three aspects: (1) the eligibility criterion (undernutrition among under-five children for recuperative; and age 6-24 months for preventive); (2) the focus, timing, sequencing, and number of sessions of BCC (see below); and (3) the timing and duration of eligibility to receive the food and BCC intervention (nine months from time child xviii is identified as being undernourished for recuperative – as decided by WV-Haiti for this approach; the whole 18-months period when children are between 6 and 24 months for preventive). Severely malnourished children (weight-for-age Z-scores < -3) 24 months or older in the preventive approach were also eligible to participate in the program for nine months. Both program approaches targeted pregnant and lactating women, who were eligible to receive a monthly food ration until the sixth month of lactation. Receipt of the food ration was conditional on regular monthly attendance at other program health services, including the Rally Posts (which provide preventive health-care services), pre- and postnatal consultations (for pregnant and lactating women only), and Mothers’ Clubs (MCs), which are small peer group education sessions. The MCs were the mainstay of the BCC strategy used by both program approaches. Although the content of the MC sessions was largely similar in both program approaches, the approaches differed in the sequencing of the sessions and the number of sessions offered. The sequencing of MC sessions was age-based in the preventive approach and mothers attended a maximum of 18 monthly sessions, while in the recuperative approach, the session topics were chosen to be relevant for mothers of malnourished children, and 9 monthly sessions were offered (as per original design of WV-Haiti’s recuperative approach). A total of 12 MC sessions were offered for pregnant and lactating women, 6 in pregnancy and 6 during the first six months of lactation. Health staff who facilitated the MCs were trained in the technical content of the MC sessions as well as in adult education-based communication and facilitation skills. Evaluation Design The evaluation used a community-level, cluster-randomized pre-post design, whereby 10 paired clusters of communities were randomly assigned to either the preventive or the recuperative program group. Cross-sectional surveys were conducted to assess the prevalence and severity of undernutrition among children 12-41 months of age at baseline and three years later, and statistical methods for analyzing group randomized designs were applied to evaluate impact. The baseline survey was conducted between May and September 2002 and the post￾evaluation survey was conducted exactly three years later, between May and September 2005, to minimize seasonal variations. All components of the intervention packages, except the newly developed BCC strategy, were implemented immediately following the baseline survey, i.e., in August-September 2002. The full BCC package, however, was implemented several months later (in May 2003) due to delays in material development, staff training, and field implementation. In addition to data on the anthropometric outcomes, information was also gathered on a variety of intermediate outcomes relating to the use of the program services, household food security and assets, caregiver nutrition knowledge, physical and mental well-being, and infant feeding and care practices. The surveys were designed with the UNICEF conceptual framework in mind, so as to be comprehensive in their assessment of the resources available for care at the household and caregiver level. xix The evaluation activities included a two-stage operations research process in addition to the surveys. The first stage, conducted a few months after the full intervention package was implemented in 2003, assessed the quality of program implementation and discussed the results with WV-Haiti to improve the quality of implementation. The second stage re-assessed the quality of implementation in 2004, and compared the quality of implementation between the two program approaches. No major implementation differences were seen, and implementation was of high quality, which suggested that both program approaches were operating largely as designed. Results At baseline, the two groups of communities (preventive and recuperative) were similar in all the main outcomes of the study as well as other major determinants of infant feeding and nutritional outcomes, such as food security, household socioeconomic status, maternal education, work patterns, social status, and physical and mental well-being. Impact on child nutritional status The key finding of the study is that in communities randomly allocated to receive a preventive approach of Title II-MCHN program, the prevalence of stunting, underweight, and wasting was 4, 6, and 4 percentage points lower after 3 years of operation compared to communities exposed to the recuperative program approach. The adjusted prevalence of stunting, underweight, and wasting among children 12-41 months in preventive areas was 33.9%, 14.8%, and 3.7%, respectively, whereas in recuperative communities, it was 38.2%, 20.8%, and 7.4%, respectively (using WHO reference standards (WHO 2006)). Mean height-for-age, weight-for-age, and weight-for-height Z-scores were also significantly higher in the preventive compared to the recuperative program communities. The differences (adjusted for child age and gender and clustering effect) in favor of the preventive group were + 0.14 for HAZ, + 0.24 for WAZ, and + 0.24 for WHZ. At baseline, there were no differences between program groups in any of the anthropometric indicators. The magnitude of differences in favor of the preventive group for mean anthropometric indicators is comparable to other effectiveness trials aimed at reducing undernutrition through improved complementary feeding (Caulfield, Huffman, and Piwoz 1999), and to the average impact of USAID Title II MCHN programs documented by Swindale and collaborators (2004). Although the studies included in these reviews, which used before/after or post-intervention designs with a control group are not directly comparable to our study design (which compared two food-assisted MCHN program approaches), they are indicative of a range of effect that may be expected from this type of intervention. If we assume that our recuperative approach had some impact on reducing undernutrition (as suggested by the review of USAID food-assisted MCHN programs), then the larger impact of the preventive approach must be viewed as additional to that of the recuperative approach. Plausibility of impact: The results of the main impact analysis are also supported by the fact that children who were exposed to the preventive program for the whole period of greatest xx nutritional vulnerability (i.e., from 6-23 months of age) benefited more from the intervention than children who exposed only partially during this time period. When compared to baseline, children’s nutritional status appears to have deteriorated among the recuperative group, especially with regards to the prevalence of underweight and wasting, which increased from 18.0% to 20.8% (underweight) and from 3.4% to 7.4% (wasting). Stunting increased by 0.2 percentage point in the recuperative group, while it decreased by 3.5 percentage points in preventive areas. Underweight also declined by 2.8 percentage points in the preventive areas, while it increased by the same amount in the recuperative group. Wasting increased by 4 percentage points in the recuperative group, and decreased marginally in the preventive group. These results suggest that the preventive approach may have helped mitigate the deleterious effects on childhood malnutrition of the economic and political crisis that occurred in Haiti during the study period. Relative costs and cost-effectiveness The total direct costs for the two program approaches are the same, but the total variable/outside costs of the preventive approach, as designed and implemented in the evaluation area, are much higher than the costs of the recuperative approach. When examining costs per beneficiary month, rather than total costs, we find that the direct program costs per beneficiary￾month are higher in the recuperative than in the preventive approach ($21 USD versus $15 USD). The outside program costs (which include the costs of the food commodities and health￾care supplies) are the same in both groups – $12 per beneficiary month. The differences in costs (both total and per beneficiary month) is due to the much larger number of beneficiary-months in the preventive compared to the recuperative approach, owing to the design features of the preventive approach, as well as the differential participation rates for the two program approaches. Specifically, (1) There is a larger number of eligible children in preventive compared to recuperative areas because the prevalence of underweight children among the under-5s is less than the proportion of children 6-24 months among children less than 59 months); (2) The duration of eligibility is longer, by design, in the preventive compared to the recuperative approach: children in the recuperative approach were enrolled in the program for 9 months – as determined in WV-Haiti’s development assistance program plan, while children in the preventive approach would be eligible to remain in the program for the entire period between 6 and 24 months, i.e., for up to 18 months. Thus the duration of intervention, by design, was double in the preventive approach; (3) The rates of program uptake are higher in the preventive compared to the recuperative areas: our analysis of program uptake (see below) suggests that the preventive approach seemed to elicit higher participation rates among eligible children than the recuperative approach, thereby further increasing the number of beneficiary-months in the preventive approach. In sum, the larger cost of the preventive compared to the recuperative approach is due to the larger number of beneficiary-months, which, in turn, is due to a combination of factors including design, undernutrition prevalence, and program uptake patterns. The relative cost and relative cost-effectiveness of the preventive approach over the recuperative approach can, therefore, differ from context to context. xxi Pathways of impact The impact of the program is dependent on good service delivery, as well as good participation in the program services by eligible beneficiaries, and finally, by appropriate use of the program inputs (food and BCC) by program beneficiaries that then lead to the intended intermediate impacts on food security, maternal knowledge, and childcare and feeding practices. As noted above, there were no implementation differences between the program approaches, and the program was implemented in all evaluation communities. Thus, differences between program approaches on child nutrition outcomes may have been achieved through differences in program participation and take-up. Other potential explanations for differences between program approaches include differences in impacts on the care environment at the household and caregiver levels, and/or differences in biological vulnerability (under two years vs. older children). In addition, the substantially higher proportion of children exposed to the fortified foods in the preventive group also likely contributed to the differences in outcome. We briefly review the different steps in the pathways of impact below. Program participation and uptake The enrollment rates in the food assistance and BCC package among pregnant and lactating women were similar in both program communities. Enrollment among children was different and was true to the design (as intended). Overall, more children in the preventive communities had ever been enrolled in the program (73%) than in the recuperative communities (28%). Around 75% of eligible children in the preventive program (6-23 months of age) were receiving food assistance at the time of the final survey. In the recuperative group, only 29% of children 6-59 months who were underweight at the time of the final survey were enrolled, most likely because a smaller proportion of eligible children were brought to the Rally Posts - the main entry point into the program. Targeting was excellent in preventive communities, with 93% of enrolled children meeting the age eligibility criteria; in the recuperative group, around 57% of currently enrolled children were underweight. Thus, program uptake in the two approaches was quite different, most likely because of differences in population understanding of the targeting mechanisms in the two approaches. Impact on household care context and resources At final survey, households in preventive communities had statistically significantly lower food insecurity than households in recuperative areas, but differences were small. More meaningful differences were observed between current participants and nonparticipants in both program groups, suggesting a positive short-term impact of food assistance on household food security. Compared to baseline, overall food insecurity did not improve over the three-year duration of the study, and continues to be severe in the program area. There was thus no evidence of a long-term effect of the program on food security; the same was true for asset ownership, which had not improved since baseline, and was not different between program communities or between participants and nonparticipants. Thus, overall, the program seems to xxii have had a positive impact on food security in the short term, suggesting that the slightly greater impact on households in preventive compared to recuperative communities is due to their longer eligibility to receive food assistance. There was, however, no evidence that the program had a longer-term impact on food insecurity (i.e., beyond the period of receipt of food aid), which is most likely due to the severe economic constraints faced by poor household in Haiti at the time of the study. It is possible that more sustained impacts of such a program could be obtained in less constrained times. Impact on maternal context and resources We did not find any differences between program groups on maternal resources such as education, social support, and autonomy in decisionmaking either at baseline or at the final survey. Impacts on other resources are discussed below. Maternal knowledge: Maternal knowledge of several topics related to infant and young child feeding and general health and nutrition topics was higher among the preventive compared to the recuperative group at final survey, but differences were generally small. Larger differences were found between mothers who had ever participated in the program, compared to those who had never participated. Maternal knowledge also significantly improved from baseline to final survey in both program groups, suggesting that overall, the BCC strategy was successful in improving overall maternal knowledge of health and nutrition in the program communities. The longer duration of exposure to the BCC in the preventive compared to the recuperative group and the more age-specific and timely delivery of messages, however, resulted in only small differences in knowledge between groups. Physical and mental well-being: At baseline, we found that food insecurity was strongly associated with all measures of women’s well-being, particularly mental well-being. At final survey, respondents in the preventive communities were better off than those in the recuperative communities on four of the women’s well-being measures – self-rated health, mental stress, and two measures of life satisfaction. Current participants in preventive communities also had better self-rated health, lower mental stress, and lower time stress than nonparticipants. Thus, it appears that the program’s short-term impact on food security had positive benefits on women’s mental well-being; and given that women in preventive communities received food assistance for longer periods of time (for up to 30 months if they were enrolled in the program during pregnancy and the first 6 months of lactation), they benefited more in terms of improved mental well-being than women in recuperative communities. Again, there was no evidence of long-term benefits of the program on these aspects, as reflected by the lack of difference in well-being between mothers who had ever participated and those who had not. Impact on child feeding and care practices Awareness, trial, and adoption of recommended practices The final survey included an assessment of the awareness, trial, and adoption of seven key practices recommended by the program. For most key practices, respondents in preventive program areas were more likely to report awareness, trial, and adoption than were respondents in recuperative areas. In most cases, however, differences between program areas were of xxiii relatively small magnitude, while differences between those ever exposed to the program and those never exposed were large. Thus participation in either program approach had a beneficial effect on awareness, trial, and adoption of recommended child feeding and care practices, with mothers in preventive communities doing slightly better on these aspects than women from recuperative communities, probably as a result of their longer exposure to the BCC intervention and the more timely delivery of the messages (i.e., at the age when knowledge and adoption of specific practices was most relevant). However, the reported reasons for non-trial and adoption of practices suggests that overall, practices that required few material resources to try and adopt were more likely to have improved with exposure to the program. Infant and young child feeding practices Breast feeding: There were no differences between program groups in early feeding practices or in the timing of introduction of liquids and complementary foods. This was expected since the two program approaches offered exactly the same services until the child reached 6 months of age. Large improvements in these practices were seen since baseline, however, and equally large differences were observed between participants and nonparticipants for both program approaches. Breastfeeding duration was the same in both program groups at baseline and at final survey and ranged from 18 to 24 months of age. Complementary feeding practices, including consumption of the donated commodities: There were few differences between program groups in complementary feeding practices at final survey, with the exception of diet quality and the consumption of animal source foods, which were slightly higher in the preventive group. The infant and young child feeding indicator – which combines information on breastfeeding, number of meals/day and dietary diversity – also showed a statistically significant, but modest difference in favor of the preventive compared to the recuperative communities. As with maternal knowledge, differences in many of the feeding practices were larger between ever participants and never-participants than between program groups. These include meal frequency, use of baby bottles, vitamin A supplement consumption, and consumption of fortified donated commodities (WSB) (the latter being expected). Several practices had markedly improved since baseline; these included a reduction in the use of baby bottles (which was halved since baseline), increases in vitamin A supplementation, and in appropriate feeding during and after diarrhea. The overall consumption of animal source foods, however, was lower at final survey than at baseline, probably a reflection of the economic crisis and related price increases in Haiti over the study period. The proportion of children consuming recipes made with WSB was not different between current participants in the program groups. However, since there is a larger number of program beneficiaries receiving WSB and other fortified foods in the preventive group, particularly among the younger children, it can be assumed that the fortified commodities contribute more to nutrient intake among children in the preventive approach than in the recuperative approach. Preventive and curative care-seeking and hygiene Immunization rates were not different between groups at baseline or final surveys, but they had improved from a very low 11% of children fully immunized at baseline to approximately 30% at final survey. Patterns of attendance at Rally Posts were similar between xxiv groups for children during the first year, which explains the lack of difference in immunization rates between groups. The low immunization coverage seemed to be largely due to poor supply as opposed to low demand, as suggested by our operations research results. Care-seeking for fever, cough, fast breathing, and diarrhea were not different between program groups at baseline or at the final survey and were not different between participants and nonparticipants. However, care-seeking rates were lower at the final survey when compared to the baseline survey, possibly due to decreased severity of illness and/or better home management of illness. Use of oral rehydration (ORS) and sugar-salt solutions (SSS) were higher since baseline, suggesting improved home care for diarrhea as a result of the program. There were no differences in markers of hygiene practices between program groups either at baseline or final surveys, but there was a slight decrease in hygiene scores since baseline. There were also no meaningful differences in markers of hygiene practices between participants and nonparticipants. Conclusions on the pathways of impact Our findings suggest that the pathways of impact, which led to better child nutritional outcomes among preventive communities, operated mainly through the changes in the childcare context resulting from participation in the program and through greater availability of the fortified commodities to children in the preventive approach. More specifically, participants in preventive communities had greater food security while in the program, which positively affected the caregivers’ mental well-being; this, combined with better knowledge, awareness, trial, and adoption of several recommended feeding and care practices, likely resulted in a generally more supportive care environment, which, in turn, could have had a greater impact of the preventive compared to the recuperative approach on child nutrition outcomes. Furthermore, a larger number of children in the preventive approach received fortified commodities in preventive approach than in the recuperative approach. Since consumption of the fortified commodities is high among those who receive them, this is also a potential pathway of impact. It is important to note that in general, differences between the preventive and recuperative approaches in the different aspects of the childcare context were relatively small. This was somewhat surprising, given the much longer duration of exposure to program inputs among participants in the preventive approach, the higher participation rates, and the explicit effort to deliver the BCC intervention in the most timely fashion in the preventive compared to the recuperative approach. The results thus suggest that it may be the cumulative effect of relatively small differences in the multiple aspects that comprise the care environment, which are responsible for the larger benefits of the preventive compared to the recuperative approach in improving child nutritional status in our study. Our results also showed that the two program approaches were operating equally well and that none of the program implementation and staff-related factors differed between the two approaches. This allows us to conclude with certainty that the greater nutritional impact observed in preventive communities was truly due to a more effective program approach. Implications for Programs and Policies This section briefly summarizes the overall lessons learned from this evaluation and their implications for programs and policies. xxv A preventive approach to addressing childhood undernutrition is more effective than a curative approach. The direct implications of our results are that in order to improve effectiveness, food-assisted MCHN programs should target all children under the age of two years, as opposed to malnourished children under-five, and continue to target pregnant and lactating women. Severely malnourished children up to 5 years of age should continue to be screened and receive appropriate care. Focusing on the under-twos is both feasible and successful in a programmatic context. There is renewed global attention around this critical age group, but few examples exist of feasible, successful, and effective programs. This evaluation provides an example of the feasibility and effectiveness of this approach, in a programmatic context, as well as an example of how such programs can be developed, strengthened, and monitored under real programmatic conditions. Our work thus provides programmatic evidence for current policy to focus nutrition interventions on the under-twos. A well-designed and well-implemented behavior change strategy can improve infant feeding practices regardless of whether a preventive or recuperative approach is used for targeting. This evaluation provides an example of an approach and a specific set of tools that were used for developing and implementing a locally relevant, programmatically feasible BCC strategy for improving child feeding and care practices among children under two years of age. Variable cost per beneficiary-month is approximately the same for the two programs but total costs are higher for preventive approach as designed in this study. The implications of these findings are that programs should carefully review their program design, geographic priority areas, and targeting mechanisms based on their resources and target number of beneficiary-months. Programs can also attempt to balance or reduce the costs of the preventive approach by changing the age range for targeting, the duration for which children are enrolled in the program, and/or even the amount of the food assistance provided. The impact of making any of these changes to the preventive approach, however, should be rigorously tested to ensure that these modifications do not result in losses in effectiveness and nutritional impact of the program. Investing in formative and operations research is important for program success. This evaluation provides strong evidence that investing in formative research can help design effective BCC programs that are grounded in the sociocultural context, locally relevant, and programmatically appropriate. The study also provides evidence that operations research provides critical insights regarding the quality of implementation and service delivery for evaluators and program implementers. Recommendations for Future Research Evaluation of the preventive approach in other settings, preferably using a control group. Future evaluations of the preventive approach should include a control group, so that the magnitude of the absolute effect of the preventive approach could be assessed. Evaluations in other contexts should also pay adequate attention to program theory so as to generate knowledge and consensus on developing and implementing programmatic approaches for delivering food￾assisted MCHN services to the critical under-two age group. xxvi Separating contributions of food assistance and BCC components. Since the food component contributes the majority of the cost of the program approach, it would be useful to conduct evaluations than allow separating the contributions and cost of food assistance from the BCC component. At the same time, the role of limited food and economic resources in dampening the potential impact of BCC programs in impoverished contexts should be kept in mind. Research should also assess the long-term impact of both program approaches and determine whether short-term benefits in nutritional status are maintained over time. Testing alternative designs for a preventive approach. Future evaluations should test different delivery systems for the preventive approach and compare the cost of alternative designs such as modifying the age of eligibility, reducing the duration of eligibility or the size of the food assistance package. These evaluations should include a control group so that the cost￾effectiveness of the different approaches could be assessed. Lessons for Program Evaluations Using program theory to develop evaluations. The approach to this evaluation was based on program theory that considered the full pathway of expected impacts. Developing the evaluation activities based on this framework was useful not only for the measurement of the different program inputs and outcomes but also to help identify bottlenecks that could influence the program’s impact. We paid careful attention to the design of the preventive approach in particular because it was important to ensure that the approach was truly preventive in nature. This attention to design is important to consider in all evaluations, because evaluating a package that is not designed to be true to the concept behind it is inefficient and does not allow appropriate interpretation of the results of an evaluation. Documenting implementation quality and program utilization. Attention to the quality of implementation of the intervention package as well as utilization of the intervention by intended users is critical in any impact evaluation. As with the design, a poorly implemented and poorly utilized intervention is not likely to yield the expected impact. Thus, attention to design and to implementation quality, as well as patterns of utilization, are both essential to draw conclusions about the effectiveness of interventions. Age of intervention versus age of expected impact. Finally, we suggest that it is extremely important that evaluations focusing on child growth outcomes pay attention to the age at which an intervention to improve growth is delivered versus the age at which impact is expected. Outcome assessments should be conducted within the age group that is most likely to show impact, which is not necessarily the same as the age of targeting of interventions. Conclusions This evaluation shows that a preventive approach to a Title-II MCHN program is more effective than the traditional, recuperative approach at reducing childhood undernutrition among children aged 12-41 months. The variable costs of both the program approaches was approximately the same on a beneficiary-per-month basis but the preventive approach was more xxvii expensive in the Haiti setting because of differences in design and participation rates between the two approaches. Moreover, the relatively low levels of undernutrition in this population (compared to other equally poor countries) results in a larger number of children under two years of age, compared to underweight children, and thus more beneficiary-months in the preventive approach. Both programs also had a significant impact on improving maternal knowledge and feeding practices compared to baseline, showing that a well-designed and well-implemented BCC strategy integrated within a Title-II MCHN program can be highly effective. These benefits were obtained with two carefully designed and implemented program approaches operating under particularly difficult field conditions in rural Haiti. We believe the findings of this study are generalizable, given the remarkably similar patterns of child growth globally, and the similar prevalence of undernutrition in Haiti and other countries in the world. We conclude by suggesting that there are four conditions for the results of this study to be replicable: (1) good program design based on sound formative research; (2) effective implementation and service delivery monitored with regular operations research; (3) good incentive structure and high staff motivation monitored and fostered by effective supervision; and (4) similar or higher levels of undernutrition than in Haiti (e.g., 25% - 30%). Although a similar preventive program could be effective in a population with lower levels of undernutrition than in Haiti, the cost per case of undernutrition prevented would be higher. However, most poor countries in Africa and Asia, and even some countries of Latin America, currently have a higher prevalence of undernutrition than Haiti. For these countries, a preventive approach is strongly justified for MCHN programs. xxviii 1 1. INTRODUCTION AND SCIENTIFIC RATIONALE FOR THE STUDY This report presents the results of a three-year project evaluating a food-assisted (Title II) Maternal and Child Health and Nutrition (MCHN) program implemented by WV-Haiti in the Central Plateau of Haiti. The evaluation compared two different approaches of targeting food assistance and a behavior change and communication strategy to improve child-feeding and care practices: (1) the recuperative1 approach, which targets children with mild or moderate undernutrition (weight-for-age < -2 Z-scores); and (2) the preventive approach, which targets all children 6-24 months of age, in an effort to prevent, rather than cure, childhood undernutrition. The project was motivated by a critical programmatic question for Title II program implementers around the world, i.e., whether blanket targeting of Title-II MCHN programs to children less than two years of age is more effective than targeting undernourished children. Scientific evidence from controlled intervention trials suggests that children under two years of age are more responsive to nutrition interventions than older children (Schroeder et al. 1995). Research also shows that the earlier and the longer food supplementation is provided before the child reaches 2 years of age, the greater the benefits on growth and on a series of long-term physical, cognitive, and reproductive outcomes (Rivera and Habicht 1996, 2002; Martorell 1995). Based on this, and other, evidence, the USAID Food for Peace office has recommended applying a preventive approach to human capital formation in Title II MCHN programs, including investing in early childhood nutrition.2 A recent World Bank report also urges governments to invest in preventing undernutrition among children under two years of age as a means of promoting national development and human capital formation (World Bank 2006). The focus is on the prevention of undernutrition, rather than cure; and on achieving cost￾effectiveness by allocating resources to children who have the greatest potential for response. Program implementers are thus increasingly encouraged to reallocate their resources to focus on prevention among children under two, rather than on the recuperation of under-five children with mild to moderate undernutrition. Without programmatic evidence showing the feasibility and effectiveness of the preventive approach, however, program implementers have been hesitant to move in that direction. To address this knowledge gap, researchers from the International Food Policy Research Institute and Cornell University undertook a study commissioned by the Food and Nutrition Technical Assistance (FANTA) project at the Academy for Educational Development (AED) to compare the cost-effectiveness of the preventive and recuperative approaches, as implemented by WV-Haiti in the context of their Title II MCHN program in the Central Plateau. This report summarizes the research and program development activities that were undertaken by the team to design, implement, and evaluate the impact and cost-effectiveness of the two program approaches on alleviating childhood undernutrition in rural Haiti. 1 The term “recuperative” is used in this report to refer to mild and moderate undernutrition, i.e., weight-for-age or height-for-age less than -1 (mild) or -2 (moderate) Z-scores. The report does not directly address severe malnutrition defined as weight-for-age or height-for-age < -3 Z-scores and the community therapeutic care approach used for the treatment of severe acute malnutrition. 2 From the Food for Peace Expanded Conceptual Framework for Food Aid and Food Security, 2002. 2 The remainder of this chapter presents some background information on Haiti and on Title II MCHN programs, and describes the scientific rationale that motivated the study. It concludes by describing the organization of the report. 1.1 Haiti: The Poorest Country in the Western Hemisphere Haiti is the poorest country of the Latin American and Caribbean (LAC) region and is ranked 153rd out of 177 countries by the Human Development Index (UNDP 2005). Three￾quarters of the Haitian population are poor (living on less than $2 USD per day), and over half are extremely poor (living on less than $1 USD) (Sletten and Egset 2004). Extreme poverty is largely a rural problem, with 77% of the extreme poor living in rural areas. The country’s economy has been deteriorating steadily since the 1980s and real per capita GDP has since been falling at an average of 2.5% per year (http://devdata.worldbank.org). Not only is Haiti the poorest country of the region, it also has the most unequal income distribution, with a Ginni coefficient of 0.65, next to Brazil at 0.59 (World Bank 2003). Not surprisingly, this impoverished country has the worst health indicators of the region, with an under-five mortality rate of 118 per 1,000, a prevalence of chronic undernutrition of 23% (reaching 30% in the poorest regions), an excessively high maternal mortality rate (680 per 100,000 live births), and the highest prevalence of HIV/AIDS outside Sub-Saharan Africa (2.5%) (http://web.worldbank.org). Life expectancy is a mere 51 years. Access to services is a key constraint: only half of the population has access to clean drinking water and 28% to sanitary facilities; childhood measles immunization coverage is only around 50%, and less than one-quarter of all births are attended by skilled health staff (http://devdata.worldbank.org). Public spending on health was 3% of GDP in 2002, compared to about 6.5% of GDP in Mexico (in 2005).3 Education indicators are equally alarming: half of the population is illiterate (UNESCO 2002) and primary school attendance rates were 52% for men and 56% for women in 2000 for the country as a whole, and as low as 34% in some of the poorest rural areas (EMMUS-III 2001). Haiti’s current economic, social, and development profile is the result of years of internal conflicts and climatic shocks that have devastated its economy, depleted its natural and human capital, weakened its institutions, and inflicted severe hardship on its population. The years 2002-2006, when the study took place, were no exception. In fact, the period between 2003 and 2005 brought greater hardships than the preceding years, with political unrest in 2003 leading up to the exile of the president, Jean-Bertrand Aristide, in early 2004, and continued political strife and violence following the exile. Although much of the violence was restricted to the capital city, the political climate had serious impacts on the economy of Haiti and on food and fuel prices in the country (International Crisis Group 2005). 1.2 Title II-MCHN Programs USAID spends around $90 million/year on Title II-funded MCHN programs (Swindale et al. 2004). These programs either use donated food commodities as direct food assistance to 3 From the OECD Health Data 2007 (accessed July 27, 2007) http://www.oecd.org/document/16/0,3343,en_2649_34631_2085200_1_1_1_1,00 html 3 program beneficiaries and/or monetize the donated commodities to support the MCHN services offered through the programs. Most Title II MCHN programs are intended to be community￾based, and to offer or support a variety of interventions with proven impacts on child nutrition and child survival, i.e., immunizations, breastfeeding promotion, complementary feeding promotion, vitamin A capsule distribution, etc. (Swindale et al. 2004). A recent review of Title II-funded MCHN programs (Swindale et al. 2004) suggests that overall, MCHN programs implemented using the Title II mechanism have yielded reductions in stunting that average 2.4 percentage points per year and reductions in underweight that average 1.9 percentage points per year. Although few of the studies reviewed used rigorous evaluation designs, overall the results are indicative of positive, albeit relatively small, benefits on childhood undernutrition. In Haiti, USAID provides Title II resources to four major private voluntary organizations (PVO): CARE, Catholic Relief Services, Save the Children USA, and WV-Haiti (referred to as Cooperating Sponsors (CS)). All four CS in Haiti use the Title II resources to provide MCHN services directly or through collaboration with the government services, and Title II support is usually provided to program beneficiaries as direct food assistance. All four CS use a recuperative approach that provides food assistance to malnourished children under the age of five years. In its 2002-2006 Development Activity Program (DAP) proposal to receive Title II resources, WV-Haiti noted their intention to pilot test the targeting of food assistance to children under two in one of their priority areas (WV-Haiti 2001). This was a unique opportunity to test the approach under programmatic conditions and compare it with the recuperative approach. Thus FANTA/AED, with support from USAID, commissioned an evaluation, which is described in this report. 1.3 Rationale for a Preventive Approach to Reducing Undernutrition This section presents the scientific rationale that motivated this study; it describes what was known at the onset of the study regarding the potential usefulness of a preventive approach to tackle childhood undernutrition. The focus is on food supplementation studies - as opposed to nutritional interventions to improve child-feeding and care practices - because this is where most of the scientific evidence from controlled trials is available. The section also presents the behavioral rationale for the specific set of child-feeding and care practices that the study focused on in its behavior change and communication strategy (see Chapter 4 for a description of the strategy). 1.3.1 The recuperative versus the preventive approach Food distribution programs have often failed to improve the nutritional status of their beneficiaries (Beaton and Ghassemi 1982) and their lack of impact has been attributed to the fact that they often failed to target children who were most likely to benefit. The traditional recuperative approach has the major drawback of intervening too late, i.e., once children’s growth has fallen below a certain cutoff point. This often happens after months of growth 4 faltering and at ages when children have limited potential to respond to nutrition interventions, particularly in such a way as to improve stunting. The preventive approach, on the other hand, is designed specifically to address this concern by targeting children before their growth falters and by intervening during their period of highest growth velocity, and of maximum potential to benefit from nutrition interventions (0-24 months of age) (Schroeder et al. 1995; Lutter et al. 1990; Allen 1994; Rivera and Habicht 1996, 2002). The preventive approach thus aims at preventing children from getting malnourished and thus needing to be recuperated. By maintaining children’s growth at a higher level throughout their most vulnerable period (0-24 months of age), it is expected that not only short-term, but also long-term, benefits on growth will be achieved (Martorell 1995). 1.3.2 Biological rationale for targeting food supplementation to children under 24 months The rationale for targeting food supplements (and other nutritional interventions) to children under 24 months of age is based primarily on current knowledge regarding the patterns of growth of young children and their response to food supplementation interventions. A brief overview of this scientific evidence is presented below. 1.3.2.1 Patterns of child growth in Haiti and other developing countries Recent national-level data from Haiti show that approximately one-third of children less than three years of age in Haiti have low height-for-age (< -2 Z-scores) (EMMUS-III 2001). Growth curves using data from the Demographic and Health Surveys data from 2000/2001 for Haiti (see Figure 1.1) show the typical pattern found in most developing countries around the world where mean height-for-age (and weight-for-age) decline almost linearly from birth to approximately 18 months, after which they tend to stabilize at a low level (Ruel 2001; Shrimpton et al. 2001). The data on patterns of child growth in Haiti (and in most of the developing world) show that children under two are clearly the most vulnerable to growth faltering and this period is, therefore, a time when interventions to prevent the rapid decline in nutritional status are most needed. Note that the similarity in growth patterns between children from Haiti and from other contexts provides reassurance that the information currently available on optimal timing and duration of supplementation (presented in the following sections) applies to all developing country contexts. 1.3.2.2 Patterns of response to food supplementation Evidence from randomized controlled trials suggests that the impact of supplementary feeding interventions on child growth is determined by two key factors: (1) the timing of the intervention (child age at the time of the supplementation), and (2) the duration of supplementation. A brief overview of relevant findings for the present evaluation is presented below. 5 Figure 1.1 Mean weight-for-height (WHZ), weight-for-age (WAZ), and height-for-age (HAZ) of rural children in Haiti (EMMUS-III 2001) Child age groups (months) 0-5 6-11 12-23 24-35 36-47 48-59 Mean-z-scores -2.0 -1.5 -1.0 -0.5 0.0 0.5 WHZ HAZ WAZ Effect of timing of supplementation interventions on overall growth impact Schroeder et al. (1995) showed that in rural Guatemala, the greatest impact of food supplementation was achieved among children in their first and second years of life, and that no impact was found from three to seven years of age. In a different study setting, urban Colombia, Lutter et al. (1990) demonstrated that within the first 24 months, the greatest response to supplementation was seen in infants between 9 and 12 months of age, the peak period of diarrheal morbidity in this population. Finally, observational research from the Nutrition Collaborative CRSP in Mexico, Kenya, and Egypt showed that improved diets after the age of 18 months were not associated with better nutritional status (Allen 1994). Based on some of this research, Beaton provides convincing arguments in his review of targeting for supplementary feeding more than 10 years ago, that the “effective age range for substantive impacts on linear growth seems to be six months to two or perhaps three years” (Beaton 1993, p. 47). He clarifies that the implications are not that no benefit at all can be expected beyond this age, but that the magnitude of benefits may not justify the additional costs of providing food supplements to children after three years of age. 6 Thus, evidence suggests that maximum benefits from improving dietary intake, including through programs that provide food supplements, will be most effective in preventing malnutrition in the period of approximately 6-24 months of age. This is not surprising because this is the period of maximum expected growth velocity and also the period of greatest risks of growth faltering due to inappropriate complementary feeding practices and increased risks of infectious diseases rates, especially diarrhea. Effect of timing of supplementation interventions on faltering and recovery rates While it is important to examine the overall impact of supplementation at different child ages, it is also useful to understand through longitudinal analysis how supplementation affects the rates of growth faltering and the rates of recovery from faltering. Only two studies, both using the Guatemala longitudinal supplementation study conducted in the 1970s in four rural communities, have examined the differential impact of supplementation on faltering and recovery rates in weight-for-height (WHZ) (Rivera and Habicht 1996, 2002). The analyses confirm that the impact of supplementation on the prevention of faltering (maintaining a weight/length category during a specific supplementation period) is age￾dependent. The authors found a much larger impact on the prevention of faltering in WHZ among children who were 6 to 24 months old at the time of the intervention (Rivera and Habicht 2002). In this age group, the faltering rate among those receiving the food supplementation intervention was 0.19 in contrast to 0.45 among nonsupplemented children, a difference of 0.26, which was due to the supplementation. The much smaller difference of 0.08 for the same comparison among children between 24 and 48 months of age was not significant. Recovery from faltering was also found to be age dependent. Among 6-24-month-old children who had received the supplementation for 12 months, the rates of recovery from faltering was 0.78 for supplemented children and only 0.41 for those without the supplement, a difference of 0.37, which was due to the supplementation. Again there was no effect among the 24-48 month old children. Effect of the duration of supplementation There is limited research on the optimal duration of food supplementation needed for maximal impact. The only information available that we are aware of comes from analyses of the Guatemala longitudinal trial (Rivera and Habicht 1996). In this context, although 59% of infants had recovered from faltering in WHZ within 3 months of supplementation, greater impacts were achieved with 12 months of supplementation, reaching almost 80% of children. These data suggest that longer durations of supplementation (6-12 months) are likely to have more impact than shorter durations (3 months). Overall, scientific evidence regarding patterns of growth faltering in early childhood and the impact of food supplementation on child growth in countries like Haiti, suggests that: ™ The age of “active” growth faltering is between 6 and 18-24 months of age; 7 ™ The greatest growth response is achieved when food supplementation is provided during the first 24 months of age and for a duration of at least 6-12 months. These insights suggest that a preventive approach of food supplementation targeted to all children between 6 and 24 months is likely to contribute to reducing childhood undernutrition in poor communities. Food supplements without appropriate care and feeding practices, however, are unlikely to make a significant dent in undernutrition prevalence. Thus, food supplementation interventions must be combined with a strong behavior change and communication strategy to help caregivers use the donated food commodities optimally and adopt recommended child￾feeding and care practices. The behavioral rationale for combining food supplementation and BCC interventions is presented below. 1.3.3 Behavioral rationale for combining food supplementation and behavior change communication strategies In children, positive changes in health and nutrition manifest themselves in many forms, including improvements in growth and development and reductions in morbidity and mortality. Each of these outcomes is the result of complex interactions between familial caregiving practices and the biological underpinnings of health and nutrition. For example, in order to protect a child from a vaccine-preventable disease, such as measles, the family must know when and where to take the child for the vaccination, must have the resources to carry out these actions, and the vaccine itself must be safe and effective. The availability of the vaccine is powerless to prevent the disease in the child without the parental actions. Similarly, to enable children to grow normally, there are many parental caregiving practices related to food that are essential to ensuring adequate nutritional intake. These practices include obtaining and selecting foods that meet nutritional requirements, preparing them safely and in a form that is appropriate for the child’s age, and feeding them in a manner that encourages adequate intake. In order to engage in these critical caregiving practices, parents need access to the foods their children require; they need access to fuel, water, and other materials to prepare and preserve these foods, and time and physical energy to carry out the activities. They also need knowledge. These are essential underpinnings of nutrition and health￾giving practices, which in turn are the prerequisites for child health and well-being. Because caregiving practices are the links between resources and knowledge, on one hand, and child health on the other, programs that seek to improve child health and nutrition must, by definition, change caregiving practices. Thus, it is important that both the preventive and recuperative approaches implemented by WV-Haiti ensure that in addition to providing food to children under the age of 24 months, adequate information is provided to caregivers to ensure that these foods are fed appropriately to these young children. Along with this, it is also crucially important to ensure that other aspects of feeding and care that are important for the preventive age group are also addressed. The key aspects of care and feeding to address in the vulnerable period of 0-24 months of age are breastfeeding, complementary feeding, and other preventive and curative health-related practices like good hygiene, timely immunization, appropriate home health care, and care-seeking during illness. This section presents a summary of current recommendations for the feeding of infants and young children under the age of 24 months and also briefly summarizes current evidence 8 regarding the impact of BCC programs on influencing these practices. The technical basis for the feeding recommendations is not described here. However, they are described in detail in an article in the Food and Nutrition Bulletin (Dewey and Brown 2003). 1.3.3.1 Current infant and young child-feeding recommendations Current infant and young child-feeding recommendations are derived from “Guiding Principles for Complementary Feeding of the Breastfed Child” (PAHO/WHO 2003), and provide guidelines for appropriate feeding of breastfed infants from 0-23 months of age in developing countries. The following specific dimensions of infant feeding are covered in these Guidelines: ¾ Duration of exclusive breastfeeding and age of introduction of complementary foods: Practice exclusive breastfeeding from birth to 6 months of age, and introduce complementary foods at 6 months of age while continuing to breastfeed. ¾ Maintenance of breastfeeding: Continue frequent, on-demand breastfeeding until 2 years of age or beyond. ¾ Responsive feeding: Practice responsive feeding, applying the principles of psychosocial care. ¾ Safe preparation and storage of complementary foods: Practice good hygiene and proper food handling. ¾ Amount of complementary foods needed: Start at 6 months with small amounts of food and increase the quantity as the child gets older, while maintaining frequent breastfeeding. ¾ Food consistency: Gradually increase food consistency and variety, as the infant gets older, adapting to the infant’s requirements and abilities. ¾ Meal frequency and energy density: For the average healthy breastfed infant, meals of complementary foods should be provided 2-3 times per day at 6-8 months of age and 3-4 times per day at 9-11 and 12-23 months of age. ¾ Nutrient content of complementary foods: Feed a variety of foods to ensure that nutrient needs are met. Meat, poultry, fish, or eggs should be eaten daily, or as often as possible. Vitamin A-rich fruits and vegetables should be eaten daily. ¾ Use of vitamin-mineral supplements or fortified products for infant and mother: Use fortified complementary foods or vitamin-mineral supplements for the infant, as needed. ¾ Feeding during and after illness: Increase fluid intake during illness, including more frequent breastfeeding, and encourage the child to eat soft, varied, appetizing, favorite foods. 9 Although Guiding Principles do currently exist for non-breastfed children (Dewey, Cohen, and Rollins 2004), they were not available at the time our BCC strategy was designed, and were therefore not considered. Other special situations that were not addressed in our BCC strategy include the feeding of children recuperating from severe malnutrition and the feeding of infants born to HIV-positive mothers. 1.3.3.2 Evidence of the impact of behavior change communication interventions on feeding practices and growth outcomes A review of complementary feeding interventions provides evidence that BCC programs can be effective in reducing child undernutrition in a variety of contexts (Caulfield, Huffman, and Piwoz 1999). Studies published after the review by Caulfield, Huffman, and Piwoz (1999) also continue to provide evidence that targeting of behavior change communication (BCC) to caregivers of children in the 6-24 month age range can have beneficial impacts on child growth. Studies from settings such as India (Bhandari et al. 2004, 2005) as well as Peru (Penny et al. 2005) show that different approaches of targeting BCC programs to mothers of children 0-24 months of age can yield benefits in terms of improved breastfeeding practices as well as improved complementary feeding practices, and thus better growth outcomes. A few studies also point to the impact of targeting mothers of undernourished children with nutrition education. In particular, the success of the Hearth approach and the Positive Deviance approach in settings such as Haiti and Vietnam are notable (Pachon et al. 2002; Mackintosh, Marsh, and Schroeder 2002; Schroeder et al. 2002). A recent study from Bangladesh also points to the added benefit of combining nutrition education with provision of supplementary food (Roy et al. 2006) rather than either intervention alone on improving growth outcomes for undernourished children. At the same time, although this evidence suggests that undernourished children can recuperate through intensive BCC interventions, there is no evidence to suggest that targeting BCC interventions to mothers of undernourished children has impacts on overall community-wide undernutrition. Furthermore, as with studies on supplementation of young children, there are no studies that compare the behavioral and biological impact of targeting behavior change communication to caregivers of children in the critical age group of 0-24 months to the impact of targeting caregivers of undernourished children. 1.3.3.2.1 Factors that influence the success of BCC programs The review by Caulfield, Huffman, and Piwoz (1999) also provides a basis for understanding the various factors that influence the success of behavior change communications programs to achieve their goals. The authors found that the success of BCC programs depends not only on the design, targeting, and outreach of the program, but also on contextual factors, such as community involvement and political commitment. Specifically, the successful programs reviewed by Caulfield and collaborators used very similar approaches to program design. These approaches all included a number of stages of formative research, such as a review of existing materials related to infant feeding in the program areas, ethnographic research to understand current infant feeding practices and their motivations, an assessment of current complementary foods, and recipe trials to develop enriched complementary foods. Additionally the development of program strategies in all cases used a comprehensive approach, which took 10 into account contextual facilitating factors and the findings from the formative research. These programs showed substantial improvements in caregiver knowledge and recall of program messages, increased intakes of complementary foods, and, furthermore, improvements in child nutritional status (height-for-age and weight-for-age) that were similar to improvements seen from food supplementation studies. Even taking the problematic design of some of the program evaluations into account, the authors estimated that BCC programs could improve child nutritional status by as much as 0.1 to 0.4 Z-scores. Some of the features of the programs that could have contributed to these successes are likely to be their attention to the local context within which the program was to operate, an in￾depth understanding of infant feeding practices that was based on solid formative research, and a program strategy that used a comprehensive approach in its design and implementation. Almost all the programs reviewed, for instance, used very clear key messages that were age-appropriate and action-oriented and that would allow caregivers to make easy changes in infant feeding that were adapted to the child’s stage of development. The programs also used multiple approaches to reach caregivers, usually combinations of mass media and individual advice and counseling. In addition, they used a variety of communications methods, such as radio spots, cooking demonstrations, story telling and drama, and all of these were accompanied by appropriate visual communications materials like posters, counseling cards, and take-home cards on infant feeding. 1.3.3.2.2 Implications for the design of the preventive and recuperative approaches The implications of the evidence on the success of BCC programs for this evaluation are that the preventive approach of targeting food assistance could be expected to benefit from a BCC program that is also designed to be preventive in nature, and that addresses the special needs of children in the 6-24 month age group. This is particularly important because the changing nature and complexity of recommended infant and young child-feeding practices in the 6-24 month age range present a daunting challenge both to caregivers and to program implementers. To be effective, a BCC program must address the range of practices recommended at different ages, and must do so in ways that are culturally appropriate and timely. In addition, current evidence suggests that successful BCC programs are those that are grounded in formative research and locally relevant, and that use multiple channels to reach program clients. These principles have been applied in developing the preventive and recuperative approaches, and the process for doing so is described in Chapter 4 of this report. 1.3.4 Conclusions This section presented the biological and behavioral rationale for designing nutrition interventions that have a preventive focus and that target children less than two years of age. In addition, it laid the basis for the discussion on infant and young child-feeding practices, which is further discussed in Chapter 4. This section also provided evidence that the growth patterns of Haitian children are similar to those of children in other developing countries worldwide. In doing so, it provides justification for the use of global evidence on child growth and nutritional interventions to design this particular evaluation in Haiti. It also provides some reassurance that the results of this evaluation can be used to inform decisions about the potential impact of a prevention-oriented nutrition intervention in other settings. 11 This evaluation is the first study to compare two approaches of targeting a Title II￾MCHN program, and to do so with a full consideration of both the biological and behavioral issues around targeting the food assistance and BCC packages. The next two chapters present an overview of the context within which the program is implemented and the process used to design the full preventive approach and the strengthened recuperative approach. In doing so, they address some of the issues raised in this section about the optimal timing and duration of targeting under-twos as well as the behavioral issues to be considered in developing a BCC program targeting this age group. 1.4 Organization of the Report This report is organized into 12 chapters that lead from the objectives and design of the study to the programmatic implications of the findings of the evaluation. Chapter 2 presents the objectives and design of the evaluation, and discusses the role of different research activities within the overall evaluation. Chapter 3 describes WV-Haiti’s Haiti MCHN program and the services provided through the program. Chapter 4 describes the process of designing the preventive and recuperative approaches and provides an overview of formative research and program development activities. Chapter 5 provides an overview of the operations research process used to strengthen program delivery. Chapter 6 describes the program participation patterns in the two program approaches as well as the characteristics of participants and nonparticipants. Chapter 7 describes the primary nutritional impact of the preventive approach relative to the recuperative approach, while Chapter 8 describes the impact of the program on behavior change outcomes, including maternal knowledge, trial, and adoption of key recommended practices as well as infant and child-feeding practices. Chapter 9 presents results that demonstrate the impact of the program on household food security while Chapter 10 describes indirect benefits of the program on household assets and maternal well-being measures. Chapter 11 presents the relative cost-effectiveness of the two program approaches. The report concludes with Chapter 12, where key results are summarized, and their program, policy, and research implications are discussed. 12 13 2. OBJECTIVES AND DESIGN OF THE EVALUATION 2.1 Introduction This chapter presents the objectives of the evaluation and an overview of the program theory behind this evaluation. It also describes how the various project activities were designed to address the intended objectives and how they fit with the theoretical framework that guided this evaluation. Finally, the chapter describes the collaborative engagement of the research teams from the International Food Policy Research Institute (IFPRI) and Cornell University with the implementation team from WV-Haiti. 2.2 Objectives of the Evaluation The overall objective of the evaluation was to compare the impact and cost-effectiveness of the preventive and recuperative approaches of integrated nutrition and health interventions including a food supplementation component. The specific objectives were to: 1) Compare the impact of the preventive and recuperative approaches on the following outcomes among children aged 12-41 months: a. Attained growth (mean WAZ, HAZ, WHZ, and their distributions); b. Prevalence of undernutrition (stunting, wasting, underweight). 2) Compare the cost of the two approaches with respect to financial and human resources such as the amount of food required, staff training, and time. 3) Compare the cost-effectiveness of the two approaches, combining information from 1 and 2. 2.2.1 Additional objectives In addition to the objectives mentioned above, the overall study also had the following additional objectives: 4) Assess differences between the two interventions in coverage of their respective targeted age groups (preventive: 6-24 months; curative: 6-60 months). 5) Document, with the use of operations research methods, differences between the two intervention groups in (1) the effectiveness of delivery of the various components of the two intervention packages; (2) the quality of the services provided; and (3) the institutional setup that appears to facilitate successful implementation. 6) Document, using qualitative research methods, the intrahousehold utilization and consumption of the food commodities, particularly consumption by the target individual. 14 7) Assist WV-Haiti in the design and implementation of a fully developed preventive approach to be compared with the recuperative approach. This will include designing new education messages that emphasize prevention of growth faltering, and designing a delivery mechanism to ensure the timely delivery of the messages to the targeted audience. 8) Assist WV-Haiti in reviewing and improving (if necessary) the set of education messages currently used in the recuperative approach. This report presents detailed results that pertain to the main three objectives (impact, cost, and cost-effectiveness) as well the additional objective pertaining to the coverage of the program approaches. Objectives pertaining to the operations research, the design and implementation of the preventive and the strengthening of the recuperative approach are briefly described here, and have been elaborated in previous reports (Menon et al. 2002a, 2002b, 2005; Loechl et al. 2003a, 2003b, 2004). The objective pertaining to the use of the donated food commodities is partially discussed in the first round of operations research (Loechl et al. 2004). It is also addressed briefly in the chapter on child-feeding practices in this report. 2.3 Design of the Overall Evaluation 2.3.1 Program theory and pathways of influence Specification and assessment of “program theory” pathways (Rossi, Lipsey, and Freeman 2004) is important for understanding how and why a set of program activities achieved (or did not achieve) their intended impact. This is particularly true for complex programs that can involve multiple intervention components, each of which place different demands both on program implementers and on program clients. According to Rossi, Lipsey, and Freeman, program theory can be expressed as impact theory as well as process theory. Impact theory postulates the pathways through which an intervention is expected to achieve stated objectives. Process theory, on the other hand, is more concerned with the pathways that ensure that an intervention is implemented and utilized as designed. In the case of the WV-MCHN program, the impact theory is based on the tenets of the Title-II programming environment, where it is recognized that it is important to provide food assistance to improve family food security and increase access to fortified foods in the household. At the same time, it is also recognized that to reduce child undernutrition, provision of other inputs such as behavior change communication (BCC) are necessary to ensure a home environment conducive to good childcare practices and child growth. Thus, the impact theory for the WV-MCHN program in Haiti is that provision of food and nutrition education inputs can ensure good care practices and consumption of fortified food commodities, which in turn should ensure good nutrition. The process theory for the WV MCHN program is based on the institutional demands and needs for implementing a Title II program providing both food assistance and BCC inputs. Working backwards from the most immediate program inputs, it is apparent that distributing food assistance requires capacity in monitoring beneficiary eligibility, ensuring smooth logistics, preventing leakage and corruption. On the other hand, ensuring good behavior change 15 communication requires staff who are trained in technical content as well as counseling and facilitation skills. In both cases, it is essential that other managerial and organizational processes are in place to ensure that workers are supported, motivated, and supervised appropriately. Process theory can also be used to describe expectations regarding program uptake and utilization by intended beneficiaries. In the program approaches being evaluated, there are some common expectations about uptake and program use. Specifically, in both groups, the expectation is that all families with children under 60 months of age will use the RPs so as to receive general preventive health-care services. Then, for children identified as program beneficiaries, the expectation is that caregivers will fulfill their responsibilities to attend MCs and continue to bring their children to RPs. Figure 2.1 provides a broad view of the “flow” of programmatic inputs from WV-Haiti to the child. This broad framework is intended to identify the contributing role of each step in this flow of activities to the achievement of the programmatic goals of improved child health and nutrition. It recognizes the variety of factors that influence program delivery as well as those that influence program uptake by caregivers and households in this context. In doing so, it demonstrates the complexity of the actions and actors that need to come together to lead to the ultimate intended outcome of the program, i.e., child health and nutrition. (1) WV-Haiti program management: Factors that are important at this level are those aspects of the WV management that facilitated the design and implementation of the preventive and recuperative approaches, and the changes to the program implementation based on the operations research carried out in 2003. In particular, the management’s commitment and investment in ensuring high quality program implementation are important considerations at this level, as are general organizational structures, principles, and resources. (2) WV-Haiti program implementation level: Staff-related factors that are likely to influence the efficiency and quality of program operations are important to consider at this level. Some of these factors include technical expertise, job motivation, supervision, etc. In addition to these, factors external to the frontline staff but internal to the program, such as resources needed for smooth operations (transport, medical supplies, fuel, interruptions in the food pipeline, etc.), can influence program operations. Similarly, factors external to the program, for instance, infrastructure and political disruptions, can also exert an influence on program implementation. (3) WV program operations and inputs: At this level, we consider the Maternal and Child Health and Nutrition (MCHN) program inputs provided to the program beneficiaries. These include the food assistance package, as well as the health and nutrition knowledge provided by the BCC strategy and other WV MCHN program services. Finally, other WV programs in the evaluation areas, such as water, agriculture or sanitation, to name a few, can modify the influence of the MCHN “intervention package.” The program inputs are considered separately from the contextual issues described above that influence the delivery of these inputs. 16 Figure 2.1 Program impact pathways (4) Care context: The program inputs provided by the WV MCHN program are expected to influence the context within which childcare occurs. Specifically, the program inputs can have an influence on community, household, and caregiver resources for the care of young children, including resources such as food availability, access and utilization, as well as knowledge, access to health care, etc. In addition, quality of program services and the rapport WV Program Implementation Level Staff-related factors (motivation, supervisory structure, technical expertise of staff) Program resources available to staff Food pipeline issues External factors (road, local infrastructure, political disruptions, etc.) Program operations and inputs Food ration Health and nutrition education (BCC) Social support Preventive health-care services Childcare context Care resources (community household, caregiver) Care practices Impact on child nutritional status and health WV Program Management Level (Factors that facilitated research and set up of operations + operational changes) Organizational policies re. training, supervision, etc. Program delivery Program access & uptake? Child impacts 17 between program and the communities can influence how communities and families attend and use program services. Consequently, they are expected to have an impact on how the program inputs are translated into care practices for children, and ultimately into child outcomes. The program inputs can change the resources available for childcare practices, as well as the nature of the relationship between the care resources and care practices. (5) Child impacts: The impacts at this level are the final impacts of interest of the program evaluation. Indicators of child nutritional status (HAZ, WAZ, and WHZ) are the primary impact indicators of this evaluation. 2.3.2 Designing evaluation activities based on program theory The nutrition intervention delivered through the WV MCHN program includes two major components: the food assistance component and the BCC component. Both of these components required different skills and capacities from program implementation staff. They also required different kinds of “compliance” from program beneficiaries. In short, the full implementation and uptake of the program requires a diverse set of actors to act in concert. Recognizing this complexity, the entire evaluation process was designed to ensure that the key program components were designed from a “best practices” perspective and that they were implemented as best as possible. This was accomplished by engaging in a set of activities that informed the design and implementation of the program approaches. However, since the research team was not engaged in any implementation per se, all activities that pertained to program development and implementation were conducted through active engagement with WV￾Haiti (see Figure 2.2). This resulted in a program that was designed and implemented as well as possible under the extant field conditions in Haiti. From an evaluation perspective, the complexity of the program theory was captured through attention to assessment of each step in the program pathway. This was accomplished through the careful documentation of program implementation via operations research studies. In addition, the baseline and final surveys gathered detailed data on care resources and program utilization patterns from the survey communities. This ensured that all steps along the program pathway were documented well to aid in the interpretation and understanding of evaluation results. Ensuring program components were designed well: The evaluation paid close attention to the design and implementation of the BCC strategy used in both program approaches by engaging in a solid formative research study as well as a consultative BCC strategy development process with WV-Haiti. Since WV-Haiti’s commodities tracking system was already designed and was operating well, the design process only involved discussions that resulted in the definition of the preventive and recuperative food ration packages. Assessing and enhancing the implementation of program components: The implementation of the MCHN program, including the implementation of the food assistance and BCC components, was examined through a two-stage operations research process. Furthermore, with the recognition that program management and supervision have a strong influence on how 18 program activities are implemented in the field by frontline staff, the evaluation also examined the management structures of the program and the work context of frontline program staff. Figure 2.2 An “engaged” model of evaluation and technical assistance The information gathered on program implementation in the first operations research study was used to discuss implementation problems and concerns with the WV-Haiti management and field staff, which led to an internal program improvement process implemented by WV-Haiti. In a subsequent round of operations research, the research team assessed these Baseline assessment (2002) Formative research for program planning (2002) Program development and staff training (2003) Impact assessment (2005) Changes in program implementation (2004) Program implementation (2003) Operations Research (2003) (Round 1) Operations Research (2004) Round 2 Consultative Program Improvement Workshop (2004) 19 program improvements and also compared implementation between the two program approaches. Assessing program uptake: The program evaluation activities were also grounded in the understanding that accessing and translating the WV-Haiti MCHN program inputs (food assistance and BCC) into child health outcomes requires substantial investments and attention by caregivers and households. This understanding was used to design the BCC strategy in ways that could encourage the best use of program resources for the child’s well-being. Also, the research activities were designed to ensure that caregiver and household use of the program resources could be assessed to allow for a better and more nuanced understanding of how program inputs were translated into child health. Specifically, a variety of household and caregiver resources that could influence the use of program inputs were measured both at baseline and final surveys. Since some of these household and caregiver resources (particularly household food security and caregiver knowledge) could have been modified by the program activities, the evaluation process paid close attention to assessing them. In addition, the second round of operations research gathered qualitative data on participant adoption of the BCC recommendations. Finally, at the time of the final survey, a variety of data was gathered to evaluate the use of general program services as well as the use of the specific nutrition inputs, i.e., the food assistance and the BCC recommendations. An overview of the program design and research activities that were employed in this evaluation is described in subsequent sections. Details about specific activities are presented in Chapters 3 through 11. 2.4 Project Activities The study objectives were met through an engaged process with WV-Haiti that included two major components: a program design component that designed and developed the two program approaches in collaboration with WV-Haiti, and an evaluation component that compared the impact and cost-effectiveness of the two intervention approaches. The program and implementation plan design component included a research stage that preceded the program development stage, and included the following activities: A) Research i. A desk review of best practices for child nutrition and care, and current approaches and materials used in BCC programs in Haiti ii. An exploratory pre-survey qualitative study iii. A formative research study including a focused ethnographic study (FES) and household trials of improved practices (TIPs) iv. Two rounds of operations research to inform program implementation4 4 Note that the operations research studies were done as a sequence. However, the first operations research study was intended to assess the overall implementation of the newly developed program activities and use that information to correct any implementation problems. The second round, on the other hand, was conducted more to examine potential differences in implementation and utilization between the two program models. 20 B) Development of the behavior change and communication (BCC) strategy and preventive approaches i. Identification of priority programmatic actions for the BCC component ii. Development of the BCC strategy iii. Development of BCC materials and training plans iv. Training of field staff v. Developing an implementation plan The evaluation included four main components: A) An impact evaluation; B) A qualitative study, to guide the design of the baseline survey instrument; C) A round of operations research that assessed and compared program implementation and uptake between the two intervention groups; D) A cost-effectiveness study. The research phase of the program design component was carried out largely by the IFPRI-Cornell team, while the program development phase was carried out with continuous engagement with the WV-Haiti team. Furthermore, although the research team from IFPRI and Cornell were highly involved in the design of the preventive approach and the development of the behavior change and communication strategy, all program implementation was done and supervised by WV-Haiti. A broad timetable of the study activities is presented in Table 2.1. Table 2.1 Summary of study activities Year 2002 2003 2004 2005/2006 Design of program and implementation plan Research ¾ Desk review ¾ Exploratory qualitative study ¾ Formative research study Development ¾ Identification of priority actions for BCC ¾ Development of BCC strategy ¾ Development of materials and training plan ¾ Training ¾ Development of implementation plan ¾ Implementation of both the preventive and recuperative approaches ¾ Operations research (Round 1) ¾ Program improvement consultation (with WV) Evaluation activities ¾ Pre-survey qualitative research ¾ Baseline survey ¾ Operations research (Round 2) ¾ Cost study (mid-term) ¾Cost study ¾Final survey 21 The remainder of this chapter briefly summarizes the evaluation activities. Specific objectives, methods, and results that pertain to each of the components are described in Chapter 5 (operations research), Chapters 6 through 10 (results of the impact evaluation), and Chapter 11 (cost-effectiveness study results). 2.4.1 Evaluation activities The four evaluation components are described below. 2.4.1.1 Impact evaluation The evaluation used a community-level randomized pre-post design, whereby 10 paired clusters of communities were randomly assigned to either the preventive or the recuperative program group. It was designed to use a probability design while providing enough evidence on the plausibility of the impact as to inform future evaluations of this nature, as well as future programs that use this approach (Habicht, Victora, and Vaughn 1999). The main outcomes of the evaluation were mean height-for-age Z-scores (HAZ), weight￾for-age Z-scores (WAZ), and weight-for-height Z-scores (WHZ), and the prevalence of childhood stunting, underweight, and wasting. The baseline survey was conducted between May and September 2002 and the post￾evaluation survey was conducted exactly three years later, between May and September 2005, to minimize seasonal variations. All components of the intervention packages, except the newly developed BCC strategy, were implemented immediately following the baseline survey, i.e., in August-September 2002. The full BCC package, however, was implemented several months later (in May 2003) due to delays in design, staff training, and implementation. Twenty clusters of communities, each attended by one Health Agent (WV-Haiti local staff in charge of program activities), were selected for the evaluation from WV-Haiti’s program areas. Each cluster was paired with another one selected to be similar in location (e.g., distance to the main highway and/or main town), geographic and ecologic conditions (e.g., whether located in the plains or the mountains), access to a health-care center, and the existence of a WV￾Haiti private sponsorship program (Area Development Program) (Menon and Ruel 2003). Within each pair of clusters, one was randomly assigned to the preventive approach and the other one to the recuperative approach. Thus, the unit of randomization was the cluster of communities covered by one Health Agent. The number of clusters (20) was determined by balancing the cost of conducting the surveys with the need to have enough clusters to be able to achieve the desired sample size of 750 children per program approach. A larger number of clusters, with fewer second-stage￾sampling units (in this case, households) within each cluster is usually preferable, but the cost of including more clusters is higher than that of including households within clusters. The baseline survey was used primarily to examine the differences between the program communities at baseline, and by doing so, to assess the success of the randomization. The impact assessment was based on comparisons between the two approaches at the final survey. 22 Further details about the target age groups for the surveys, sample sizes, and other design issues are provided below. 2.4.1.1.1 Sample size estimation Sample sizes were estimated using an equation for estimating sample sizes for differences in proportions between the preventive and recuperative groups (Cohen 1988). The necessary sample sizes to examine differences in prevalence rates are the same as those necessary to examine differences between distributions, and they are larger than those necessary to examine differences in means (Brownie, Habicht, and Cogill 1986). The effect size (magnitude of differences between intervention groups at post-intervention) used for the calculation was based on previous studies of the effect of supplementation on child growth, which ranges from 0.25 to 0.46 Z-scores for WAZ and 0.04 to 0.35 Z-scores for HAZ (Caulfield, Huffman, and Piwoz 1999). An improvement of +0.35 Z-scores of nutritional status in a population with average Z￾scores around –2.0 will result in a decline in prevalence of undernutrition from 50% to 38% (a decrease of 12 percentage points). Since the intervention in this case was randomized at the cluster level rather than at the level of the individual child, it was necessary to account for the clustering of characteristics within a cluster (called the design effect5 ). This must be taken into account when calculating sample sizes because it increases the sample size needed when the intervention is randomized at the cluster, rather than the individual level. Since information on design effect was not available at the time of sample size calculations, we assumed a design effect of 1.5 based on discussions with WV-Haiti and others working in Haiti about the homogeneity of communities in the evaluation area. We estimated a sample size of 75 children per cluster, for a total of 1,500 children. This sample size provided the ability to detect differences between groups in the final survey of 7.5 percentage points or larger in the prevalence of stunting, assuming an average design effect size of 1.5 (clustering of characteristics within cluster), an alpha of 0.05, and power of 0.90. This sample size also provided the ability to detect differences larger than 7.5 percentage points in underweight, 5 percentage points in wasting, and differences larger than –0.2 in mean Z-scores for HAZ, WAZ, and WHZ. 2.4.1.1.2 Age group selected for impact assessment Children 12-41 months of age were selected for the impact assessment, based on available scientific evidence on the age of greatest nutritional vulnerability and largest potential for response to nutritional interventions. This estimation was based on research showing that nutrition interventions (such as protein energy supplementation) are more effective in improving growth if provided as early as possible before the child reaches 24 months of age (i.e., from 6 months on) (Schroeder et al. 1995), and if sustained for at least 12 months (Rivera and Habicht 5 The design effect is the ratio of the variance for the cluster sample divided by the expected variance of a simple random sample of the same size. Since the design effect is dependent on the variance between cluster, it will be smaller if the number of clusters is large and the number within each cluster is small (Foreman 1991). For complex nutrition surveys, it has been shown that clusters with 30 children in each cluster lead to design effects for stunting (HAZ %<-2) that range from 0.44 to 2.13 and 1 to 1.62 for underweight (defined as W/A < 60%) (Katz 1995). 23 2002). Moreover, growth patterns of Haitian children, which are similar to worldwide patterns documented for children living in impoverished environments, show the greatest rates of growth faltering during the period between 6 and 17 months of age (EMMUS-III 2001; Shrimpton et al. 2001). Children considered most likely to benefit from the preventive approach were those who were first exposed to the supplementation between 6-11 months of age, and for the whole duration of their period of greatest vulnerability (i.e., up to 24 months of age). These children would be 24-41 months old at the final survey. In addition, we also included children 12-23 months, who were only partially exposed (i.e., had not yet reached 24 months at final survey). For the recuperative approach, the 12-41 months age range was also expected to include mostly children who had already been exposed to the program, with some possible truncation among children in the younger age range, given that the peak prevalence of underweight children in Haiti is 12-17 months (EMMUS-III 2001). As noted above, program implementation started immediately after the baseline survey, except for the new BCC strategy, which was implemented 9 months later. This meant that children who were 36-41 months at final survey were not fully exposed to all program components in either program model. Thus, the sample at final survey includes children 24-35 months of age who were fully exposed to the preventive program, and two groups of partially exposed children (12-23 months; and 36-41 months). In 2002, the survey sample included 801 children in the preventive group and 801 children in the recuperative group, for a total of 1,602 children between 12 and 41 months of age. In 2005, data were collected on 749 children in preventive and 751 children in the recuperative group, yielding a total of 1,500 children between 12 and 41 months of age in both groups. 2.4.1.1.3 Survey design The baseline and final surveys included both a household and community questionnaire. The household questionnaire was administered to the mother of the index child6 (referred to as the caregiver), whereas the community questionnaire was administered using a group interview methodology with key community members, such as religious leaders, medical staff, or schoolteachers. The community questionnaire provided data at the community level that allowed a comparison of the different clusters after the randomization process was completed. Community-level data were gathered on the smaller community units (called localités) within each cluster, because these were more meaningful and finite geographic entities than the clusters themselves. Information was gathered on access to the nearest major town, the main activity of the residents, key geographic characteristics, access to public services such as schools and markets, and access to health services such as hospitals, dispensaries, and the services provided by the WV-Haiti program. 6 The “index child” is defined as the 12-41 month-old child of interest for assessment of impact on nutritional status. 24 The household questionnaire gathered data on household characteristics such as household composition, socioeconomic status, and food insecurity; and on caregiver characteristics such as education, childcare knowledge and experience, women’s empowerment and involvement in decisionmaking, and mental and physical health. It also gathered data on six types of childcare practices: (1) early infant feeding practices (i.e., around birth); (2) current child-feeding practices of index child including meal frequency, dietary diversity, intake of animal sources foods; (3) child feeding practices for younger sibling, (4) preventive and curative health-care seeking practices (e.g., antenatal care, childhood immunization, treatment for diarrhea); (5) hygiene practices (e.g., child, maternal, and house cleanliness); and (6) discipline practices (see list of baseline and final survey modules in Annex 2.1). Anthropometric measurements (height and weight) were taken on the caregivers, the target child (12-41 months of age), and his/her younger sibling if any. Weight was measured using a UNICEF Seca scale (SECA Ltd., Birmingham, U.K.) accurate to 0.1 kg, and height was measured using wooden length boards accurate to 1 mm (Irwin Shorr Productions, Olney, Maryland, U.S.A.). Weight and length were measured by fieldworkers who were standardized using recommended protocols. Child health and appetite were assessed using a visual analogue scale, where caregivers are asked to rate the health (or appetite) of her child as compared to other children his or her age and indicate where it fell on a 10-cm-long line drawn on the questionnaire. Caregivers were also asked to recall whether the child had symptoms of illnesses (diarrhea, cough/cold, pneumonia, or fever) in the two weeks prior to the survey. Finally, spot-check observations were used to assess the cleanliness of the child, caregiver, and the household. 2.4.1.1.4 Selection of survey households The survey households were selected after conducting a complete census of the general population residing in the communities included in the program evaluation. A pool of households eligible for inclusion in the survey was generated for each cluster by identifying those households that had a child in the target age range, where the child’s mother also resided in the same house. Eighty-five households were selected at random (without replacement) from each cluster, and the survey administered to the first 75 of these households. When possible, the field team conducted more household interviews than the minimum 75 required to provide a larger sample size within the existing logistical constraints of the survey. The census gathered information on household composition, headship, and the date of birth of all children under 5 years of age. For all children under 5 and for women of reproductive age who were identified as being either pregnant or lactating, data were gathered on whether they had ever been program beneficiaries. 2.4.1.2 Qualitative study An exploratory qualitative study was carried out before implementing the baseline survey to assist in the design of the quantitative survey questionnaires. This study was carried out in communities outside of the study area and provided the basis for designing the questionnaires 25 used in the survey. More information on this study is provided in Chapter 4 and in Menon et al. 2002a. 2.4.1.3 Operations research studies Operations research (OR) methodologies, including both qualitative and quantitative approaches, were used to gather information on the effectiveness of delivery of the interventions in the communities included in the evaluation. Although the first round of OR was intended to inform program implementation, it is described here along with the second round (which was more pertinent to the impact evaluation), since both rounds of OR were linked and shared the same broad objectives. These were to (1) assess program implementation and quality; (2) improve quality of operations and fidelity to program approaches and plans; and (3) gain sufficient knowledge about implementation issues and quality in the two program approaches to allow appropriate interpretation of impact evaluation results. Two rounds of OR were undertaken, with the first occurring in July-September 2003, shortly after full implementation of the BCC strategy, and the second approximately one year later, in June-August, 2004. The first round (2003) focused on the first broad objective and aimed to assess the fidelity of implementation of the program relative to plans, assess the quality of delivery of the various services, and explore perceptions of stakeholders (i.e., participants/beneficiaries7 and field implementers) regarding the operations and quality of services provided by the program. This first round of OR (OR1) was followed by a process of consultation with program implementers, including management and field staff. The consultative process was directed toward the second broad objective: through this process, program operations were improved and better aligned with program design. After allowing approximately 5 months for staff to implement corrective actions/program improvements, a second round of OR was undertaken, with the objectives of assessing implementation of the corrective actions/program improvements identified during the consultative process. Notably, this round also served to document differences between the two program approaches in program implementation, staff work context-related factors (job motivation, supervision, etc.), and finally, beneficiary mothers’ knowledge and experiences with trial and adoption of specific infant and young child-feeding and care practices promoted in the BCC, in order to gain understanding of constraints to adoption of these practices in the two program areas. By doing so, the OR process enriched the impact evaluation through identifying any relevant differences between program areas in implementation, staff-level factors, or participants’/beneficiaries’ experiences. 2.4.1.4 Cost-effectiveness study Two cost studies were conducted to assess the cost and cost structures of the two program approaches, one in 2003-2004 (Maluccio and Loechl 2004) and the other in 2005 (Maluccio and Loechl 2006). In evaluating costs, the cost studies considered direct program costs, off budget 7 “Participants” include those who access any of the available range of services; “beneficiaries” are those who live in households receiving food assistance. 26 program costs, as well as private (beneficiary household) costs. Knowledge of the structure of costs helped identify which components of program costs were truly different between the program approaches, thus, which were important to consider in the estimates of cost￾effectiveness. The difference in costs between the program approaches was applied to the difference in program outcomes at the final survey to calculate the relative cost-effectiveness of the two targeting approaches. Details of the cost-effectiveness calculations, the various considerations that went into the calculations, as well as the implications of the results, are discussed in Chapter 11. 2.5 Conclusions This chapter presented an overview of the evaluation process, focusing on the consideration of program theory in the design of the evaluation activities. In subsequent chapters, we discuss how the program theory was translated into program design activities as well as the specific evaluation activities, such as the operations research and the impact evaluation. 27 3. WV-HAITI’S MCHN PROGRAM 3.1 Introduction This chapter provides an overview of the various services provided by the WV-Haiti MCHN program in the Central Plateau of Haiti (see map below). It describes the services and their main delivery points and briefly describes the organizational and staffing structure of the program. 3.2 Services Provided As with many other USAID Title-II funded MCHN programs, the program implemented by WV-Haiti in the Central Plateau of Haiti provides a range of MCHN services. These services are targeted to pregnant and lactating women as well as to children between 0 to 5 years of age. Many of the services offered are open to all members of the community in these groups, while some other services are provided in a targeted fashion. Food assistance, for instance, is targeted, and is provided on a conditional basis, i.e., program beneficiaries are required to use other preventive services in order to receive monthly food rations. Box 3.1 presents a summary of the program service delivery points (these are also described in more detail in section 3.3) and Table 3.1 provides an overview of the general and targeted services provided by WV program. Box 3.1 Five service delivery venues for the WV-Haiti food-assisted MCHN program Rally Posts (RPs): Health education, growth monitoring and promotion (GMP), immunizations, and other preventive health-care services are provided to all participants, and program beneficiaries are identified. Mothers’ Clubs (MCs): Small peer groups of beneficiary mothers (or mothers of beneficiary children) gather for facilitated discussions of health and nutrition topics laid out in the program’s behavior change and communication (BCC) strategy and curriculum. Food Distribution Points (FDPs): Venue for distribution of monthly food rations to beneficiaries. Pre- and postnatal consultations: Preventive health care and education are offered. Home visits: WV health staff provide additional education and referrals to mothers of newborn infants, severely malnourished children, and children whose growth has faltered. The research was undertaken in the Central Plateau region, where WV operates in all 12 communes and serves a population of approximately 600,000. The evaluation covered three communes: Hinche, Thomonde, and Lascahobas. 28 Table 3.1 Services provided by the WV MCHN program in Central Plateau Group Services available to all Targeted services to program beneficiaries Conditional servicesa (1) Pregnant women - Prenatal consultations and check-up - Tetanus toxoid immunizations - Iron-folate supplements - Mothers’ Clubs - Home visits (in late pregnancy) - Food assistance (2) Lactating mothers with children 0-6 months of age - Postnatal consultations and check-up - Postpartum vitamin A supplements (via home visits) - Mothers’ Clubs - Home visits (targeted to mothers of newborn children) - Food assistance (3) Children 6-59 months of age - Immunizations - Vitamin A capsules - ORS - Deworming - Growth monitoring and promotion - Group education (at Rally Posts) - Mothers’ Clubs (targeted differently in the preventive and recuperative approaches, see chapter 4) - Home visits (targeted to severely malnourished children) - Food assistance (4) Women 15-49 years old - Family planning a Food assistance is conditional upon beneficiaries attending monthly Rally Posts (children) or pre-natal (pregnant women)/postnatal (lactating women) clinics and monthly Mothers’ Clubs. Map of Haiti Source: https://www.cia.gov/library/publications/the-world-factbook/geos/ha/html. 29 3.3 Program Service Delivery Points As highlighted in Box 3.1, the WV MCHN program offers services at five major points of contact between program staff and participants. These are (1) Rally Posts (RPs), where health education, growth monitoring and promotion, and preventive health care are provided and beneficiaries are identified; (2) Mothers’ Clubs (MCs), where smaller groups of participants gather to discuss health and nutrition topics in the context of the program’s behavior change and communication (BCC) strategy; (3) Food Distribution Points (FDPs), where beneficiaries receive their monthly food rations; (4) Pre- and Postnatal Consultations, where pregnant and lactating women receive preventive health care and education; and (5) Home Visits, where beneficiary households with a newborn infant, a severely malnourished child, or a child with growth faltering are visited by the WV health staff. A brief description of the services offered and operational plan at each of these service delivery points is provided below. 3.3.1 Rally Post services Rally Posts (RPs) are open to all pregnant women, mothers with children less than 5 years of age, and women 15 to 49 years old in the communities attended. Services provided include health and nutrition education, growth monitoring and promotion of children under 5 years of age, immunization, vitamin A supplementation, deworming, free distribution of ORS, and information about the family planning component.8 The RPs are the main entry point into the program and are used to identify beneficiaries and to refer them to the appropriate program services. New beneficiaries are identified at the RPs every month, and new eligible children are admitted on a monthly basis. For pregnant and lactating women, however, new entrants are invited into the program only every four months. Monthly weighing of beneficiary children and monthly attendance at the RP by the child’s caregiver are mandatory for receiving food assistance. Either the mother or another caregiver can take the child to the RP. RP meetings are held on a monthly basis in each community and are managed primarily by the health agent responsible for that community. The health agent is usually assisted by at least two other health agents and two colvols. The sequence of activities described in the implementation plan is as follows (also see Figure 3.1 below): A) The education session is the first activity to be carried out at the RP. The sessions run for about 10 minutes, depending on the topic. WV sets up a calendar of monthly topics, which include the following topics: immunization, pre- and postnatal care, preparation for child delivery, diet for pregnant and lactating mothers, weaning techniques, description of kwashiorkor and marasmus, hygiene and environment, diarrhea and preparation of oral rehydration salts, acute respiratory infections, family planning, and HIV prevention. It is expected that appropriate visual materials will be used to communicate more effectively. The health agent or colvol is also expected to 8 WV-HAITI offers hormone pills and three monthly injections. Women can receive these services administered by WV nurses at mobile clinics, in health centers during pre- and postnatal consultations, or at Area Development Program clinics. 30 conduct several sessions on the same topic throughout the day to allow all mothers to participate and to restrict the size of the group to 10-15 persons. B) After the education session, the health agent registers attendance of each participant. The health agent determines whether the participant is due to receive immunization, vitamin A supplementation, and/or deworming tablets. She also updates the health cards with the information on immunization and vitamin A supplementation, and for food aid beneficiaries, she signs attendance on the ration card. This same health agent also does the distribution of vitamin A (every six months for children less than 5 years of age), anti-helminths tablets (every six months to children 2-5 years of age), and oral rehydration salts (ORS) sachets (three sachets per month per household). C) The next activity at the RP is growth monitoring and promotion. Each child is weighed and the weight is recorded on the growth chart printed in the health card kept by the caretaker. It is expected that at least the caregivers of malnourished children, especially of moderately and severely malnourished children (M2 or M3 for weight￾for-age according to the Gomez classification), will receive brief counseling about feeding practices and prevention of childhood illnesses. D) After growth monitoring and promotion, children are directed to receive their immunizations. Children receive vaccinations based on their age and previous immunization history (previously verified by the health agent in Step 2 above). Figure 3.1 Flow of activities at the Rally Post 3.3.2 Mothers’ Club services Originally, the Mothers Clubs (MCs) were designed as a venue for small peer group discussions between program health staff and beneficiary mothers on general health and nutrition topics. Monthly attendance at the MCs is a requirement for receiving food assistance. Unlike the RPs and the food distribution points, mothers are required to attend the MCs themselves and cannot send another family member to use this service. This ensures that mothers are the direct participants in the education activities offered at the MCs. Results from our initial formative research (Menon et al. 2002b) suggested that the MCs would be an ideal setting for the more comprehensive behavior change and communication (BCC) strategy that the research team in partnership with WV staff was developing. Reasons for Education Colvol or health agent: ¾ Conducts various sessions in groups of 10-15 persons for about 10 minutes on topic of the month Registration Health agent in charge: ¾ Records names, vaccine, weight, ORS, vitamin A, deworming ¾ Distributes vitamin A, anti-helminthes, ORS Growth monitoring and promotion Colvol: ¾ Weighs all children < 5 years ¾ Records weight in health card ¾ Does individual counseling of caretaker Immunization Colvol or health agent: Immunizes: ¾ children < 5 years ¾ pregnant women ¾ women 15-49 years of age 31 this are that the MCs are located close to the mothers’ homes and include only a small group of participants, resulting in minimal distraction (especially compared to the RPs). Thus, the WV program decided to use the MCs as the primary venue for BCC activities. The MCs are implemented as monthly small peer group discussion sessions, facilitated by health workers trained in technical aspects of nutrition and child health as well as in adult education techniques. Chapter 4 describes the re-organization of MCs following the formative research and related development activities, and provides more detail on how the MCs’ organization differs between preventive and recuperative approaches. 3.3.3 Food Distribution Point (FDP) services The distribution of food aid commodities to the MCHN beneficiaries of the WV program occurs on a monthly basis at special distribution points. Beneficiaries from several communities (localités) are scheduled to receive their food rations at a central distribution point on the same day. The distribution is done by community and starts with beneficiaries from the communities that are far away from the distribution point. Within each community food rations are provided first to lactating and pregnant women and then to caregivers of child beneficiaries. There are a total of 10 central FDPs covering 50 RPs in the area included in the research (including preventive and recuperative program communities). Unlike at the MCs, a beneficiary can designate another family member to collect the food ration by handing over the beneficiary card to this person. The beneficiary households receive both direct (individual) and indirect (household) rations. The amounts and commodities vary with respect to the beneficiary category (see Table 3.2). Even if a household has two direct beneficiaries participating in the program, only one indirect ration is provided. The caloric content of the rations is also indicated in the table below. Table 3.2 Composition of direct and indirect food rations, per beneficiary category Children 6-23 months of age (preventive approach) Undernourished children 6-59 months of age (recuperative approach) Pregnant and lactating women (both approaches) Type of commodity Direct child ration (kg) Indirect household ration (if child is beneficiary) (kg) Direct women ration (kg) Indirect household ration (if mother is beneficiary) (kg) WSB 8 SFB 10 5 5 Lentils 2.5 2 2 Vegetable oil 2 1.5 1.5 Caloric value (kcals/day) 1,534 1,414 1,233 1,233 Total caloric value 2,949 2,467 32 The sequence of activities at the FDPs is as follows: A) Eligibility of the beneficiary is verified by food monitors and health agents mainly based on the information on the beneficiary card. This card contains information about the beneficiaries and indicates attendance by the beneficiary at the other MCHN activities (i.e., Mothers’ Clubs, Rally Posts, and pre- and postnatal consultations) that are required in order to receive the food rations. B) Once their eligibility is verified, the beneficiaries proceed to collect their rations. A team of trained beneficiaries assists the WV staff during the distribution and is responsible for opening food sacks, measuring out appropriate amounts of each of the commodities, and handing over the food to beneficiaries. C) Finally, once the beneficiary (or designee) has received his/her rations, the food monitor verifies the rations received and the beneficiary card. In some cases, the food monitors re-weigh the rations to verify that the right amounts have been given to the beneficiary. Once this final check is complete, the food monitor signs the beneficiary card to indicate that the correct ration has been delivered to that beneficiary. 3.3.4 Pre-and postnatal consultations Prenatal consultations are provided to all pregnant women in the WV program areas who decide to use the WV services. Typical prenatal services are provided and include physical examinations, provision of iron-folate supplements, tetanus toxoid immunizations, etc. Attendance at prenatal consultations is mandatory in order to receive food rations. Postnatal consultations are also provided to all women who have recently given birth. The key services at the postnatal consultations include physical examinations for the mother and the newborn infant, provision of postpartum vitamin A (if the mother did not already receive this via a home visit soon after delivery), and other health services. As with the prenatal services, attendance at postnatal consultations is mandatory in order to receive food rations. The pre- and postnatal consultations are provided by WV nurses, usually at the WV clinics or dispensaries or at mobile clinics in the more remote areas of Central Plateau. 3.3.5 Home visits WV health staff make home visits in the areas covered by the program services to check on women in late pregnancy, mothers of newborn infants, or to follow up on severely malnourished children. The services provided at home visits usually include only a visit by the health agent or nurse, except for mothers of newborn infants who also receive a dose of postpartum vitamin A in the first home visit after delivery. 3.4 Program Organization/Staffing The program services described in the preceding sections are provided by two teams of staff: all health services are provided by a team of health staff, while all food distribution 33 activities are conducted by a team of Commodities staff. The organizational structure for both of these staff teams are provided in Annexes 3.1 and 3.2. The roles of the main WV-Haiti program implementers and the staff organizational structure are outlined below. WV-Haiti health promoters and assistant health promoters (previously called health agents and colvols, respectively). The health promoters (HPs) and assistant health promoters (AHPs) are the direct implementers of the MCHN program in the field and thus are the frontline staff in contact with the program participants. They are in charge of the interventions being delivered at the RPs and MCs and of assisting the food monitors at the FDPs. They attest to the attendance by beneficiaries at the different program activities (which determines their eligibility to receive food rations). HPs are WV employees and receive a monthly salary. AHPs, previously called colvols, used to be community volunteers who received only a small incentive from WV. However, with the growth of the program over time, it became apparent that colvols were in fact more than volunteers in terms of their time commitment to the program activities, and their level of responsibilities. In 2003, they were promoted to AHPs and received an increase in salary. They also started to participate in all training for health staff. WV-Haiti food monitors. Food monitors are responsible for the distribution of the food rations at the FDPs and for ensuring that only eligible beneficiaries receive the food. They are also responsible for verifying the beneficiary lists provided by the health team. This is done mainly by consulting the beneficiary card. WV-Haiti health (MCHN) and commodity supervisors at the local level. MCHN supervisors are nurses responsible for the supervision of the health agents and colvols, and commodities field supervisors are responsible for the supervision of the food monitors. The MCHN supervisors generate monthly lists of beneficiaries eligible to receive the food rations, based on lists of attendance prepared by the health agents for each service delivery point. The Commodity Section uses these lists to program the food amounts needed per distribution point. WV-Haiti management staff at the national and regional levels in MCHN and Commodity. The Assistant of the Regional Health Coordinator in Hinche supervises the nurses. This assistant is, in turn, supervised by the Regional Health Coordinator. The National Health Coordinator for WV is based in Port-au-Prince and oversees the activities in all the program areas of WV in Haiti. The organizational structure for the health component of the program is presented in Annex 3.1. The commodity supervisors work under the supervision of the Assistant of the Regional Commodity Officer. This Assistant is, in turn, supervised by the Regional Commodity Officer in Hinche. The organizational structure for the food component of the program is outlined in Annex 3.2. In the evaluation areas, the health team consisted of 4 nurse supervisors, 20 health promoters (HPs), and around 20 assistant health promoters (AHPs). The Regional Coordinator supervises the nurse supervisors as well as other nurses who provide services in other communes of the Central Plateau region. Each HP-AHP pair provides services to about 300 children. The remoteness and dispersed nature of households in this region of Haiti necessitate lower health worker-client ratios than in other more dense regions. 34 The commodities team provides services for the entire Central Plateau DAP area, with no distinction between staff who serve the preventive and recuperative areas. 3.5 Conclusions This chapter described the basic service provision structure of the WV MCHN program in the Central Plateau. The next chapter describes how the program design steps outlined in Chapter 2 were used to design the preventive program approach, strengthen the recuperative approach, and design the BCC strategy to be used in the MCs for both approaches. The subsequent chapter provides a detailed view of how the two rounds of operations research were used to assess and improve the services delivered at the venues described in this chapter. Together, these chapters provide a solid example of how applied program research can be used in a collaborative fashion with implementation staff to design and implement high quality programs. 35 4. GETTING IT RIGHT FROM THE START: DESIGNING THE PREVENTIVE AND RECUPERATIVE APPROACHES9 4.1 Introduction This section describes the process used to develop the preventive program and to strengthen the recuperative program. The two main sets of interventions that needed to be adapted in order to design a truly preventive approach were the food distribution (targeting and duration of intervention) and the education sessions at the Mothers’ Clubs. It was agreed at the onset of the study that one of the most critical components of a good preventive or recuperative food-assisted MCHN program was an effective behavior change and communication (BCC) strategy. The research and development process involved in helping WV-Haiti design and plan the implementation of a new BCC strategy for its program is described in this chapter. The design of the food assistance component for both program approaches is described in the next section, followed by the research and development process used to design the BCC strategy. Implementation protocols were already in place for the preventive health-care components of the program at the Rally Posts (i.e., immunization, vitamin A supplementation, and deworming) and for pre- and postnatal consultations, and these were not modified (see Chapter 3). These activities are therefore not discussed in this chapter, as is the case for the home visit protocols. Thus, the chapter focuses on the screening of beneficiaries for food assistance (at the RPs), the food distribution protocols (implemented at FDPs), and the BCC strategy mostly implemented at the MCs. The chapter concludes with a description of the similarities and differences between the preventive and the recuperative program approaches being compared in this study. 4.2 The Food Assistance Component The protocol for food assistance for the recuperative program remained as originally designed by WV-Haiti. For this approach, malnourished children (identified during growth monitoring)10 are enrolled in the program between 6-59 months of age for 9 months. According to WV-Haiti management, the rationale for providing food supplements to malnourished children for 9 months is based on programmatic experience, which suggests that 9 months is sufficient for most children to recover from malnutrition. Note, however, that to our knowledge, this assumption has not been verified by evaluation research. Furthermore, while anecdotal evidence suggests that the duration of participation for children in Title II programs is between 6 and 12 months, there are no systematic reviews available of the average duration of enrollment of children in recuperative programs implemented by other Cooperating Sponsors. Design of the food assistance component for the preventive program was based on empirical research showing the benefits of providing nutrition interventions early in life in order to prevent malnutrition. Discussions were held with program managers at WV to review the 9 Cornelia Loechl led the writing for this chapter. 10 Malnourished children are defined as M2 and M3 according to the Gomez classification. In this classification, normal (N) corresponds to ≥ 90% of the median of the weight-for-age CDC/NCHS/WHO standards; mild malnutrition (Grade M1) to 75% to < 90%; moderate malnutrition (Grade M2) to 60% to < 75%; and severe malnutrition (Grade M3) to ≤ 60% (Cogill 2003). 36 scientific rationale for a preventive approach and discuss how the benefits of food supplements could be maximized, while remaining within the resource capacities of the program. It was decided that children would be enrolled in the preventive program between 6-18 months of age. This age range was selected because research suggests that the period between 6 and 24 months is the age of maximum response to nutrition interventions (see Chapter 2, section 2.4.1.1.). Beneficiaries would continue to receive food supplements up to the age of 24 months, thus ensuring that even those who enter the program as late as at 18 months of age would receive 6 months of supplementation. Food assistance for pregnant and lactating women also remained as originally designed by WV, i.e., pregnant and lactating women are eligible to receive food assistance for 6 months during pregnancy and for the first 6 months of lactation. Food assistance to pregnant and lactating women was to be exactly the same in preventive and recuperative program communities. Information regarding ration size and composition - for direct and indirect rations - is provided in Chapter 3. 4.3 The BCC Component The rationale for implementing a BCC strategy in the context of a food-assisted MCHN program is that, in addition to providing food for children, it is also important to ensure that mothers are well informed and able to adopt optimal child-feeding and caregiving practices. The key aspects of care and feeding that are particularly important to prevent malnutrition during the vulnerable period of 0-23 months of age are breastfeeding, appropriate complementary feeding practices, and other preventive and curative health-related practices like good hygiene, timely immunization, appropriate home health care, and care-seeking during illness. Mothers also need to be educated about how to address childhood malnutrition and how to detect signs of severe malnutrition or childhood illness. These topics were the focus of the BCC strategy developed for WV-Haiti’s preventive and recuperative programs. The strategy was designed in two phases: a research phase and a development phase (see Figure 4.1). In the following section, details of each step of both phases are presented. The data from the research phase were used to assess the adequacy of current infant and young child-feeding practices and to identify constraints and facilitating factors to adopting optimal feeding practices. Following this, programmatic actions to improve nonoptimal practices and sustain optimal practices were developed consultatively with WV-Haiti staff. This was achieved through the different steps of the program development phase. 4.3.1 Research phase 4.3.1.1 Review of existing communication materials The first review of program communication materials commonly used by PVOs in Haiti was initiated in November-December 2001 and has been reported previously (Menon et al. 2001). 37 The purpose was to gather information on the existing nutrition and health education models currently used in Haiti. Two guides used by different PVOs in Haiti were reviewed. It was found that both guides covered breastfeeding practices and practices related to the prevention and treatment of childhood illnesses quite extensively. Messages related to complementary feeding practices, however, were minimal and psychosocial care was generally absent from both education packages. Messages related to complementary feeding focused mainly on nutrient density and dietary diversity and did not address feeding frequency or portion size. Figure 4.1 Steps of the design process of the BCC component11 Research Phase Development Phase Review of existing communication materials (Nov-Dec 2001) Identification of priority programmatic actions for BCC (Oct 2002) Preliminary qualitative study (Jan 2002) Development of BCC strategy (Oct 2002- Apr 2003) Formative research study (May-Jul 2002) Development of BCC materials and training plans (Oct 2002- Feb 2003) Training of WV staff (Mar 2003) Development of implementation plan (Apr 2003) Implementation (May 2003) 4.3.1.2 Preliminary qualitative study As a first step in the formative research process, a rapid qualitative study was conducted in January 2002 in the Central Plateau to gather information on general patterns of infant and child-feeding practices. The data were used for two purposes: (1) to design the formative research required for the development of the BCC strategy (Menon et al. 2002a) and (2) to guide the development of the baseline survey for the evaluation (see Chapter 2). The results suggested that the current infant and child-feeding practice departed substantially from international feeding recommendations, especially with regard to the recommendations to exclusively breastfeed infants up to 6 months of age, and to complement breast milk with frequent feeding of energy- and micronutrient-dense complementary foods after 6 months of age. 11 All BCC communication materials and information about the development of the program are available at the following FANTA website: www fantaproject.org. 38 4.3.1.3 Formative research study A more extensive formative research study was undertaken between May and August 2002. The objectives of the study were (1) to gather in-depth information on current infant feeding practices and their determinants; (2) to identify constraints and factors that may facilitate adoption of optimal feeding practices; (3) to conduct recipe trials to develop improved complementary foods based on the use of donated, fortified food commodities and other nutrient￾rich foods locally available; and (4) to understand current WV program activities in the Central Plateau of Haiti and identify potential program delivery points that could be used for the behavior change and communication (BCC strategy) (Menon et al. 2002b). Several data collection techniques were used, including individual and group interviews with mothers of young infants, grandmothers, fathers, and WV program staff. Participatory group recipe trials with groups of local women were conducted to develop recipes for enriched complementary foods and to discuss their feasibility, acceptability, and affordability under real￾life conditions in the program areas. Finally, observations of WV’s program activities were conducted in the Central Plateau area as well as on the island of La Gonâve to understand the implementation of different program components and to explore the feasibility of enhancing ongoing educational activities. Key results of the formative research are summarized below. Infant and young child-feeding practices: The formative research provided information that allowed us to characterize typical infant and young child-feeding practices in rural Haiti and to understand the rationale for these practices. Specific factors likely to either facilitate or constrain adoption of optimal practices were also identified for each specific dimension of child￾feeding practices studied. Development of enriched complementary foods: The recipe trials confirmed that traditional complementary foods are low in micronutrient-density, although they are generally of adequate energy density. The process also demonstrated that it was feasible for the recipe trial participants to develop a number of improved recipes using traditional preparation methods, fortified donated commodities, and adding locally available nutrient-dense foods such as fish, eggs, beans, and vitamin A-rich vegetables and fruit. Exploring the potential of different program delivery points for the behavior change and communication ( BCC) strategy: Observations at the Rally Posts (where growth monitoring, immunization, and health education activities are held) indicated that while the Rally Posts may be a promising entry point for the BCC program, some aspects of program implementation, such as the timing of the education sessions, the communication techniques of staff, and the allocation of time for communication and counseling, could be strengthened to improve their potential for effective communication with participants. The food distribution points (where food commodities are distributed) were identified as the least promising delivery point for the BCC intervention because of their crowded, busy, and distracting environment. The Mothers’ Clubs (group meetings held in the communities and used primarily for discussions on health education topics) were seen to be the best forum for group communication and discussions, and thus a promising “main” venue for the BCC program. However, it would be important to strengthen the content and teaching and communication approaches used to foster more effective learning and behavior change communication. 39 4.3.2 Development phase 4.3.2.1 Identification of priority programmatic actions for the BCC component Priority programmatic actions for the BCC component were identified through discussions with WV staff at all levels. These discussions were held through a series of workshops involving decisionmakers and program staff within WV-Haiti, as well as staff from the U.S. Agency for International Development (USAID) Mission in Haiti and other private voluntary organizations (PVOs) working in the area of child nutrition in Haiti. The workshop discussions focused on prioritizing practices to be promoted through the BCC component, as well as on reviewing the design and the technical and operational aspects of the BCC strategy. The selection of priority actions for the BCC component was facilitated greatly by the use of a decision tool that summarized and organized the formative research results in the form of a matrix (presented in Annex 4.1). The matrix compares current practices in the program areas to best practices, as summarized in the Guiding Principles (PAHO/WHO 2003). The matrix also summarizes results of the formative research regarding facilitating factors and constraints that could influence the ability of program participants to adopt recommended practices. An example of the decision tool is provided in Box 4.1. Box 4.1 Example from decision matrix to organize formative research results on exclusive breastfeeding Goal Practices to promote Practices in Haiti Facilitators Constraints Exclusive breastfeeding (EBF) for 6 months - Avoidance of pre￾and post-lacteal feeds - Using expressed breast milk when mothers leave home - Pre-lacteals and post-lacteal liquids and gruels widely used - Breast milk expression not widely practiced - Concept of EBF already introduced - Positive role models exist - No cultural barriers to use expressed breast milk - Gruels given when mothers leave home - Lack of training and support on breast milk expression, use and conservation of expressed breast milk As a next step to the results matrix, we developed a “program-planning matrix” that examined the programmatic actions that would be necessary to address each specific constraint or facilitating factor (see Annex 4.2). The consideration of feasible programmatic actions (presented in the second column) was based primarily on the existing WV program infrastructure and capacity (human, financial, technical). However, future needs and other supporting programs (particularly to support the BCC component) were also considered and these are presented in the third column of Annex 4.2. The program planning discussions held with WV￾Haiti focused on identifying modifiable behaviors, constraints, and facilitating factors that could be addressed within the current programmatic options available to them. An example of how constraints to exclusive breastfeeding were addressed is presented in Box 4.2. 40 Box 4.2 Example from program-planning matrix to address constraints to exclusive breastfeeding Constraints to exclusive breastfeeding Program options within current structure and delivery system Gruels given because mothers need to leave home for work or other activities Use of expressed breast milk is rare; milk expression unknown in some areas ⇒ BCC program: ensuring adequate training in the use and appropriate storage of expressed breast milk 4.3.2.2 Development of the BCC strategy Following the formative research process and the discussion of the results with WV￾Haiti, the BCC strategy was developed. This was done using a “BCC strategy planning matrix,” which outlines the various aspects that need to be addressed in order to ensure that the behavioral change objectives defined through the program planning discussions are achieved. The matrix is presented in Annex 4.3. It identifies, for each age-specific set of practices to promote, (1) who needs to be targeted in order to ensure that the desired feeding practice is achieved, (2) when the communication related to a specific practice has to reach the identified audience in order to maximize its effectiveness, (3) where the communication has to take place in order to reach the desired audience at the right time, (4) how specific practices should be promoted at the different program venues and for different program audiences, and (5) what is needed to ensure that the communication strategies identified for each type of practice, program venue, and participant are implemented appropriately. An example of the matrix regarding expression of breast milk is presented in Box 4.3. Box 4.3 Example of BCC strategy planning matrix on expression of breast milk Practice to promote: Use expressed breast milk as needed (avoid other liquids and foods) Who will messages be targeted to? Lactating mothers/fathers/grandmothers When will messages be delivered? First 1-2 months of lactation Where will communication be delivered? Postnatal consultations and Mothers’ Clubs How will communication be delivered? - Individual counseling at postnatal consultations - Group discussions and problem solving related to expression of breast milk at Mothers’ Clubs (including demonstration and practice) What is needed to help with communication? - Training of health staff in communication methods and content of practices to encourage - Provision of resource materials for 4.3.2.3 Development of BCC materials and training plans Following the identification of the BCC strategies to be used at the different program venues, program communication materials were developed for use in the BCC component. Since the Mothers’ Clubs were identified as the most promising main venue for the BCC, the 41 material development process focused on materials to be used at the Mothers’ Clubs. Further, WV was already in the process of developing other simple materials for use at the Rally Posts. The materials developed for use in the Mothers’ Clubs focus mainly on infant and young child-feeding practices. These practices were considered the most important to address in a program whose goal was to prevent malnutrition among children 0-24 months old. Other WV materials were available that covered general aspects of health care and care during illness for infants and children. Details about the communication materials have been reported previously (Loechl et al. 2003b). The development of the BCC materials consisted of five steps: A) A second review of program communication materials used in Haiti: In conjunction with the formative research process, a review of two additional sets of BCC materials related to infant feeding in Haiti was conducted to identify newly developed local materials that could potentially be adapted for use in the WV program. The two modules on breastfeeding and young child feeding developed by Freedom from Hunger (FFH) in 2001 and used in conjunction with the FFH Credit for Education program in Haiti were identified as the most appropriate for adaptation and permission was obtained to use them. They addressed breastfeeding practices as well as complementary feeding practices, and included messages related to feeding frequency, responsive feeding, portion size, psychosocial care, and good hygiene practices during food handling. Each module consisted of several learning sessions. These learning sessions were highly detailed and comprehensive. Each session included a set of explicit instructions to the fieldworker, accompanied by activities for them to carry out with the group of participants in order to achieve the objectives of the learning session. The materials were intended for use with a communication strategy that incorporated the principles of adult learning as well as of trials of improved practices. B) Pretesting and adaptation of newly developed messages: A first step in the adaptation of the FFH materials was the pretesting of new messages developed based on the formative research. The pretest process consisted of four individual and two focus group discussions in the areas where the BCC component was to be implemented. For each item, the interviews gathered information on participant comprehension, the believability of the message, the perceived importance and benefit of the actions implied in the message, and whether the participants would consider changing their practice after hearing the message. After modification of the messages according to the results of the pretest, they were incorporated into the communication materials. An example of the adaptation of the message on expression of breast milk is presented in Box 4.4. 42 Box 4.4 Example of message adaptation for expression of breast milk New message tested Modified message included in communication materials If the mother has to leave home, she can express breast milk so that another person can give this to the child with a little spoon when she is away. If the mother has to leave home, she can express breast milk in a cup so that another person can give this to the child with a little spoon when she is away. [To avoid that they express it in a bottle] C) Adaptation of BCC sessions based on the formative research and WV program context: Based on the results of the pretest exercise and the current infant feeding guidelines, the content of FFH modules on breastfeeding and young child feeding was adapted in collaboration with a local firm in Haiti. The adaptation also took into account findings from the formative research. The materials were also adapted to the programmatic context of WV as this differed considerably from the context of the Credit for Education program that FFH had used them in. In the preventive program the order of the topics was changed to be age-specific and to take into consideration the notion that critical pieces of information should reach mothers at what is likely to be the most appropriate learning moment for each set of practice. In addition, the length of the learning sessions was extended, while still maintaining the same structure as with the FFH learning sessions. An example of a learning session that includes instructions on the expression of breast milk is presented in Box 4.5. Box 4.5 Example of learning session on expression of breast milk Title: Always promote breastfeeding Overall objective: To analyze constraints to exclusive breastfeeding and identify solutions Activity Objective Methods used to achieve objective 1 To share experiences with exclusive breastfeeding and find solutions to problems encountered ¾ Paired discussion among participants about breastfeeding experience ¾ Question/answer session on recommendation of exclusive breastfeeding for first 6 months ¾ Review of proper child position during breastfeeding, feeding frequency, and care for breasts and nipples by participants 2 To learn how to express breast milk and how to conserve it ¾ Demonstration of expression of breast milk by experienced mother ¾ Discussions of how to conserve expressed breast milk and how to give it to the child 3 To drink water while breastfeeding to avoid fatigue ¾ Story about drinking water while breastfeeding and further explanations by health promoter D) Testing and adaptation of visual aids for BCC: The instructions for conducting a learning session are accompanied by visual materials. For several of the learning sessions, a large-format, laminated chart on child growth, development, and feeding is 43 used to facilitate discussion of infant and child-feeding recommendations in relation to the physical development of a child. In addition, a series of images is used to support verbal presentations of the health staff. The images present scenes to illustrate stories and specific feeding recommendations. These visual materials developed by FFH were adapted in collaboration with a local firm to ensure that the technical information was up-to-date and relevant, and also to ensure that the materials would be culturally relevant and accepted. For example, visual instructions showing manual breast milk extraction techniques were adapted for use in the Mothers’ Clubs. E) Adaptation of BCC training guides: The FFH training materials included manuals and resource materials for training of trainers as well as for training of field staff. For the WV staff, the trainers’ guide and toolkit for the modules on breastfeeding and young child feeding were adapted to reflect the changes in the content of the learning sessions. In addition, the schedule of learning sessions was created specifically to address the needs of the WV program. The manual on adult learning principles and practices, which is used along with the training materials on breastfeeding and young child feeding, needed only slight adaptations in terminology. 4.3.2.4 Training of WV-Haiti staff The formative research suggested that although the WV health promoters and assistant health promoters were highly motivated to transfer skills and knowledge related to child health to the participants in the Mothers’ Clubs, they were constrained by a lack of training in appropriate methods of adult education. Therefore, WV program staff was trained in the use of these communication methods that are grounded in principles of adult learning. Using this approach to teaching and learning, program staff learned how to create a learning environment where people feel safe and respected, how to facilitate group discussions, offer open-ended questions, create dialogue, animate role plays, and build on the ideas of the participants. In addition, the staff was trained in the technical issues related to breastfeeding and young child feeding. The training was done in two steps. First, the supervisory-level staff was trained in a “training of trainers” session, followed by the training of field staff in WV. Details of the training are described elsewhere (Loechl et al. 2003b). Training of trainers was done in two stages. In the first stage, all MCHN staff above the level of health promoters and assistant health promoters (i.e., the MCHN National Coordinator, Regional Coordinators, and field supervisory staff) was trained in the use of adult learning principles for communication. In the second step, the same staff was trained over a period of six days on the use of the adapted communication materials on breastfeeding and young child feeding. Training of field staff (i.e., health promoters and assistant health promoters) was conducted through a six-day workshop. The training was conducted by a group of five WV supervisors who had previously been trained in the Training for Trainers workshops. The health promoters and assistant health promoters were trained in the use of the breastfeeding and young 44 child learning sessions, and the use of the technical content of the sessions was linked to the principles and practices of adult education. 4.3.2.5 Development of the implementation plan for BCC intervention The implementation plan for the BCC activities at different program points was developed and finalized through a round of discussions held with WV program staff. A first outline of the implementation plan was developed together with the national MCHN coordinator for WV. This plan was further complemented through several meetings with the regional MCHN coordinator and the field supervisory staff in Hinche. 4.4 The Implementation Plan for the Preventive and Recuperative Program Approaches The WV MCHN program (both preventive and recuperative) offers services at five major points of contact between program staff and participants. These are described in Chapter 3. Following the development of the preventive approach and of the BCC strategy, changes were made to only three service delivery points: (1) the Rally Posts (RPs); (2) the Mothers’ Clubs (MCs); and (3) the Food Distribution Points (FDPs). Table 4.1 presents a summary of the activities conducted at these three service delivery points and compares the intervention packages offered in the preventive and recuperative approaches at these delivery points. Shaded areas highlight where the two program approaches differ. Services offered at the pre- and postnatal consultations and at home visits are not included in the table because they are described in Chapter 3 and are exactly the same for both program approaches. 4.4.1 At Rally Posts (RPs) The services to be provided at the RPs are identical for the recuperative and preventive programs, with the only difference being the criteria for selection of food assistance beneficiaries. In the recuperative program group, children are selected based on their nutritional status, whereas in the preventive program group, the selection of beneficiaries is based on age. In the recuperative program, 6-59-month-old children are eligible for food assistance if they are malnourished (M2 and M312) based on the weighing results from growth monitoring activities. There is no defined upper age limit for admittance, although there is an upper age limit for eligibility for program services. For instance, a child can be admitted at 58 months, but s/he would have to exit the program one month later when s/he reaches the age of 59 months. In the preventive program, children are eligible for food assistance based on their age: all children between 6 and 18 months of age are eligible to enter and remain in the program until they reach 24 months of age. Thus, in this approach, the upper age limit for admitting children into the preventive program is 18 months, to ensure that all children in the program receive food assistance and other services for at least six months (up to 24 months of age). In the preventive 12 Malnourished children are defined as M2 and M3 according to the Gomez classification. In this classification, normal (N) corresponds to ≥ 90% of the median of the weight-for-age CDC/NCHS/WHO standards; mild malnutrition (Grade M1) to 75% to < 90%; moderate malnutrition (Grade M2) to 60% to < 75%; and severe malnutrition (Grade M3) to ≤ 60% (Cogill 2003). 45 Table 4.1 Package of interventions provided in the recuperative and preventive approaches Recuperative Approach Preventive Approach At the Rally Post: At the Rally Post: Preventive health and nutrition activities: - Vitamin A supplementation (6-59 mo old children and women in first mo. postpartum) - Immunization - Anti-helminth (12-59 mo children) - Family planning - Iron supplementation (pregnant and lactating women) Preventive health and nutrition activities: - Vitamin A supplementation (6-59 mo old children and women in first mo. postpartum) - Immunization - Anti-helminth (12-59 mo children) - Family planning - Iron supplementation (pregnant and lactating women) - Identification of children 6-23 mo – admission in food distribution program Growth monitoring: - Weighing - Screening of malnourished (M2,M3; 6-59 mo) IF MALNOURISHED (and 6-59 mo) - Brief individual counseling - Admission in food distribution program Growth monitoring: - Weighing - Screening of malnourished (M3) (24-59 mo) IF MALNOURISHED (and 24-59 mo) - Brief individual counseling - Admission in food distribution program Record keeping: health promoters maintain records of nutrition/health status of all children measured Record keeping: same as recuperative At the Mothers’ Clubs At the Mothers’ Clubs Behavior change communication sessions with small groups of participants, organized: - For pregnant and breastfeeding (BF) women: specific to stage of pregnancy & lactation - For mothers of malnourished children: fixed set of sessions Maximum continuous attendance of mothers at the MCs: 21 months Topic of BCC sessions - Pregnant women: Diet, dangerous signs, preparation of child delivery, BF - Lactating women: BF, complementary feeding (CF), child development - Malnourished child (6-59 mo): causes of malnutrition and recuperation of malnourished children, CF, child development, BF, immunization, diarrhea and hygiene in food preparation, handling and storage; other topics: HIV/AIDS, family planning Behavior change communication sessions with small groups of participants organized: - For pregnant and lactating women: same as recuperative - For mothers of 6-23 months old children: age-specific set of sessions Maximum continuous attendance of mothers at the MCs: 30 months Topics of BCC sessions: - Pregnant women: Same as recuperative - Lactating women: Same as recuperative - Malnourished child (24-59 mo): Same as malnourished child from recuperative group - Child 6-23 months: CF, child development, hygiene in food preparation, handling and storage, causes of malnutrition; other topics: diarrhea, immunization, hygiene, use of moringa olifeira, HIV/AIDS, family planning, home gardening, vitamin A, parasites, water treatment, and ARI infections (continued) 46 Recuperative Approach Preventive Approach At the Food Distribution Post At the Food Distribution Post Verification of eligibility (malnutrition) Verification of eligibility (age) Food distribution: MALNOURISHED CHILDREN (M2, M3; 6-59 MO): for up to 9 months duration Monthly ration: 1 individual (direct) and 1 family (indirect) Food distribution MALNOURISHED CHILDREN (M3; 24-59 MO): for up to 9 months duration ALL CHILDREN 6-23 MONTHS: up to the age of 23 months (maximum 18 months in program) Monthly ration: 1 individual (direct) and 1 family (indirect) Note: Shaded areas correspond to aspects that were different between the two program approaches. approach, severely malnourished children between 24 and 59 months of age are also eligible to participate in the preventive program. As in the recuperative approach, these children (classified as M3 according to the Gomez classification) are identified through the regular growth monitoring activities done at the RPs. The services provided for the severely malnourished children in preventive program communities are also similar to those received by children in the recuperative approach. They include (1) distribution of food rations for 9 months, (2) two meetings for the mothers where issues related to malnutrition and recuperation are discussed, and (3) home visits by health staff during the first weeks after identification. In both the preventive and the recuperative programs, pregnant and lactating mothers (until their infant reaches 6 months of age) are also eligible for food assistance. For mothers of children 6-23 months old in the preventive approach and mothers of malnourished children in the recuperative approach, monthly attendance at the RPs and at MCs is mandatory to be eligible to receive the monthly food rations offered by the program. Pregnant and lactating women are also required to participate in MCs and pre- and postnatal consultations to be eligible for the monthly food rations. 4.4.2 At the Mothers’ Clubs (MCs) The formative research identified the MCs as the ideal setting for effective BCC activities; the reasons being that MCs are usually located close to the mothers’ homes and include only a small group of mothers, resulting in minimal distractions (especially compared to the RPs). Thus, WV decided to use the MCs as the primary venue for its BCC strategy. Organization of MCs and timing of sessions: Results of the formative research emphasized the need to reorganize the MCs to include peer groups of mothers of a particular physiological state and/or child age. Based on the BCC implementation plan, separate MCs were organized for pregnant mothers, lactating mothers, mothers of children 6-23 month old (in preventive areas), and mothers of malnourished children (in recuperative areas). For the pregnant and lactating MCs from both the preventive and the recuperative groups, and for the MCs with mothers of 6-23-month-old children in the preventive group, the schedule of sessions for the MCs was planned to be specific to the physiological status of mothers (pregnant/lactating) 47 and to the age of the child (6-23 months), and to address practices that are immediately relevant for the child’s health, development, and growth at a particular age. Duration of participation: The duration of participation by mothers at the MCs (and other BCC activities) also differs between the two program groups; mothers in the preventive program may benefit from the BCC activities for longer than mothers in the recuperative program. For example, a mother in the preventive group should start attending the MCs during pregnancy for a period of 6 months, continue to attend the same club throughout her first 6 months of lactation and subsequently, for another 18 months, when the child is between 6 and 23 months of age. Thus, mothers in the preventive approach may attend the MCs for up to 30 monthly sessions, without interruption. In the recuperative approach, this continuity of participation is not the norm. In order to be eligible for the MCs, mothers from this group must be either pregnant or lactating, and/or have a malnourished child 6-59 months of age. Thus, the length of participation of mothers in the MCs depends on their physiological status and on their child’s nutritional status. Mothers are eligible to attend the MCs for 6 months during pregnancy, 6 months during lactation (the first 6 months postpartum), and up to 9 additional months if they have a malnourished child at any point in time between 6 and 59 months of age.13 Thus, the maximum possible non-interrupted time mothers in this program group can attend the MCs is 21 months, and this will happen only if they start attending during pregnancy, continue through lactation, and have a malnourished child when they reach 6 months postpartum (the malnourished child can be the 6-month-old one or an older child, as long as s/he is less than 60 months of age). Content of MC learning sessions: The modules on breastfeeding and young child feeding were adapted from materials developed by FFH (see previous section). The modules are organized into learning sessions, which are designed to be completed in about one hour. Each of these sessions covers specific key practices using a variety of communication methods. All the sessions consist of a set of instructions to the health promoters/assistant health promoters, accompanied by activities for them to carry out with the group of participants. Other communication materials are used for topics other than infant and child feeding, such as an album of images with key messages related to other aspects of maternal and child health. The MC learning sessions covered a range of topics and skills related to infant and young child feeding (Box 4.6). The material covered in the learning sessions closely followed current international “Guiding Principles” for infant and young child feeding (PAHO/WHO 2003; WHO 2005). They were also adapted to address key local constraints to following the Guiding Principles. Families who participate in WV programs are encouraged to care for their children by accessing RP preventive health-care services such as immunizations, by seeking help from the health agent or other medical professional when their child is sick, and by using the ORS that is distributed at RPs. These caregiving and care-seeking practices are promoted within the larger MCHN program, and thus in both preventive and recuperative areas. 13 Children are eligible to re-enter the recuperative program if they are still undernourished one year after having exited the program. 48 Box 4.6 Mothers’ Club learning sessions Prenatal learning sessions (last trimester) 1. The importance of good breastfeeding practices 2. Good breastfeeding practices Postpartum learning sessions (0-6 mo) 3. Always promote breastfeeding 4. How to breastfeed better 5. Lactational amenorrhea method 6. Start giving other rich foods to complement breast milk when children are 6 months old 7. Learning to eat: how to breastfeed and feed children less than 12 months old 8. Preparing nutritious foods for children Sessions for mothers of older children (6-23 months in preventive areas; malnourished children 6-59 months in recuperative areas) 9. Helping children eat well in health and in sickness 10. Variety of food combinations appropriate for children 6-12 months 11. Protecting your food – protecting your children 12. Feeding children beyond 12 momths of age 13. What we can do to combat malnutrition Timing and schedule of MC sessions: The sessions were designed to be timely and to teach women about key practices before the practices should be initiated. For example, two sessions on breastfeeding were meant to be covered in prenatal MCs, during the last trimester of pregnancy. Similarly, information on appropriate complementary feeding was meant to be introduced in MCs when the child was 4-6 mo of age. At around 6 mo, an important session allowed women to practice preparation of recommended gruels/recipes (Session 8 in Box); this session was covered twice in preventive areas. The first 8 of the 13 learning sessions were used identically in MCs in both program areas. The last five sessions also covered similar material, but in recuperative program areas, mothers joining MCs had children of varying ages and topics were covered in a different order, and to meet the needs related to feeding somewhat older children. The full schedule of learning sessions at the MCs is presented in Annex 4.4 (A through C). For the MCs with mothers of malnourished children in the recuperative group, the schedule consists of a fixed set of nine learning sessions, which do not take the age of the child into consideration because the purpose is to discuss the recuperation of malnourished children, irrespective of their age (Annex 4.4 C). Facilitation of learning sessions: The learning sessions were facilitated by health staff trained in adult education principles and techniques, as well as in the content of the learning sessions. Throughout the learning sessions, a variety of interactive techniques were used, including demonstrations and role-playing, and discussion and practice were encouraged. Recommended practices were discussed and local practices compared to recommendations. 49 Constraints to following recommendations were identified and discussed. Chapter 5 discusses some of the key elements of the facilitation when the operations at the MCs are described. 4.4.3 Food Distribution Points (FDP) Activities at the FDPs are identical for the two program groups and were intended to be implemented as described in Chapter 3. The only difference is the duration of participation, which is a maximum of 9 months for children in the recuperative communities and 18 months for children in the preventive communities. 4.5 Conclusions This chapter presented an overview of the process used to develop the preventive and recuperative program approaches, with a focus on the BCC strategy. In the next chapter we examine the quality and fidelity of implementation of these approaches, which was assessed through two rounds of operations research studies. Notably, we discuss how the operations research results were used consultatively with WV-Haiti to improve program implementation. The next chapter thus follows the same thread initiated in this one, and illustrates how action￾oriented and program-linked research can be used to improve the design and implementation of programs. 50 51 5. DELIVERING IT WELL: THE OPERATIONS RESEARCH PROCESS14 5.1 Introduction This chapter discusses the operations research (OR) process used in the present study, which was outlined in Chapter 2. In this evaluation, the OR process was used first to assess the quality of implementation; the information generated by the research was then used to strengthen program processes and implementation. OR was also used to assess differences in program implementation between the two program approaches, and by doing so, to identify critical implementation issues (if any) that might lead to a differential impact of the two program approaches on child outcomes. In presenting the methods and outcomes of the OR process, this chapter demonstrates how program-oriented research can be used to benefit program implementation and impact evaluations. It also highlights the successful consultation process used in this study, which ensured that research findings were effectively communicated to program implementers and was used for action. The OR process used in this study thus illustrates how the translation of knowledge into effective action is enhanced by joint collaboration and good communication between researchers and program implementers. 5.1.1 Integration of operations research into implementation and evaluation processes OR typically focuses on program processes and implementation issues, and is often referred to as “process evaluation.” In this chapter, we use the term “operations research” as synonymous to process evaluation since we focus primarily on the process of program delivery. While the usefulness of OR for program improvement is well-established in the family planning and reproductive health literature, there are relatively few documented examples from nutrition programs in the published literature. There are even fewer documented examples of the integration of OR with impact evaluation. Yet, in addition to facilitating program improvement, OR has the potential to enhance impact evaluations in several ways. Information about processes can clarify the pathways through which impact occurs (see, e.g., Robert et al. 2006) and strengthen plausibility when evaluators attribute impact to an intervention. When programs fail to meet objectives, OR results describing implementation problems can provide insights into programmatic factors that could have constrained success. OR can also play a role in studies such as this evaluation in Haiti, where program approaches are being compared. In this case, comparing the efficiency of implementation of the two approaches is important to help interpret the impact evaluation results. If, for instance, one program proves to have a greater impact on growth than the other program, it is important to be able to rule out the possibility that the greatest impact may in fact be due to better implementation, rather than to the truly greater effectiveness of one approach compared to the other. Thus, when comparing different program approaches, OR should be an integral component of impact evaluation in order to provide a “fair test” of concept. 14 Mary Arimond led the writing for this chapter. 52 In this project, OR was conceived from the outset as an integral part of the process, with multiple purposes for its use. Figure 5.1 provides a schematic overview of the different steps involved in overall program development and evaluation, and situates two rounds of OR in this scheme. Figure 5.1 An “engaged” model of evaluation and program development Baseline assessment (2002) Formative research for program planning (2002) Program development and staff training (2003) Impact assessment (2005) Changes in program implementation (2004) Program implementation (2003) Operations Research (2003) (Round 1) Operations Research (2004) (Round 2) Consultative Program Improvement Workshop (2004) 53 Detailed results from both rounds of OR have been previously reported (Loechl et al. 2004; Menon et al. 2005); this chapter provides a summary of objectives, methods, and main results from this process. 5.2 Objectives The broad objectives of the OR were to (1) assess program implementation and quality of service delivery; (2) use the information to strengthen the quality of operations and ensure fidelity to program approaches and implementation plans; and (3) gain sufficient knowledge about implementation issues and quality of service delivery in the two program approaches to allow appropriate interpretation of impact evaluation results. Two rounds of OR were undertaken, with the first occurring in July-September 2003, shortly after full implementation of the BCC strategy, and the second approximately one year later, in June-August 2004. The first round (2003) focused on the first broad objective and more specifically aimed to: A) Assess the fidelity of implementation of the program relative to plans at all service delivery points; B) Assess the quality of delivery of the various services; and C) Explore perceptions of stakeholders (i.e., participants/beneficiaries15 and field implementers) regarding the operations and quality of services provided by the program. This first round of OR (OR1) was followed by a process of consultation with program implementers, including management and field staff. The consultative process (described in Sections 5.3.3 and 5.4.2) was directed toward the second broad objective: through this process, program operations were improved and better aligned with program design. After allowing approximately 5 months for staff to implement corrective actions/program improvements, a second round of OR was undertaken, with these specific objectives: A) To assess implementation of the corrective actions/program improvements identified during the consultative process; B) To document differences between the two program approaches in 1) Program implementation at the Rally Posts and Mothers’ Clubs; 2) Staff -level factors that could lead to differential implementation between the two approaches (e.g., job satisfaction; motivation; perceptions of supervision, etc.); 3) Beneficiary mothers’ knowledge and experiences with trial and adoption of specific infant and young child-feeding and care practices promoted in the BCC, 15 “Participants” include those who access any of the available range of services; “beneficiaries” are those who live in households receiving food assistance. 54 in order to gain understanding of constraints to adoption of these practices in the two program areas. The second round of OR (OR2) allowed continued documentation of program operations and quality of implementation, and allowed us to document improvements in operations. This round of OR also allowed us to meet the third broad objective of OR in this study, which was to enrich the impact evaluation through identifying any relevant differences between program areas in implementation, staff-level factors, or participants’/beneficiaries’ experiences. In the following sections, we define the program context and the service delivery points that were studied in the OR, as well as the stakeholder groups whose views were sought. This is followed by a section describing the research methods used in both rounds of OR, the main results from each round of OR, and a description of the consultative processes that were used to discuss the results with program staff. 5.3 Methods 5.3.1 Program context and stakeholders A first step in any OR process is to map out the program context and relevant stakeholders so as to ensure that the OR process is comprehensive in its reach. The program context for the OR includes the processes involved in the delivery of services, the norms established by the program for implementation and service delivery, and the main venues for the deliver of services in the WV program. In the OR, we focused on three of the five service delivery points used by the program (see Chapter 3 for description): the Rally Posts (RPs), the Mothers’ Clubs (MCs), and the Food Distribution Points (FDPs). The additional two program delivery points - prenatal and postnatal consultations, and home visits - were not included in the OR due to financial and logistical constraints, but also because the services provided at those venues were identical for the two program groups. The stakeholder groups from whom we sought views in the OR were participants in program activities, beneficiaries of the food assistance component of the program, and the WV administrative and field staff at different levels. WV staff included the health promoters, assistant health promoters (community program volunteers) and food monitors, who are the direct program implementers, their supervisors (the MCHN supervisors and the commodity supervisors), the regional commodity officer, and the regional and national health coordinators. 5.3.2 Data collection methods A variety of qualitative and quantitative methods were used in the OR process. They include structured observations at service delivery points, structured exit interviews with participants and beneficiaries at selected service delivery points, semi-structured interviews with participants and beneficiaries in their homes, and with health staff at program offices, and focus group discussions with various groups of WV staff. A summary of the research methods used in OR1 and OR2 is presented in Table 5.1. 55 Table 5.1 Data collection methods used in the OR process OR1 OR2 Research Method Where (with whom) Where (with whom) Structured observations - RPs (n = 19) - MCs (n = 20) - FDPs (n = 10) - RPs (n = 20) Growth monitoring and promotion at RPs: 5 children/RP (n = 100) - MCs (n = 20) Structured exit interviews - RP (participants/ beneficiaries): 2-3/RP (n = 59)a - MC (beneficiaries): 2-3/MC (n = 41) - FDP (beneficiaries): 4-5/FD (n = 45) - RP (beneficiaries only): 2-10/RP (n = 128)b Semi-structured interviews - Homes (beneficiaries) (n = 30) Semi-structured interviews - WV offices or other venue with: - Health promoters (n = 20) - Regional health coordinator - National health coordinator - Regional commodity officer Structured interviews - WV offices or other venue with: - Health Promoters (HP) (n = 19) and Assistant Health Promoters (AHP) (n = 19) Focus group discussions Various locations with: - HP (2 groups) - AHP (2 groups) - Food monitors (1 group) - MCHN supervisors (1 group) - Commodity supervisors (1 group) Various locations with: - Beneficiary mothers (7 and 5 groups of mothers from preventive and recuperative areas, respectively) a In addition to the 59 exit interviews with a random sample of respondents, exit interviews with “tracked” respondents were also conducted (n = 38). These were participants who were identified as they arrived at the RP and were followed through the different activities they went through during their visit at the RP. b Mothers were randomly selected, but there were several criteria. Mothers were selected if they had a beneficiary child, and if they had attended at least 5 MCs. Because the interview included an assessment of maternal knowledge, and because we wanted a “fair” comparison between areas, we chose to include only women with this level of exposure. Else, a random selection would have been likely to include women with higher exposure in the preventive program areas, and could have included many respondents with no exposure in the recuperative areas. The service delivery points observed during OR1 consisted of a subset of all RPs and MCs. As previously described, the overall evaluation included 20 clusters, with one health agent responsible for program services in each cluster. In OR1 we aimed to observe one RP and one MC in each agent’s area of coverage; we achieved 19, rather than 20 RPs and 20 MCs. Note that at that time, each health agent was responsible for between 2 and 5 RPs and up to 12 MCs. Therefore our sample represents a small subset of these. FDPs cover larger geographic areas, so it was possible to observe all FDPs (n = 10) in the evaluation area. For the exit interviews with mothers at RPs, MCs, and FDPs, selection was done randomly; the target and achieved sample sizes were 2-3 per RP for general exit interviews (achieved = 59); 2 per RP for tracked respondents (achieved = 38); 2-3 per MC (achieved = 41); and 4-5 per FDP (achieved = 45). OR2 gathered data at RPs and MCs as in OR1, but not at the FDPs. The same approach was used in both rounds to select 1 RP and 1 MC per health agent, but the two rounds did not necessarily include the same RPs and MCs. This was due to timing and logistical constraints, 56 which required the selection of program distribution points that held sessions during the period of the fieldwork and could be visited by the research team. A similar mix of methods as in OR1 was used, but with some differences in target groups and venues. Differences between the two rounds of OR were generally dictated by differences in the specific objectives of each round; e.g., in OR2 it was necessary to employ more open-ended methods (group discussions) with beneficiary mothers in order to understand their experiences and the constraints they encountered with trial and adoption of recommended practices. Conversely, staff interview methods in OR2 were more structured; structured interview instruments were developed based on themes emerging from staff focus groups in OR1. More detailed descriptions of methods are available in Loechl et al. 2004 (OR1) and Menon et al. 2005 (OR2). 5.3.3 Consultative workshop Findings from OR1 were presented by the IFPRI-CU team in Haiti in February 2004. A general presentation of the results to PVOs and USAID Mission staff was followed by a 1.5 day consultation of the IFPRI-CU team and the WV-Haiti management staff. Participants from WV￾Haiti included field-level supervisors as well as senior management. The main objectives of the consultation were 1) To review and discuss the findings of the OR; 2) To discuss the constraints to implementation that were identified in our assessment and to identify potential solutions to address these constraints and to strengthen program operations and quality of service delivery; and 3) To prioritize the constraints to be addressed and the potential solutions to be implemented, and to develop an “action plan” for implementing the selected actions. A matrix was used to guide the discussions and to facilitate prioritization of the constraints to be addressed by the program in the short term (see the example in Box 5.1, and the full matrix in Annex 5.1). At the outset of the workshop, the matrix listed the operational constraints identified at the different service delivery points. During the workshop, the group filled in a column listing potential solutions and one summarizing discussions of the feasibility of adopting the proposed solution(s) in the short term. The group also assessed the potential impact of addressing each problem or constraint, both on program operations and also on the overall impact of the program on its targeted beneficiaries. Summary judgments on potential impact were also recorded on the matrix, in the final column. Toward the end of the workshop, discussion centered on prioritizing actions to be taken, using three criteria: (1) the possibility of identifying a solution (corrective measure) that was within the scope of current program activities; (2) the feasibility of implementing these corrective measures, given the program’s current financial and human resources; and 57 (3) judgments concerning the potential impact of implementing these corrective measures on the overall program’s effectiveness and impact. It was considered most important to identify simple and low-cost solutions, which, if implemented, could have a major impact on operations and on effectiveness. These criteria for identifying and prioritizing actions were carefully gauged in the discussions held in Haiti. 5.4 Results 5.4.1 First round of operations research (OR1) Detailed results have been reported previously (Loechl et al. 2004); key results are presented here in order to illustrate how OR was used for the purposes identified above. 5.4.1.1 Key results from the Rally Posts (RPs) Structured observations at the RPs gathered information on operations, including the sequencing and flow of activities, the time allocated to different activities, the quality of implementation of the different activities (registration, education sessions, growth monitoring and promotion (GMP) activities, the distribution of vitamins, deworming tablets and oral rehydration salts, and immunization). A qualitative assessment of the general ambiance and quality of the venue for the RP was also done. Exit interviews with beneficiaries assessed access, services used and the perceived importance of the different services offered by the program. Semi-structured interviews and focus groups with health staff gathered information about their perceptions related to the functioning and quality of services at the RPs and their general impressions relative to their current responsibilities, the constraints on their performance, the supervision structure of the program, the coordination between the health and food Box 5.1 Example of prioritization matrix used at the consultative workshop Constraint(s) identified Potential corrective action(s) Feasibility of implementing corrective action(s) Potential impact of improving this aspect on program effectiveness and impact Children are weighed, weights are recorded, nutritional status is assessed. But information is not used to communicate with mother and give her tailored individual advice regarding her child’s growth and progress. Retrain personnel to ensure that each mother is: - Informed of child’s weight, nutritional status, progress (gained/lost weight) - Given brief message: a) encouragement (if child growing well); b) recommendation to participate in other program activities (e.g., Mothers’ Clubs) (if child not growing well); c) scheduled for home visit (if severely malnourished). Feasible; will require that supervisors motivate staff and monitor that these actions are implemented. Not feasible to require more in-depth individual counseling due to overcrowding of Rally Posts and scarcity of staff time. Mothers reported in qualitative research that they like to know their child’s weight and progress. This information can help raise awareness and motivate mothers to engage more in behavior change and communication (BCC) activities and adopt recommended practices, which in turn can increase impact on child nutrition. 58 distribution (commodity) components of the program, and to gather their suggestions relative to potential program improvements. Program operations and service delivery at the Rally Posts OR1 results suggested that RPs were generally operating according to the program’s implementation plan, and in general were providing participants with nearly the full range of planned services and activities. Areas where the quality of service delivery appeared to call for improvement were the group education sessions, and the communication between staff and mothers during the growth monitoring and promotion (GMP) activities. More specifically, the following constraints to operation and quality of service delivery were observed: ™ RPs were crowded, with a high participant-to-staff ratios, and bottlenecks (particularly at registration) and long waiting times for mothers; ™ The crowded and noisy conditions also impacted negatively on the quality of group education sessions; ™ Education sessions were generally held only once, near the beginning of the RP, and late arriving mothers could therefore not attend these sessions; ™ Education session topics generally did not follow the plan for the month; ™ Communication between staff and mothers during GMP was limited and messages given to mothers about their child’s growth were often vague and inaccurate; ™ Measurement errors when weighing children resulted in the misclassification of a substantial proportion of children into inaccurate categories (Gomez classification); and ™ There were problems with availability of supplies (e.g., vitamin supplement capsules, ORS, deworming medicine). Staff perceptions of Rally Post services and activities Staff perceived the RPs to be functioning well in general, and both health promoters and supervisors viewed the RPs as making important contributions to the families they served. Further, they noted that the RPs were of importance because they served the whole community, not just those identified as being eligible for food assistance. Staff also mentioned many of the same problems and constraints identified by the RP observations, including crowding, high participant-to-staff ratios, and lack of a wide range of supplies and equipment, including chairs and tables as well as scales, cold chain, and medical supplies. In addition, they cited late arrival by participants as a problem as the participants then would miss the (single) education session. Health promoters requested more training for assistant health promoters, so that they could better assist them with their multiple tasks. Staff also identified the lack of transport for themselves as a deeply felt problem, and one which could result in RPs starting late and longer waits for participants. 59 5.4.1.2 Key results from the Mothers’ Clubs (MCs) The OR1 observations of MCs focused on assessing the quality of teaching and facilitation at the MCs, and also on assessing whether the MCs had been constituted based on child age subgroups, as described in Chapters 3 and 4. Exit interviews with beneficiaries assessed issues related to access to the MCs and their perceptions regarding the importance of the topics discussed at the MCs. Interviews with health staff gathered information on their perceptions and opinions regarding the importance and quality of the services offered at the MCs. Program operations and service delivery at the Mothers’ Clubs Results pertaining to the quality of delivery of the learning session were very positive, as health staff demonstrated both good mastery of technical content and good use of new facilitation skills. However, the MC observations indicated that the reorganization of MCs in the preventive group was still a work in progress. More specifically, analysis of operations at the MCs revealed the following: ™ Quality of education was high, both in terms of technical content and teaching and communication skills. ™ Organization of MCs based on child age in preventive program communities did not follow the plan; this resulted in many mothers not receiving appropriate learning sessions for their child’s age. Staff and beneficiary perceptions of the Mothers’ Clubs All health staff felt that the MCs were, in general, functioning well, and they expressed a variety of positive impressions in the semi-structured interviews and focus groups. These included that the MCs provided a good forum for an exchange of ideas, and that exposure to MCs did lead to behavior change on the part of the mothers, and that mothers enjoyed the MCs. Staff also felt that the MCs had improved since the development of the new BCC strategy and subsequent staff training. Some of the problems mentioned with the MCs were logistical in nature, and included late arrivals and absenteeism, and frustrations that women did not always bring ration cards with them. Lack of transport for staff, poor venues, and lack of seating for mothers were also commonly cited; less common were inadequate staff time to prepare for the MCs, and timing/scheduling of MCs. In addition, a number of staff mentioned the challenges in eliciting participation, especially in newly formed MCs. Beneficiaries were asked for their views of the MCs during the semi-structured interviews in their own homes. Views expressed were generally very positive, with mothers reporting that they felt the MCs were important for their children’s health. Most felt that the MCs were the easiest of the MCHN services to access, as the clubs were organized close to women’s homes, and the time commitment was not excessive. 60 5.4.1.3 Key results from the Food Distribution Points (FDPs) Information gathered about the functioning of the FDPs covered the following domains: overall organization of the FDPs including timing, crowding, and waiting times; quantity of food relative to allocated rations; quality of food; and staff and beneficiary perceptions of FDP operations. Program operations and service delivery at Food Distribution Points There are many challenges inherent in commodity food distributions; these include tracking and accounting for food at all points in time, and physically transporting large quantities of commodity foods on bad roads and in all seasons. These inherent challenges require that food be distributed from a centralized point. The result of this, for beneficiaries, can be long travel times to reach the distribution point and overcrowded distribution venues. These realities were reflected in OR1 results for the FDPs: ™ The quality of distributed commodities was generally good, with almost all sampled bags/containers free of infestation or visible changes in color. ™ FDPs were crowded, with high attendance (range 117-375 beneficiaries), high beneficiary-to-staff ratios, and long waiting times (average of 4 hours). ™ FDPs often started late, primarily due to staff transport problems. ™ On average, the amounts of food distributed conformed to the plan for the FDP (kgs and number of beneficiaries), but the averages masked frequent measurement errors in allocating household food rations. Staff and beneficiary perceptions of Food Distribution Points Staff interviews and focus groups revealed that health staff and commodities staff shared some common understandings, but that views diverged on several other issues. Both health and commodities staff viewed the delays in starting food distribution to be a major problem, and both attributed delays to transport problems, including bad roads, rains, and equipment breakdowns, as well as availability of transport. In contrast to this shared perception of transport problems, numerous health staff viewed crowding at the FDPs as a problem, while this was not raised as a point of concern in interviews/focus groups with commodities staff. Similarly, several health staff raised concerns about hygiene practices during food handling at the FDPs, but these concerns were not raised in interviews with commodities staff. Views also diverged on problems related to confirmation of eligibility for individuals. Individuals could be denied rations for a number of reasons related to record keeping (beneficiary lists, entry and exit dates on a master list and on ration cards, etc.) or in cases where ration cards were forgotten. Health staff tended to view commodities staff as being overly restrictive or even punitive, whereas commodities staff felt constrained to follow strict rules regarding eligibility. Commodities supervisors expressed more flexibility regarding rules, but 61 this flexibility did not appear to have been communicated to frontline staff. All agreed that better communication as beneficiary lists were prepared could help. In general, there appeared to be a need for more dialogue and collaboration between these two arms of the program. As would be expected, beneficiaries expressed great appreciation for the food that they received, and many specifically expressed appreciation for the wheat-soy blend (WSB), because they understood that it contained vitamins that were beneficial for their children. The main problem for beneficiaries was the long-time commitment involved in attending the FDP, which they related both to the distance traveled, and the long waits at FDP. Despite the variability we observed in ration sizes, only two respondents expressed this as a problem. 5.4.2 Outcomes of consultative workshop The consultative workshop described previously led to a variety of agreements on the corrective actions to be implemented by the program at the different delivery points. A summary of these is provided below and further details are presented in the matrix in Annex 5.1. 5.4.2.1 Solutions for the Rally Posts Organization ™ Few solutions were identified to solve overcrowding in the short term because resources for hiring more staff were unavailable. Splitting RPs to decrease participant/staff ratios could not be done except in very few cases. ™ Improve training of assistant health promoters (AHPs): The management staff indicated that this measure had already been implemented and that AHPs were now being included in all field staff training. The AHPs had also recently been “promoted” and their salary was raised from 30% of the health promoters’ salary to 50%.16 ™ Revise sequence of activities at the RP; use of tickets to ensure that mothers are being attended in the order in which they arrive. Group education ™ Increase the coverage of the education sessions by offering more than one education session at the RPs. ™ Keep education sessions short; use sessions to deliver simple messages and reinforce topics taught at the MCs. 16 These changes were accompanied by changes in job titles, to recognize and formalize the difference. However, for consistency, we refer to the two levels of staff as health promoters and assistant health promoters, respectively, throughout this report. 62 ™ Enforce use of 12-month calendar of education topics to ensure that participants are exposed to the complete set of topics planned by the program (supervisors to take responsibility). Supplies ™ Program management to pursue efforts to maintain adequate supplies of vitamins, ORS, deworming tablets, and vaccines (through coordination with main providers (Ministry of Health, UNICEF, etc.)). ™ No action recommended for field staff on this issue. Growth monitoring and promotion (GMP) ™ Retrain and increase supervision of health staff conducting GMP to reduce measurement errors (and consequent misclassification of children). ™ Simplify GMP process, but ensure that at least the following is provided: all mothers should be informed about the child’s weight, nutritional status, progress since last weighing; and provided with brief individual advice. The advice should consist of praising and encouraging the mother if the child is growing well; if the child is not growing well, the mother should be advised to follow up with other program activities and adopt recommended practices discussed at the MCs (e.g., exclusive breastfeeding, use of enriched porridges, etc.). 5.4.2.2 Solutions for the Mothers’ Clubs Organization of MCs by child age subgroups ™ Pursue additional efforts to ensure the organization of MCs follows implementation plan, i.e., separate MCs for mothers of children aged 6-11 months and those of children aged 12-23 months. ™ Supervisors to revise rationale for the new organization of MCs with local staff and provide necessary support for implementation. Quality of teaching and facilitation skills ™ Ensure effective supervision and feedback to maintain a high quality of education, achieve a greater consistency of performance, and maintain staff enthusiasm and motivation, and interest of beneficiaries/participants. 5.4.2.3 Solutions for the Food Distribution Points (FDPs) Logistical problems ™ Continue ongoing efforts to address constraints identified in OR1. 63 ™ Pursue efforts to improve transport and logistical problems as possible (it was well recognized that several of the transport and related logistical problems could not be fully overcome in the Central Plateau, where road conditions are bad and availability of fuel is a major constraint). Errors in amounts of food received ™ Strengthen the existing supervision system to reduce errors in the measurement of food rations (e.g., supervisors could systematically verify rations provided to a subsample of beneficiaries). Communications difficulties between commodity and health staff ™ Strengthen the supervision to ensure good communication among all staff and with the beneficiaries (treat cases of miscommunication individually, as needed). ™ Organize joint preparation (between health and commodity staff) of beneficiary lists, in advance of each FDP. 5.4.2.4 Follow-up steps from the consultative workshop The February 2004 consultative workshop was a key step in the process of program improvement. All staff present at the meeting participated very actively in the dialogue, and controversies were not avoided, but were skillfully managed by senior staff. The process used to identify and prioritize feasible solutions was effective, and this same process also appeared to ensure among staff a sense of ownership of solutions and responsibility for taking action. Notably, discussions also revealed that management had already initiated actions to resolve several of the problems identified in OR1, through such activities as joint planning between health and commodities teams. It was noted that the action plan described above would rely heavily on a strong and effective supervision system. At the time of the consultation, the management staff also indicated that they had already been taking steps to strengthen the supervision of the program at all levels, and the health team presented their newly developed supervision plan for the region. 5.4.3 Second round of operations research (OR2) The second round of operations research (OR2) was carried out in mid-2004, approximately 5 months after the consultative workshop. As noted earlier in this chapter, this round was undertaken to (1) assess implementation of the corrective actions and program improvements identified during the consultative process and (2) to document differences between the program approaches in program implementation at the RPs and MCs, in staff-level factors that could lead to differential implementation, and in beneficiary mothers’ knowledge and experiences with trial and adoption of specific infant and young child-feeding and care practices promoted in the BCC. The methods used in OR2 are summarized in Table 5.1 (page 55). For the second round of OR, the decision was made not to return to FDPs for more observations. A number of the problems identified at the FDPs were deemed not amenable to 64 change, while others were to be addressed through improvements in communications. Direct observation at FDPs in 2004 was not considered likely to capture changes in communication. This section presents the results of findings related to program implementation and operation, including improvements since OR1 and differences between program groups at OR2. In considering the differences at OR2, we also present differences between the groups in staff knowledge, motivation, and perceptions of supervision. 5.4.3.1 Key results from the Rally Posts Organization: Flow of activities, staffing, and crowding OR2 observations confirmed that the new sequence of activities proposed during the February 2004 consultation had been successfully implemented in most RPs. Staff nearly universally (97%) preferred the new sequence and all agreed that RPs were more “smooth and calm.” However, registration was still identified by observers as the most important bottleneck in almost all RPs. Regarding the crowding and long waits identified in OR1, management agreed to continue to consider splitting RPs when possible, and to consider training more assistant health promoters. Findings from the 10 RPs observed both in 2003 (OR1) and 2004 (OR2)17 show that while the average number of staff remained the same (4 staff per RP), the variability and number of participant-to-all staff ratios decreased (maximum dropped from 59:1 to 33:1). In sum, it appeared that while overall staffing remained the same, staffing levels had become more consistent for those RPs observed in both years. Comparisons between program groups in 2004 for the 20 RPs observed (10 in preventive and 10 in recuperative program areas) show that average participation and participation-to-health promoter ratio were higher in preventive compared to recuperative RPs. The age distribution among children attending the RPs was also different between program groups. Among the randomly selected children for whom GMP was observed, 94% were between 6-23 months in the preventive RPs as compared to 50% in the recuperative RPs. Only 2% of the children observed in the preventive RPs were 24 months or older as compared to 46% in recuperative areas. The difference could have had implications for coverage of preventive health services (supplementation, deworming, ORS, and vaccinations) and is assessed in Chapter 7. Group education sessions A number of improvements compared to OR1 were observed at OR2 in the group education sessions at RPs. These included an increase in the number of sessions per RP (from average of 1 to 2), a smaller number of women attending each session (average dropped from 30 to 14), a longer duration of the education sessions (from 10 minutes in OR1 to 22 minutes in OR2), and a greater use of visual materials (from 16% in OR1 to 84% in OR2). With very few exceptions, the topics of the education sessions during OR2 followed the WV calendar, which was rare in OR1. No differences between program groups were observed in 2004. 17 As noted in the methods section of this chapter, due to timing and logistics, it was not possible to observe all the same RPs in both rounds. However, half (10) of the RPs were observed both in 2003 and in 2004; comparisons focusing on the number of participants and the staff/participant ratio focus on these 10 RPs because RPs vary widely in these aspects and thus, comparisons between 2003 and 2004 would be misleading if all 20 RPs observed at each round were included in the comparisons. 65 Growth monitoring and promotion Quality of measurement. OR1 observations had revealed errors both in measurement and in plotting of weights. During OR2, observations focused on one aspect of technique that had been covered in the retraining, which was to wait until the needle on the scale had stopped moving before reading the weight. Marked improvements on this aspect were observed between the two rounds, with the proportion of readings of the scale with the needle “swinging a lot” reduced from 21% in OR1 to 1% in 2004. Improvements were seen in both program areas, but in 2004, a somewhat larger proportion of measurements were correctly taken in RPs from the preventive (68%) compared to the recuperative (50%) area. Quality of communication with mothers. Communication with mothers during GMP was identified as particularly problematic in OR1. Retraining was planned to ensure that mothers were informed and counseled appropriately, as described above in Section 5.4.2. Information on the quality of communication was gathered both through direct observation of GMP activities (n = 100) and through exit interviews with mothers (n = 128). Tables 5.2, 5.3, and 5.4 provide detailed results comparing OR1 and OR2 and comparing preventive to recuperative RPs. Key results suggest that ™ Overall, mothers were much more likely to be given information and brief counseling in 2004 than in 2003; ™ Mothers in preventive area RPs were somewhat more likely to be informed of their child’s weight, nutritional status, and progress (gain/loss) and were more likely to receive brief individual counseling than mothers in the recuperative area; ™ Among those given information on their child’s progress, information was more likely to be accurate in preventive (98%) than in recuperative areas (76%); ™ Two messages selected during the consultation—promoting use of the special gruels, and reinforcing importance of attendance at MCs—were rarely used in OR2 (10% and 2% of observations, respectively), and the most common counseling messages, while not inappropriate, remained vague (“feed more food”). Availability of supplies Because of the perceived difficulty of resolving the problem of availability of supplies, and the fact that this problem was largely outside of WV’s control, the participants in the consultative workshop concluded that expectations should be reasonable. It was considered feasible and necessary for WV-Haiti management to be persistent in attempts to maintain supplies, and for management to maintain good communication with collaborating agencies. However, these management efforts were not judged to be likely to completely resolve this issue. OR2 results confirmed this reality, and no differences between the two program groups in these aspects were identified. 66 Table 5.2 Types of information communicated to mothers during growth monitoring at Rally Posts, by year, by program area, and by research method Program area Overall Overall Preventive Recuperative Preventive Recuperative Year 2003 2004 2004 Method Observation Observation Interview Observation Observation Interview Interview Number of children n = 38 N = 100 n = 128 n = 50 n = 50 n = 85 n = 43 Percent of children Child was weighed 100 100 100 100 100 100 100 Weight was plotted / recorded on health card 76 100 n/aa 100 100 n/aa n/aa Nutritional status was assessed 84 100 n/aa 100 100 n/aa n/aa Mother was told weight 42 89 96 96 82 98 93 Mother was told nutritional status 24 81 47 96 66 50 40 Mother was told if child gained or lost weight n/ab 80 62 96 64 64 58 Mother was given individual message 37 71 55 84 58 55 54 Mother was told nutritional status if child’s weight-for-age is: Normal 22 85 46 95 65 47 33 M1 / “orange”c 13 79 50 100 71 63 46 M2 / “yellow”d 40 50 67 n/a 50 n/a 67 Mother told if child gained or lost weight: Normal n/ab 82 62 95 57 64 50 M1 / “orange” n/ab 82 67 100 76 75 64 M2 / “yellow” n/ab 50 100 n/a 50 n/a 100 Mothers given individual message if: Normal 22 71 49 81 52 52 17 M1 / “orange” 33 71 73 100 62 88 68 M2 / “yellow” 45 67 100 n/a 67 n/a 100 a Women were not asked during exit interviews if weight was plotted or if nutritional status was assessed. b This information was not recorded by observers in 2003. c OR-2004 occurred during a period of transition between two types of health card; one indicated classifications using the Gomez system (M1, etc.) and the other showed different color zones for children falling below cutoffs using the Z-score system. d Only six of the children for whom growth monitoring was observed were classified as M2 or “yellow”; all six were in the recuperative program area. 67 Table 5.3 Types of advice given to mothers at Rally Posts in 2004, by child’s progressa, b Overall (n = 100) Gained weight (n = 52) Lost weight (n = 40) Same weight (n = 8) Percent of children Mother was praised 27 50 3 0 Reinforced importance of MCs 2 0 5 0 Told to feed enriched gruels 10 6 18 0 Told to feed more food 37 27 48 50 Given other advice 16 12 20 38 No message after weighing 29 29 30 25 a Data are from direct observation of growth monitoring and promotion. b Some women were given more than one type of advice, so percents sum to more than 100. Table 5.4 Advice given at Rally Posts in 2004, by child’s progress and program areaa, b Gained weight Lost weight Preventive (n = 24) Recuperative (n = 28) Preventive (n = 21) Recuperative (n = 19) Percent of children Mother was praised 58 43 0 5 Reinforced importance of MCs 0 0 10 0 Told to feed enriched gruels 4 7 29 5 Told to feed more food 38 18 67 26 No message after weighing 13 43 19 42 a Data are from direct observation of growth monitoring and promotion. b Some women were given more than one type of advice, so percents sum to more than 100. 5.4.3.2 Key results from the Mothers’ Clubs: Improving organization and maintaining quality of communication Improving organization of MCs Clear improvements were observed in 2004 in organizing the MCs according to the two program approaches: in both preventive and recuperative areas, staff had succeeded in grouping women appropriately, according to their physiological status (pregnant, lactating in both program approaches), their child’s malnutrition (recuperative approach) and/or their child’s age group (preventive approach). In both program areas, the content of observed sessions was also found to be appropriate for maternal status and/or child age or status. MCs were also of appropriate size in 2004 (range, 7-17 women), whereas in 2003, four of the 20 MCs observed were considered too large (more than 20 women). Maintaining high-quality communication OR1 had documented a very high quality of communication and facilitation of MC sessions, so maintaining that level of quality was perceived as a potential challenge. Five domains of quality were observed in both years: (1) technical content; (2) session management and organization; (3) teaching and facilitation skills; (4) attitudes displayed (with observer to explicitly note how attitude was demonstrated); and (5) atmosphere at the learning session. 68 Within each of these domains, the observation tool listed a number of specific and observable practices; these are detailed in Annex 5.2. The results presented in Table 5.5 show that the generally high level of quality noted in 2003 was maintained in 2004. The range of scores observed, however, suggests that there was still notable variability in health workers’ performance in OR2. No major differences were observed between program areas. Table 5.5 Quality of information and facilitation at Mothers’ Clubs, by year and by program areaa Overall 2004 2003 (n = 20) 2004 (n = 20) Preventive (n = 10) Recuperative (n = 10) Percent or mean (range) 1. Technical content - Percent of key session points presented correctly 83% (64-100%) 84% (42-100%) 82% (42-100%) 86% (64-100%) - Percent of key points where incorrect information was presented 14% (7-33%) 10% (0-58%) 14% (0-58%) 2% (0-21%) - Percent of key points not presented during session 9% (0-40%) 6% (0-36%) 4% (0-18%) 8% (0-36%) 2. Session management and organization (score 0 to 6) 5.1 (3-6) 4.8 (3-6) 5.0 (4-6) 4.5 (3-6) 3. Teaching and facilitation skills (score 0-9)b 6.5 (2-9) 6.7 (2-9) 7.2 (5-9) 6.1 (2-9) 4. Attitudes displayed (score 0-4) 3.0 (0-4) 2.5 (0-4) 2.7 (1-4) 2.2 (0-4) 5. Atmosphere of learning session (score 0-6) 3.2 (1-5) 3.4 (1-6) 3.4 (1-6) 3.4 (1-6) a For each summary score presented, the HP (or assistant) received a score of one for each positive checklist behavior s/he demonstrated. These were then summed to create the score for each domain. b There were 11 possible “behaviors” to observe in the domain of teaching and facilitation skills. However, two behaviors relating to use of visual aids were not applicable for several of the observed sessions, and so those two behaviors were not included in the summary score. 5.4.3.3 Summary of improvements and differences in implementation between program areas Table 5.6 summarizes changes implemented since the February 2004 workshop and identifies the few areas where implementation differed between the two program areas. In the table, the third column indicates “no differences” between the groups if any observed differences are small and considered of no practical significance. In the fourth column, the potential influence of the observed differences is described. The overall picture presented is one of impressive success in implementing the program improvements that had been identified and prioritized in the workshop. For most changes identified and prioritized at the consultative workshop, significant progress was seen in both preventive and recuperative RPs, with no large differences between the two program groups. Among the few differences observed between the two program areas, the most notable are related to participation at RPs. As noted in Table 5.6, higher participation in preventive areas could negatively impact the quality of services through crowding and shortage of time for health staff to focus on each participant. We found, however, that quality of measurement in growth 69 Table 5.6 Summary of program changes in WV-Haiti MCHN program, and differences between preventive and recuperative program areas in 2004 Program element Change between 2003 and 2004 Differences between preventive and recuperative program areas in 2004 Potential influence of differences on impact At the Rally Posts (RPs) Organization—sequence of activities and bottlenecks Participants now provided with a number on arrival, per plan; sequence of activities is much more consistent and follows plan, as compared to 2003. No differences. N/A Organization—staffing and participation No change in size of RPs for those observed in both 2003 and 2004. No change in average number of staff, but range in number of staff at RPs reduced from 1-7 to 4-5; participant-to-staff ratio slightly reduced, and maximum ratio decreased from 59:1 to 33:1. No difference between program groups in number of staff ; the number of participants and the participant-to-staff ratio were higher in preventive RPs. No differences in staff costs; higher participant-to-staff ratios could increase waiting time or decrease quality of service in preventive areas. Age distribution of children No change between 2003 and 2004. Greater proportion of younger children (< 2 years) at preventive RPs; many fewer older children (2-5 years) at recuperative RPs. Could have implications for coverage (e.g., lower immunization coverage for children < 2 years of age in recuperative areas; lower coverage for ORS, Vitamin A and deworming for older children in preventive areas). Group education sessions Many positive changes successfully implemented: More education sessions; Longer sessions; Smaller groups of women; Follow monthly plan; More use of visuals. No differences. N/A (continued) 70 Program element Change between 2003 and 2004 Differences between preventive and recuperative program areas in 2004 Potential influence of differences on impact Growth monitoring and promotion - Quality of measurement Much improvement in targeted technique (reading scale when needle is still); other problems persist (incorrect hanging of scales, slight inaccuracy of scales). More improvement in weighing technique observed in preventive RPs; scales also slightly more accurate relative to standard weight. Effectiveness of recuperative program could be reduced if children are poorly measured and consequently misclassified as malnourished or well￾nourished. - Quality of communication Major increases in the proportion of mothers informed of child’s weight, nutritional status, and child’s progress. Mothers also more likely to receive brief advice. Frequency of delivery of specific recommended messages still low; messages still tend to be vague, although not necessarily incorrect. Increases were larger in preventive RPs; recommended messages given more frequently; information on progress more likely to be accurate. Impact of GMP could be higher better in preventive areas if advice triggers action At the Mothers’ Clubs (MCs) Organization Marked improvement in organizing MCs according to plan, and in providing education session appropriate to the group. No differences. N/A Quality of education On average, high quality of education at MCs maintained since 2003 in five domains: technical content; session management; teaching skills; attitudes, and atmosphere. Quality still varies a lot between health staff. No differences. N/A 71 monitoring was slightly better in preventive compared to recuperative areas, and so was the quality of individual counseling of mothers. Finally, the age distribution of children attending the RPs was quite different, with a greater proportion of younger children, and fewer older children attending preventive area RPs. This could have implications for coverage of various services (vaccination, vitamin A supplementation, ORS distribution, and deworming), a topic that will be discussed in Chapter 8. In addition to assessing changes and differences in implementation between program areas, we also explored two additional sets of factors that could influence the effectiveness of the program—and had the potential to differ between program areas. These are staff-level factors such as knowledge, motivation, and workload (reported in the next section) and the mothers’ knowledge and experiences with trial and adoption of recommended practices (reported in Chapter 8). 5.4.3.4 Staff skills and motivation Information from staff focus groups (OR1) was used to guide development of structured interview tools, used during OR2. Health promoters and assistant health promoters from 19 of the 20 study clusters were interviewed. Interview questions covered the following domains: job satisfaction; motivation; perceptions of supervision; technical knowledge, and time allocation and workload. Detailed results are reported in Menon et al. 2005; a brief summary is provided here. Job satisfaction. Results were nearly identical in the two program areas, all staff reported that, overall, they were at least “somewhat satisfied” with their jobs (29%). A similar number (32%) reported being “very satisfied” and the remaining 40% were “satisfied.” Motivation. Staff were asked 24 questions reflecting various qualities of the work environment and their own perceptions and beliefs about their work, each of which could bear on motivation. For example, staff were asked if they agreed/disagreed that MCs could change behaviors; they themselves received adequate training; program management valued their work; they were satisfied with their salary, etc. (see Annex 5.3 for the full list of questions). The 24 questions were summarized into scales reflecting the following dimensions: ™ Feeling of being valued by management and participants (1 scale); ™ Confidence in themselves and enjoyment of work (1 scale); ™ Adequacy of training, salary, and support from supervisors/management (3 scales); ™ Discouragement and plans to leave (2 scales). Results for these scales showed nearly identical scores for staff in the two program areas. Staff had many positive perceptions and generally felt valued and well supported, adequately trained, and confident, and they reported enjoying their work and viewing it as important. There was more variability in responses related to salary. 72 Perceptions of supervision. Staff were also asked a series of 29 questions regarding their perceptions of the quality of supervision that they received. For example, staff were asked how frequently their supervisor consulted them before changing their activities, and how often the supervisor took concerns of frontline staff up to senior management. The full list of questions is provided in Annex 5.4. The responses were summarized into two scales. The first reflected staff perceptions of the adequacy of supervision in terms of tasks; i.e., did the supervisor visit, provide appropriate feedback, ensure adequate supplies, and help organize activities. The second captured whether staff felt supported, valued, and respected by their supervisor. Results were once again nearly identical between the two program areas. Scores for each of the two scales ranged from 15-75, and mean scores were 58 (supervisor performs expected tasks) and 55 (supportive supervision). The mean scores are toward the higher end of the range in each case but ranges were quite wide, reflecting diversity in levels of satisfaction with supervision. Technical knowledge. Staff were asked 44 factual questions, covering “core” material in the MC sessions (26 questions) and additional questions covering background material related to nutrition. Results were strongly positive and consistent, and did not differ between program areas. When all 44 questions were summed into a score, the average score was 88% correct (range 79-94%). On the “core” material, scores were even better, with an average score of 96% correct (range 90-100). Time allocation/workload. Information on time use was gathered from staff and management. Staff were asked to report the number of service delivery points they were responsible for organizing and/or staffing each month (RPs, MCs, and FDPs). In addition, senior management outlined expectations for the number of days each health agent or colvol would spend in activities such as reporting, training and meetings with supervisors, community meetings, etc. When all activities are summed, 15-16 workdays are required to cover service delivery points and all meetings, training, reporting responsibilities, etc. In addition, health promoters must find time for home visits with new mothers and in households with malnourished children. Despite the higher participation and larger number of MCs in preventive program areas, health promoters in both areas reported very similar workloads. Assistant health promoters in preventive program areas reported working approximately 10 hours per month more than their counterparts in recuperative areas; the largest difference was in time spent on MCs. In summary, we found no substantial differences in staff-related factors between the two program areas, and certainly no differences that were likely to have significant implications for program implementation. We found that the work context for WV health staff was very positive, with the presence of motivating factors like good training, good relationships among staff, and a feeling of being valued by management and beneficiaries. Staff also perceived supervisory practices to be appropriate and supportive, and staff time allocation to program tasks appeared to be reasonable. We also found few de-motivating factors. Finally, staff appeared to possess both very good technical knowledge and the communication skills needed to ensure the effectiveness of the BCC arm of the program. 73 5.5 Conclusions The two rounds of OR described in this chapter allowed us to meet the three objectives outlined earlier, i.e., to assess program implementation and service delivery, to strengthen program implementation, and to gain sufficient knowledge about implementation and quality of service delivery in the two program approaches to allow appropriate interpretation of impact evaluation results. In the process, the operations research studies also provided several opportunities for active engagement and discussion among the external IFPRI-Cornell research team and the internal WV-Haiti implementation team. The OR process also provided an in-depth understanding of several key steps along the program theory pathway laid out in Chapter 2. More important for the impact evaluation, it showed that there were few, if any, differences between program groups in program delivery. Program services were being delivered largely as planned in both program areas and the quality of delivery of the BCC strategy was high. In addition, the careful comparisons between program groups at different service delivery points as well as staff knowledge, motivation, and perceptions of supervision showed very few differences in these aspects between program areas. In sum, these results suggest that differences in program impact on child outcomes are highly unlikely to be explained by differences in program delivery. They lead us to infer that any differences found in our primary impact measure—i.e., nutritional status—can reasonably be attributed to the design of the program approaches, and not to differences in implementation. 74 75 6. PROGRAM PARTICIPATION AND HOUSEHOLD AND CAREGIVER CHARACTERISTICS 6.1 Introduction This chapter describes the program participation patterns in the preventive and recuperative program communities. It examines participation among pregnant and lactating women as well as among the children. The chapter also compares the characteristics of households and caregivers who participated in the program with those who never participated in the program. We examine participation patterns as well as factors associated with participation to enable better interpretation of the program impact results presented in Chapters 7 through 10 and the cost-effectiveness results in Chapter 11. In examining the factors associated with participation, we focus on those factors that are not expected to be impacted by any of the program inputs, such as age, maternal education and house quality, and access to services. We do not, on the other hand, examine the association between participation and household assets or food security because the program can be expected to improve both of these outcomes through the provision of monthly food assistance. 6.2 Data and Analysis 6.2.1 Data and variables The data for this chapter are derived from the household survey described in Chapter 2 (section 2.4.1.1.). We compare maternal and household characteristics between program participants and nonparticipants. The following variables were used to assess household participation in the different components of the program. Any exposure to the program: Data were gathered on exposure to the program for the index child (12-41 months old), the younger sibling (if the index child had one; 0-11 months old), and any other sibling in the same age range as the index child (12-41 months). For each child, the respondent mother was asked if she had received food assistance either when pregnant or breastfeeding that child. She was also asked if the child had ever been enrolled in the program after s/he was 6 months old, either in the preventive program or the recuperative program. Finally, the respondent was asked if she was currently pregnant or lactating and receiving food assistance. If the response was affirmative for exposure to program benefits for any of the children or for the respondent herself, the household was considered as having ever participated in the program. Current household participation: The respondents were also asked whether she or any of her children was currently enrolled in the program. If the response was affirmative to any of the questions about current participation, the household was considered as currently participating in the program. Index child participation: The mothers of the index children were asked if the child had ever been enrolled in the program after the child was 6 months old. They were also asked if they 76 had been enrolled in the program when pregnant with the index child or breastfeeding the index child. Thus, for each index child, we assessed whether the child had been exposed to the program in utero, via the mothers’ participation when breastfeeding, or whether the child him/herself was a direct program beneficiary. Timing and level of exposure: We examined the timing and level of exposure of the child to the program by asking whether the mother had participated when pregnant and breastfeeding and whether the child had participated after s/he was 6 months old. It would be expected that child and maternal exposure during pregnancy and lactation would be the same between the two program groups, but that exposure would be different once the child reached 6 months of age. This is because in preventive program communities, all children 6 months or older were eligible to receive program benefits, whereas in recuperative program communities, only children with malnutrition were eligible. Participation in Mothers’ Clubs: Respondents were asked whether they had attended MCs or not when they were pregnant with or breastfeeding either the index child, a younger sibling, or another child between 12 and 42 months of age. However, since respondents were likely to have attended several MC sessions when their child was enrolled in the food assistance program, we did not directly ask about maternal attendance to MCs, but rather we asked about whether they had ever missed a session, and if so, how frequently she had missed sessions. Use of Rally Posts: Since the Rally Posts are the entry point into the program, it could be argued that differences in participation between groups could be driven by differences in participation at the RPs. We gathered data on whether children had ever been taken to the RP and whether they had been to the RP in the last month. 6.2.2 Analysis We examined program participation in each program group, using random effects regression approaches to evaluate whether the participation variables differed significantly between program approaches. We also examined use of Rally Posts in the last month by child age, to evaluate whether the two program approaches led to differential use of the RP services for children in different age groups. For household and caregiver characteristics, we examined differences by program approach and exposure, using random effects regression approaches to control for the study design. 6.3 Results 6.3.1 Program participation 6.3.1.1 Household/caregiver-level participation/exposure Table 6.1 provides information on differences between program communities in program participation rates by any member of the household, 18 either at the time of the survey or at any 18 Household participation includes participation by any beneficiary child, or the child’s mother when either pregnant or lactating. Given the possibility of multiple children participating in any household, 77 point in time since the program started, three years before. Results show that current participation is almost 2.5 times higher in preventive compared to recuperative communities. There is, however, no difference between program communities in the proportion of households ever exposed to the program benefits. As expected by design, the average duration of participation at the household level is significantly higher in the preventive group compared to the recuperative group. Table 6.1 Program participation at the household level (includes participation by all children in the household, as well as pregnant/lactating women) Preventive Recuperative (%) Households currently participating 43.4 17.4* For households currently participating: total duration of participation (mean (SD)) 15.5 (7.0) 12.0 (6.2)* Households that ever participated (including current participation) 83.2 82.7 For households who ever participated: total duration of participation (mean (SD)) 15.8 (5.1) 10.6 (6.2)* Note: * p < 0.05. 6.3.1.2 Child-level participation/exposure Table 6.2 shows the use of the program services for pregnant and lactating women and index children. In both program communities, enrollment by mothers when they were pregnant with the index child was around 57% and enrollment when they were breastfeeding the index child was 63%. The number of times mothers received food assistance during pregnancy (average of 4 months) and the first 6 months of lactation (average of 5 months) was also similar between program communities. For children 6 months or older, enrollment patterns are markedly different between program communities, as expected by design (i.e., different targeting mechanisms). Children in the preventive communities, on average, were enrolled at 8 months of age (they are eligible from 6 months on), while children in the recuperative program were enrolled later, on average at 14 months. This was also to be expected, given the timing of growth faltering in this population and the actual time it takes for a child to reach the cutoff points for WAZ that classify them into M2 or M3 malnutrition levels. Consistent with program design, children in the recuperative program received food for a shorter duration than children in the preventive program (7.5 versus 12). About 73% of the children in the preventive program communities had ever been enrolled in the program, while 28% of children in the recuperative program had ever been enrolled. Current child enrollment in the program was 38% in the preventive communities versus 14% in the recuperative communities. Timing of exposure: Table 6.3 presents the timing of exposure to the program among index children (and their mothers) in the two program communities. The proportion of children never exposed to the program either in utero, while being breastfed, or after they were older than 6 months old is very similar in the program groups. However, among children who have been exposed to the program, the timing of exposure is quite different among the two program groups: 78 a larger proportion of children in the recuperative group has been exposed only in pregnancy or in early infancy (via their mother’s participation), while in the preventive group, slightly over 50% of children have been exposed to the program both through their mother’s participation (in utero and in the first six months of breastfeeding) and after they were 6 months or older. These patterns of exposure are generally as expected, based on the design of the two approaches. Table 6.2 Enrollment of index children in the food assistance program, by program group Program communities Preventive Recuperative Percent or Mean (SD) Percent or Mean (SD) Received food assistance when pregnant (q305) 57.2 58.2 Number of times received food when pregnant 4.0 (2.0) 4.0 (2.0) Participated in Mothers’ Club for pregnant women? (q305) 62.8 62.9 Number of times attended MC when pregnant 5.0 (2.0) 5.0 (2.0) Received food assistance when breastfeeding 66.3 62.2 Number of times received food when breastfeeding 5.0 (1.0) 5.0 (1.0) Participated in Mothers’ Club for breastfeeding women 69.8 64.8 Number of times participated in MC for breastfeeding women 5.0 (1.0) 5.0 (1.0) Child ever received food assistance (survey) 73.1 28.2* Child currently receiving food assistance (survey) 37.6 14.1* Age when child was first enrolled 7.7 (2.1) Median – 7; Range (6-23) 13.6 (3.9)* Median – 12 Range (7-37) Age when child last received program benefits 22.8 (2.9) Median – 23 Range (7-36)1 21.5 (6.7) Median – 23 Range ( 7-40) Total number of times received WSB 11.7 (4.3) Median – 12; Range (1-24) 7.5 (4.3)* Median-7 Range (1-24) Notes: * p < 0.05; 1 The range in the preventive group goes to 36 months because malnourished children older than 24 months also receive food assistance. Table 6.3 Timing of exposure for index children, by program group Preventive Recuperative Overall N = 748 N = 750 N = 1,500 Timing of exposure Percent Percent Percent Never exposed in pregnancy, lactation, or child level 22.1 24.9 23.5 Only in pregnancy 0.3 3.6 1.9 Only during lactation 0.9 5.7 3.3 Only at child level 10.4 8.8 9.6 Only pregnancy and lactation 3.6 37.6 20.7 Pregnancy, lactation, and child level 52.4 16.5 34.4 Pregnancy and child 0.9 0.5 0.7 Lactation and child 9.4 2.4 5.9 79 Participation by child age: Table 6.4 presents data on program participation (ever or current) by child age and program. Note that this table includes both index children and their younger siblings. Patterns of participation by child age at the final survey are quite different between program communities, both for any exposure to the program as well as for current participation. In both program groups, there are no children directly exposed to the program in the 0-6 month age group (as expected). In the preventive program group, current and ever exposure rise to over 50% in the 6-11 month age group, while only 5% of children in that age group are exposed to the program in the recuperative group. Among older children—within the age group eligible for the preventive program, i.e., children in the 12-17 and 18-23 month age groups—participation is around 75%, both for ever and current participation. In the same age groups in the recuperative communities, participation is around 18% in the 12-17 month age group and somewhat higher in the 18-23 month age group. For children over 24 months of age, current exposure is very low as expected (since only severely malnourished children over 24 months old are enrolled in the preventive program), while past/any exposure is between 65% and 77% among all age groups above 24 months. The prevalence of severe underweight (WAZ<-3 Z-scores) is 2.2% among children older than 24 months in the preventive program, and this is consistent with the participation rates among children in this age range, especially those over 30 months of age. Table 6.4 Receipt of food assistance among index children, by child age and program participation Preventive Recuperative All children Age group n Ever Current n Ever Current n Ever Current Mean participation (index children + young siblings) 906 63.8 54.1 936 22.9 50.7 1,842 43.1 53.1 Mean participation (12-41 mo) 748 73.1 37.6 752 28.2 14.1 1,500 50.6 25.8 - 0-6 mo 99 0.0 0.0 125 0.0 0.0 224 0.0 0.0 - 6-11 mo 60 53.3 53.3 60 5.0 5.0 120 29.2 29.2 - 12-17 mo 197 76.1 75.1 187 19.3 18.2 384 48.4 47.4 - 18-23 mo 153 77.1 73.9 119 27.7 21.1 272 55.5 50.7 - 24-29 mo 180 74.4 7.7 167 29.9 11.9 347 53.0 9.8 - 30-35 mo 115 67.8 2.6 140 32.1 11.4 255 48.2 7.4 - 36 mo and older 102 64.7 1.9 138 34.8 7.9 240 47.5 5.4 In the recuperative program, rates of past/any exposure among children over 24 months of age range between 28% and 35%. Current exposure is highest among children in the 12-17 and 18-23 month age groups, and gradually decreases beyond this age. These patterns mirror the age distribution of malnutrition—and therefore eligibility for the program among children in recuperative communities; i.e., the prevalence of underweight is highest in children 12-23 months of age. Notably, the participation rates are almost halved among children over 24 months of age, compared to those between 12 and 23 months of age, even though the prevalence of underweight is not markedly lower among older children (see Chapter 7). This could suggest that fewer older children are taken to the RPs even in the recuperative communities (since eligibility to receive food assistance is contingent upon children being taken to the RP for screening). Exposure to MCs: Table 6.5 presents data on exposure to MCs among the respondents. As expected, based on the patterns of participation in the food assistance program, MC exposure 80 is not different between the program groups in pregnancy and lactation. The results also show that almost 50% of respondents reported missing at least one MC session. The proportion of respondents who reported missing an MC session “only rarely” was higher in the preventive compared to the recuperative group, while those who reported missing sessions “often” was higher in the recuperative compared to the preventive group. Use of Rally Post services: The use of RPs was not different between program groups when examined in terms of whether an index child had ever been taken to an RP since 2002 (Table 6.5). However, a slightly larger proportion of children in the preventive communities had been taken to the RP in the past one month compared to children in the recuperative communities, and the total number of times the child had been taken to an RP in the last year was slightly higher in the preventive communities than in the recuperative communities. Table 6.5 Participation in Mothers’ Clubs and use of Rally Post services Preventive Percent Recuperative Percent Mothers’ Club participation Participated in Mothers’ Club for pregnant women? 62.8 62.9 Number of times attended MC when pregnant 5.0 (2.0) 5.0 (2.0) Participated in Mothers’ Club for breastfeeding women 69.8 64.8 Number of times participated in MC for breastfeeding women 5.0 (1.0) 5.0 (1.0) MC participation by mother when child was enrolled in food assistance programa - Never missed an MC session 51.3 48.7 - Frequency of having missed MC sessions - rarely 40.3 27.4 - sometimes 51.1 58.3 - often 8.6 14.3 Rally Post utilization - Percent of children ever taken to RP (between 2002 and 2005) 96.7 97.1 - Percent of children taken to RP in month preceding survey 52.6 49.7 - Number of times taken to RP in one year preceding survey 7.5 (3.2) 7.1 (3.2)b a All beneficiary children’s mothers are obligated to attend MCs. We did not gather data on the number of MCs attended when children were enrolled in the program, but asked mothers about how often they had missed. b p < 0.05. Use of RP services by child age, however, differed between the two program communities, particularly for older children (Table 6.6). Use of RP services was very high for children who were less than 12 months old, and not different between program communities. On the other hand, RP participation rates were higher for children 12-23 months old in the preventive communities compared with the recuperative communities. For children older than 23 months, the proportion of children taken to RPs was higher in the recuperative communities than in the preventive communities. These significant differences could reflect household and caregiver understanding of the targeting mechanisms of the two program approaches, such that in the preventive program communities, caregivers were more diligent about bringing children less than 24 months of age, while in the recuperative communities, they were more likely to bring older children (who might be identified as being malnourished). The lack of difference between program communities in RP use for children younger than 12 months is reassuring, however, since it implies that regardless of the targeting mechanism for food assistance, younger children 81 are still taken to the RPs in the recuperative communities for immunization and other preventive services. Table 6.6 RP participation, by child age and program group Child was taken to RP in the past month Age group Overall Preventive Recuperative < 12 mo 86.0 88.5 83.5 12-18 mo 72.1 77.9* 65.7* 18-24 mo 64.7 73.5 53.4 24-30 mo 43.2 38.6 47.9 30-36 mo 33.5 28.2 37.8 36-42 mo 31.1 20.6 38.9 * p < 0.05 (difference by age for index children 12-41 months of age, within program group. Program uptake and targeting: We define uptake as the proportion of eligible children in each program area who are currently enrolled in the program, and targeting as the percentage of children enrolled in each of the programs who are eligible to receive food assistance under that program approach. Since the program services were provided in all the communities covered by the survey, uptake is largely driven by household decisions to bring the child to the RP for enrollment in the food assistance arm of the program. For estimates of uptake as well as targeting, we use data from our final survey data. In preventive areas, uptake was high (75%) and targeting was very effective, with up to 93% of current participants (survey) being in the targeted age range (6-24 months) (Table 6.7). In the recuperative areas, however, only 29% of currently underweight children were enrolled in the program, and only 57% of those enrolled were underweight at the time of the final survey. Table 6.7 Program uptake and targeting, by program group Program communities Preventive Recuperative Percent Percent Program uptake (survey) - percent of targeted group who currently participate 74.6 29.1 Targeting: percent of current participants who meet eligibility criteria (survey) 93.2 56.6 It should be noted that estimates of uptake and targeting in the recuperative group cannot be assessed accurately with a cross-sectional survey. This is because the eligibility criterion is underweight at the time of enrollment, not at the time of the survey. In trying to estimate uptake and targeting based on measurements taken at the time of the survey, we introduce errors due to the fact that (1) some children may have recovered since they were enrolled in the program and are no longer underweight (thereby appearing as non-eligible); (2) some children may have become underweight only recently and have not been screened into the program yet (thereby appearing as not covered); (3) some children might have been enrolled in the past but have now relapsed (or never recovered) (also appearing as not currently covered19). 19 Note that children in the recuperative group can re-enter the program after one year of exiting from the program, or if they are severely malnourished at the end of the 9-month period. Children in the preventive group re-enter if they are malnourished (WAZ < -2) between 24-59 months of age. The proportion of children previously enrolled who later re-entered the program was 8.5 percent (18 children) in the recuperative approach and 1.5 percent in the preventive approach (8 children). 82 In summary: ™ Enrollment and duration of participation in the food assistance program during pregnancy and the first six months of lactation was not different between program communities. ™ Enrollment and duration of participation after the child reached 6 months of age was significantly different between the program communities, as expected by design. ™ Timing of exposure to the program benefits was different between the program communities, with more children having been exposed only through their mother’s participation (i.e., during pregnancy and lactation) in the recuperative communities, versus a greater proportion of children having been exposed both through their mother’s participation and as direct beneficiaries (after 6 months of age) in preventive communities. This was also expected by design. ™ Child age is closely associated with participation rates (particularly current participation). In the preventive program communities, participation is high between 6 and 24 months of age as expected, while in the recuperative communities, participation is highest when children are between 12 and 23 months of age, when undernutrition is at its peak. ™ MC exposure is not different between program groups for participation during pregnancy and lactation. The frequency of missing MC sessions is slightly lower in the preventive group than the recuperative group. ™ Overall RP participation is reasonable (half of the children were taken in the month preceding the survey, and more than 95 percent had ever been taken to the RP) and similar between program groups. Age differences in RP attendance between program groups were quite marked, however, with fewer children above 24 months of age taken to RPs in the preventive communities. ™ There were significant differences in program uptake and efficiency of targeting in the two groups. Uptake was much higher in the preventive than in the recuperative approach, and more than 90 percent of children currently enrolled in the preventive group met the targeting criteria. Although it is difficult to interpret both uptake and targeting in the recuperative group, the results point to differences that could be driven by community and household perceptions about the program. 6.3.2 Household and caregiver characteristics In this section, we briefly examine some household and caregiver characteristics, focusing on those that are not expected to be influenced directly by the program. In addition to comparing household and caregiver characteristics by program group, we also compare household and caregiver characteristics of program participants and nonparticipants in both program groups. We focus on comparisons of characteristics of household or caregivers who had ever participated in the WV MCHN program to those that had never participated in the 83 program. Since current participation is largely driven by child eligibility (which differs between the two approaches), we consider comparisons based on any exposure to be more relevant to understanding the drivers of participation. 6.3.2.1 Household characteristics Table 6.8 provides information on household characteristics by program group, and comparing participants and nonparticipants within each program group. Table 6.8 Household characteristics, by program and by program participation (ever participated) Preventive Recuperative Preventive Recuperative (n = 748) (n = 752) (n = 126) (n = 622) (n = 130) (n = 622) Nonparticipant Participant Non￾participant Participant Variable Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD No. of rooms in house q27a 2.2 0.7 2.2 0.8 2.2 0.6 2.2 0.8 2.1 0.7 2.2 0.8 Household size hhsize 6.6 2.3 6.6 2.3 6.1 2.2 6.6a 2.3 6.1 2.6 6.7a 2.2 % % % % % % Own house q24a -Own house 91.8 90.0 93.7 91.5 86.9 90.7 Rents 3.7 5.3 2.4 4.0 7.7 4.8 Free housing 4.4 4.7 4.0 4.5 5.4 4.5 Own land on which house is q24b 78.6 75.9 81.7 78.0 73.8 76.4 Floor material q27b Earth/sand/rock 94.9 95.1 95.2 94.9 94.6 95.2 Concrete 5.1 4.9 4.8 5.1 5.4 4.8 Wall material q27c Clissade/earth 25.9 27.8 19.8 27.2 20.0 29.4 Wood/plank 3.2 1.1 4.0 3.1 0.0 1.3 Stone blocks/stones 2.8 4.8 0.0 3.4 6.2 4.5 Palissade 66.0 65.2 73.8 64.5 72.3 63.7 Roof material q27d Thatched roof 50.4 50.8 53.2 49.8 50.8 50.8 Aluminum 49.6 49.2 46.8 50.2 49.2 49.2 Drinking water source q21 Public tap water 28.2 31.3 22.2 29.4 31.5 31.2 Public open well 3.1 1.9 2.4 3.2 3.8 1.4 In compound covered/protected well 0.1 0.8 0.0 0.2 0.8 0.8 Protected spring 6.8 4.8 8.7 6.4 5.4 4.7 Unprotected spring 58.2 56.6 65.1 56.8 56.2 56.8 River 2.3 3.5 0.8 2.6 2.3 3.7 a Difference between participants and nonparticipants (within program group) is significant at p < 0.05 (random effects regression analysis). Overall, there were no significant differences between the program groups in terms of general household characteristics such as house construction, number of rooms in the house, home ownership, and sources of drinking water. There were also no significant differences in these characteristics among participant and nonparticipant households in the two program groups. However, household size was significantly different between those who had ever participated in the program and those who had never participated. In both program groups, 84 household size was larger among households that had ever participated in the program, compared to households that had never received program benefits.20 6.3.2.2 Caregiver characteristics Table 6.9 provides information on caregiver characteristics by program group, and comparing participants and nonparticipants within each program group. There were no significant differences between the program groups on any of the maternal characteristics examined. The sets of program communities were similar in terms of major determinants of child nutrition, such as caregiver education, education of the caregiver’s partner, marital status, and employment status. The program communities were also similar in terms of characteristics such as maternal BMI, communications between the caregiver and her partner, ownership of assets, control over household purchases, financial and material support, involvement in decisionmaking, and availability of household help. Maternal work-related characteristics were slightly different between the program communities, but differences were of small magnitude and nonstatistically significant. When comparing caregivers from households who had ever received program benefits with those that had never received benefits (Table 6.9), differences were seen in communications between the respondent and her spouse/partner, women’s ownership of assets, and marital status. In the recuperative group, communication between respondent and spouse/partner appeared to be better among participants than among nonparticipants. In both program groups, women who had ever participated in the program scored slightly higher on the asset ownership scale than those who had never been exposed to the program, and a slightly larger proportion of participants were married (or had a partner) compared to women never exposed to the program. Although Table 6.9 shows some trends toward less time spent on working away from home among participants in both program communities, these differences are not statistically significant. However, they could reflect trends that indicate that women who work away from home for long hours are unable to participate in the program. Conversely, it could also signify a program impact, i.e., that program participation reduced the amount of time that mothers had to spend away from the home on work because the program provided a food transfer to the household. In summary: ™ There were no significant differences in household or caregiver characteristics among program groups at the time of the final survey, although caregivers in the preventive communities were slightly more likely to be working away from home. ™ There were few differences in household characteristics among participant and nonparticipant households. However, households exposed to the program were likely to be larger in size than households never exposed to the program. 20 Note that all households in both the baseline and the final survey had at least one child between 12 and 41 month of age, as this was a selection criterion for inclusion in the survey. 85 Table 6.9 Caregiver/respondent characteristics, by program group and by program participation (ever participated) Preventive Recuperative Preventive Recuperative (n = 748) (n = 752) (n = 126) (n = 622) (n = 130) (n = 622) Non￾participant Participant Non￾participant Participant Variable Mean SD Mean SD Mean SD Mean SD Mean SD Mean SD Education of the respondent mother (years) edu_resp 2.0 2.7 1 9 2.7 1.8 2.5 2.0 2.7 2.0 2.7 1.9 2.7 Education of the partner/husband edu_part 3.2 3.5 3 2 3.4 3.1 3.4 3.2 3.5 3.5 3.6 3.1 3.4 Maternal body mass index bmi 21.8 3.3 21.7 3.5 21.9 3.6 21.7 3.3 21.4 3.3 21.8 3.5 Couple communications scale q802comm 6.2 1.8 6 2 1.8 6.0 1.8 6.2 1.8 5.7 1.8 6.3 a 1.8 Women's ownership of assets (scale) q803poss 2.1 1.0 2.1 1.1 1.9 1.1 2.2 a 1.0 2.0 1.1 2.1 a 1.1 Control over purchasing (scale) q805purc 3.2 2.4 3 3 2.5 3.4 2.4 3.2 2.5 3.6 2.6 3.3 2.5 Financial/material support (scale) q810supp 1.6 1.1 1.6 1.2 1.7 1.2 1.6 1.1 1.5 1.1 1.6 1.2 Social support (financial/material/ emotional) (scale) q810supn 2.2 1.5 2 1 1.5 2.3 1.5 2.2 1.5 2.0 1.4 2.1 1.5 Involvement of respondent in HH decision￾making q817invl 8.6 3.0 8.8 3.0 8.4 3.2 8.7 3.0 9.1 2.9 8.7 3.1 Number of tasks mother gets help with (scale) ntasks 5.4 3.0 5.4 3.0 5.2 3.2 5.4 2.9 5.2 3.1 5.4 3.0 Number of childcare tasks mother gets help with (scale) ch_help 2.1 1.2 2.1 1.3 2.0 1.3 2.2 1.2 2.0 1.3 2.2 1.2 % % % % % % Currently pregnant q34 14.6 16.6 18.3 13.8 18.5 16.2 Married or partnered q201 Yes married 27.3 30.2 22.2 28.3 a 24.6 31.4 a yes, placee (civil union) 62.0 56.3 60.3 62.4 55.4 56.4 yes, partnered, but partner doesn't live with her 2.0 2.5 3.2 1.8 3.1 2.4 yes, live with a man 1.6 2.4 3.2 1.3 3.1 2.3 no, not in union 7.1 8.6 11.1 6.3 13.8 7.6 Ever been to school q207 52.8 52.0 52.4 52.9 55.4 51.3 Schooling level(only those who went to school) q207c primary 90.4 89.6 93.7 89.7 88.5 89.9 secondary 8.0 7.7 5.6 8.5 9.2 7.4 higher 0.0 0.1 0.0 0.0 0.0 0.2 Currently employed q208 76.6 74.7 77.8 76.4 73.1 75.1 Respondent's occupation q209 farms own land or family land 41.7 42.3 40.5 42.0 39.2 42.9 farms land of other persons 2.5 1.9 2.4 2.6 0.0 2.3 business 0.8 0.8 0.8 0.8 0.0 1.0 market/trade 29.3 26.2 29.4 29.3 27.7 25.9 office/institution 1.1 1.2 1.6 1.0 0.8 1.3 manual labor 3.6 3.5 5.6 3.2 5.4 3.1 unpaid work 0.1 0.1 0.0 0.2 0.8 0.0 fisherman 1.3 2.3 0.0 1.6 3.1 2.1 retired 0.0 0.0 0.0 0.0 0.0 0.0 other 0.0 0.3 0.0 0.0 0.8 0.2 Unemployed 19.5 21.5 19.8 19.5 22.3 21.4 Location of work q209d home 16.7 18.9 22.2 15.6 20.0 18.6 away from home 34.2 34.8 35.7 33.9 32.3 35.4 both 29.5 24.7 22.2 31.0 25.4 24.6 Unemployed 19.5 21.5 19.8 19.5 22.3 21.4 Duration away from home, when working q209f > 1 day 2.7 2.4 3.2 2.6 1.5 2.6 whole day 22.1 23.9 23.8 21.7 26.2 23.5 1/2 day 31.0 27.5 24.6 32.3 26.9 27.7 < 4 hours 8.0 5.7 6.3 8.4 3.1 6.3 Unemployed/work at home 36.2 40.4 42.1 35.0 42.3 40.0 Childcare arrangements when works outside q213 always bring with her 4.9 6.4 4.0 5.1 5.4 6.6 always leave with someone else 49.3 45.9 46.0 50.0 46.9 45.7 both 8.7 6.9 7.9 8.8 6.9 6.9 Unemployed/work at home 37.0 40.8 42.1 36.0 40.8 40.8 a Difference between participants and nonparticipants (within program group) is significant at p<0.05 (random effects regression analysis). 86 ™ Among caregivers ever exposed to the program, differences were seen in communication between the respondent and her spouse/partner, ownership of assets by the respondent, and marital status of the respondents. 6.4 Summary of Results This chapter examined differences in program participation between the preventive and recuperative program communities, and looked at caregiver and household characteristics associated with having ever participated in the program. Our findings indicate that program participation differences are driven largely by differences in targeting mechanisms for children, since participation during pregnancy and the first six months of lactation are not different between program communities. This leads to substantial differences in the timing and duration of exposure to the program between the program groups. Our assessment of the use of Rally Posts in the two program communities suggests that differences in program enrollment by child age could be driven by the caregivers’ understanding of program targeting, since fewer older children are brought to preventive RPs and fewer younger children to recuperative RPs. We found no significant differences in household or caregiver characteristics among program groups at the time of the final survey, although caregivers in the preventive communities were slightly more likely to be working away from home. There were also few differences in household characteristics between households who had ever participated and those who had never participated in the program. Among caregivers ever exposed to the program, however, differences were seen in communications between the respondent and her spouse/partner, and the respondents’ ownership of assets and marital status. In conclusion, our assessment is that differences in program participation between preventive and recuperative communities appear to be largely driven by the design and targeting mechanism of the two program approaches, rather than differences in household or caregiver characteristics. However, a more extensive analysis of the determinants of participation to develop typologies of participants and nonparticipants based on the data on women’s work patterns and other characteristics could be useful to provide a deeper and more nuanced understanding of the drivers of participation. 87 7. NUTRITIONAL IMPACT OF THE PREVENTIVE APPROACH COMPARED TO THE RECUPERATIVE APPROACH 7.1 Introduction This chapter presents the main results of the effectiveness study comparing the impact of the preventive and recuperative approaches on attained growth and on the prevalence of undernutrition (wasting, stunting, and underweight). It first presents the results of the probability cluster randomized design and follows with additional results that document the plausibility of the results. 7.2 Objective of the Impact Evaluation The main objective of the impact evaluation was to compare the effectiveness of the preventive and the recuperative approaches in reducing community-level childhood undernutrition. Our main hypothesis was that, compared to the recuperative approach, the preventive approach would significantly increase the mean WAZ, HAZ, and WHZ and reduce the prevalence of stunting, underweight, and wasting among children 12-41 months of age who lived in the program communities and had the potential to have been exposed to the program in the previous three years (see below). 7.3 Intervention and Comparison Groups The intervention packages compared are described in detail in Chapter 4. As a reminder, the preventive approach differs from the recuperative approach in three main aspects (see Table 7.1 for summary): (1) the targeting mechanism (undernutrition for recuperative and age for preventive); (2) the focus and timing of the BCC intervention (care and feeding of the undernourished child for recuperative; optimal care, and feeding practices for 6-23-month-old child for preventive); and (3) the timing and duration of eligibility (9 months from the time the child is undernourished for recuperative; 18 months from the time the child is 6 months of age). Both approaches include a similar package of interventions for pregnant women and for lactating women until their child reaches 6 months of age. Table 7.1 Key differences between recuperative and preventive approach intervention packages Intervention component Recuperative approach Preventive approach Targeting mechanism Children 0-59 months of age with weight-for-age Z-scores (WAZ) < -2 All children 6-23 months of agea Children 24-59 months severely malnourished (M3)b Focus of BCC intervention at Mothers’ Clubs Care and feeding of undernourished child; note that other learning sessions on child-feeding and care practices may also be used. Timely delivery of age-specific, relevant, and action￾oriented messages on optimal breastfeeding, caregiving, and complementary feeding practices of children 6-23 months of age. Timing of eligibility Duration of eligibility to receive food and BCC When undernourished 9 months (originally determined by WV) When 6-23 months of age 18 months (entire period when the child is between 6 and 23 months of age) a Note that children 24-59 months of age with WAZ < -3 in the preventive group are eligible for program benefits for up to 9 months, similar to those in the recuperative group. b Severely malnourished children (M3 according to the Gomez classification) are those whose weight-for-age is between 60 and 75% of the median of the weight-for-age CDC/NCHS/WHO standards (Cogill 2003). 88 7.4 Methods 7.4.1 Evaluation design The evaluation used a community-level randomized pre-post design, whereby 10 paired clusters of communities were randomly assigned to either the preventive or the recuperative program group. The baseline survey was conducted between May and September 2002 and the post-evaluation survey was conducted exactly three years later, between May and September 2005, to minimize seasonal variations. All components of the intervention packages, except the newly developed BCC strategy, were implemented immediately following the baseline survey, i.e., in August-September 2002. The full BCC package, however, was implemented several months later (in May 2003). The main outcomes of the evaluation were mean height-for-age Z-scores (HAZ), weight￾for-age Z-scores (WAZ) and weight-for-height Z-scores (WHZ), and the prevalence of childhood stunting, underweight, and wasting. Details about the evaluation design, including sample size estimation, matching and selection of community clusters, the age groups selected for the impact assessment, and the survey design, are described in Chapter 2, Section 2.4.1.1. 7.4.2 Data analysis Data used in this impact analysis were obtained from the surveys described in Chapter 2 and in Annex 2.1. The outcomes of interest of the impact evaluation were mean HAZ, WAZ, and WHZ, and the prevalence of stunting, underweight, and wasting. We used the new WHO reference standards to derive these indicators based on children anthropometric measurements (WHO 2006). Differences between program communities in the mean HAZ, WAZ, and WHZ were tested using a pair-wise comparison at the cluster level (and a paired t-test for statistical significance). Additional analyses used random effects regression modeling with child-level data adjusting for the clustering at the pair level (Murray 1998) to test differences between the groups in child anthropometry, adjusting for child age and gender, and maternal height and schooling. Note that the approaches did not control for additional caregiver or household characteristics because many of these determinants of child anthropometry may have been impacted by the program, e.g., household socioeconomic status, food security, and caregiver knowledge and feeding practices. Differences in the prevalence of stunting, underweight, and wasting at the cluster level were tested using random effects logit approaches (xtlogit in Stata 9) that adjusted for the clustering at the zone level and controlled for child age, gender, maternal height, and schooling. Our power calculations were not conducted to detect differences between program communities in changes over time and thus, the statistical significance of these differences is not 89 reported. Where relevant, differences between baseline and end line within each intervention group (preventive and recuperative) are reported.21 7.5 Results The results section is organized as follows. The first subsection presents the results of comparisons between the preventive and recuperative group at baseline to assess the success of the randomization process. The main impact results based on the community-level probability design are presented next, and focus on difference between the two groups at final survey for the three main anthropometric outcomes: mean HAZ, WAZ, and WHZ, and the prevalence of stunting, underweight, and wasting. The next subsection examines the plausibility of the results, describing changes from baseline to final survey and addressing issues such as dose-response and age-specific response to the intervention. The final subsection compares the prevalence of morbidity symptoms between the preventive and recuperative groups. 7.5.1 Baseline characteristics Results of the paired cluster-level comparison of mean HAZ, WAZ, and WHZ presented in Table 7.2 show that there were no differences between the program groups in nutritional status at baseline. The same is true for differences in the prevalence of stunting, underweight, and wasting tested using random effects regression approaches to adjust for clustering at the zone level. Similarly no differences were found between the two groups in individual-level comparisons of child age, gender distribution, or reported child-feeding practices (Table 7.2). Comparisons between the two program groups to test differences in community, household, and caregiver resources are also presented in Table 7.2. The findings suggest that the randomization process was effective because very few differences between the groups were found in spite of the large number of variables compared. For instance, at the community level we found that the pairs of clusters were largely comparable in terms of key geographic characteristics and access to various services such as closest town, school, market, and health services (not shown). Caregiver and household characteristics of the two groups were also very comparable (see Table 7.2 for subset of variables compared). For instance, no differences between the groups were found in caregiver age, education level, occupation, or in gender of household head, household size, or access to basic water services. In sum, the recuperative and preventive program communities were very similar at baseline, suggesting that randomization was successful. 21 Even though our sample size was estimated for comparisons between program groups at the endpoint, they are adequate for examining the significance of change within each group between baseline and final surveys. However, our sample size is not adequate to detect the statistical significance of the difference of differences. 90 Table 7.2 Comparison of child, maternal, and household characteristics between program groups at baselinea Indicator Recuperative Preventive Nutritional status indicators – cluster level b [n = 10 clusters] Mean (SE) [n = 10 clusters] Mean (SE) Height-for-age Z-score (HAZ) -1.65 (0.10) -1.69 (0.04) Weight-for-age Z-score (WAZ) -1.02 (0.06) -0.97 (0.08) Weight-for-height Z-score (WHZ) -0.18 (0.03) -0.18 (0.05) Other child characteristics (individual level)c,d N = 792 N =7881 Stunting prevalence (%) 37.4 36.7 Underweight prevalence (%) 17.8 17.6 Wasting prevalence (%) 4.3 5.2 Age (mean, SD) 29.4 (7.6) 29.3 (7.9) Gender (% female) 48.0 51.4 Breastfed within 1 hour (%) 19.3 16.2 Fed meals at least minimum recommended number of times (3 times/day) at 12-23 months (%) 58.7 57.6 Mean number of food groups consumed by child (mean, SD) 5.1 (1.5) 5.0 (1.5) Consumed meat, fish, or eggs in previous 24 hours (%) 87.3 89.2 Caregiver characteristicsd (n = 1,514) N = 765 N=759 Age (mean, SD) 30.8 (7.0) 30.8 (8.0) Maternal height at baseline (mean, SD) 157.9 (11.6) 157.6 (15.4) Years of schooling (mean, SD) 1.4 (2.3) 1.6 (2.5) Never attended school (%) 53.2 50.7 Occupation - % Unemployed 16.1 16.5 - % Farming 43.1 42.5 - % Trade/market 32.7 32.0 Household characteristicsd (n = 1,514) N = 765 N = 755 % male head 90.8 90.1 Occupation of head - % Unemployed 2.0 1.4 - % Farming 85.5 86.8 Household size (mean, SD) 6.8 (2.3) 6.7 (2.3) % who own house 94.1 91.1 % who have electricity 2.1 1.9 % who have sanitation facility 57.3 56.0 % who have tap water in the house (%) 1.6 0.9 a None of the differences between the groups were statistically significant. b Differences in mean were tested using cluster-level pair-wise comparisons and paired t-test. c Differences in the prevalence of stunting, underweight, and wasting were tested using a random effects logit model controlling for cluster effect. d Differences in other child, caregiver, and household characteristics were tested at the individual level, with t-tests (for means) and chi￾square tests (for proportions). 7.5.2 Intervention impact: Results from probability design analysis Table 7.3 shows the differences in mean anthropometric outcomes between the two program communities at the end of the intervention period. Both unadjusted, cluster-level means, and means adjusted by random effects regression modeling for cluster effect and for child age and gender are presented. The preventive group had higher mean HAZ, WAZ, and WHZ than the recuperative group. All differences were statistically significant, except the unadjusted difference in mean HAZ. Results of random effects regression approaches controlling for child age and gender, however, show that all three mean anthropometric indicators are significantly higher for the preventive compared to the recuperative group. Additional models controlling for maternal height and schooling in addition to child characteristics showed similar results (not shown). 91 Table 7.3 Mean anthropometric outcomes at final survey, by program group Recuperative Preventive Difference (preventive – recuperative) Child anthropometric outcome n Mean SE n Mean SE HAZ Unadjusted (n =10 clusters/group)a 10 -1.68a 0.05 10 -1.53 0.06 0.15 Adjusted for child age and genderb 746 -1.67* 0.05 735 -1.53 0.05 0.14 WAZ Unadjusted (n clusters) (ICC: 0.021) 10 -1.21* 0.04 10 -0.97 0.06 0.24 Adjusted for child age and gender 746 -1.20* 0.05 735 -0.96 0.05 0.24 WHZ Unadjusted (n =10 clusters) 10 -0.46* 0.04 10 -0.23 0.06 0.23 Adjusted for child age and gender 746 -0.46* 0.05 735 -0.22 0.05 0.24 Note: * Statistically significant (p < 0.05). a Statistical significance of differences in unadjusted means was tested using a paired t-test of cluster level means. b Mean random effects regression models were used to analyze child-level data and adjust for the clustering at the pair level and for child age and gender. Differences between program communities in the prevalence of stunting, underweight, and wasting at final survey are shown in Figure 7.1. At the end of the study, the prevalence of stunting was 4 percentage points lower among children from the preventive (33.9%) compared to the recuperative group (38.2%), while for underweight the difference was 6 percentage points (14.8% for preventive vs. 20.8% for recuperative) and for wasting, the differences in favor of the preventive group was 4 percentage points (3.7% vs. 7.4% for preventive and recuperative, respectively). 7.5.3 Results of plausibility analysis Two sets of additional analyses were conducted to assess the plausibility of the results. The first one is an analysis of differences between preventive and recuperative communities among the age group that was estimated to have the greatest potential to respond to the intervention; this group is composed of children 24-35 months of age at the end of the study— i.e., children who had been exposed to the preventive program benefits from the time when they were 6-23 months of age. Anthropometric measures at final survey were compared between these fully exposed children and those who had been exposed only partially, i.e., who were either 6-23 months at final survey, or were 36-41 months. The second analysis examines the difference between groups in the nutritional status of children who were in their first year of age at the time of the final survey. These children have been minimally exposed to the interventions themselves (since direct benefits for children start at 6 months of age), but their mothers were exposed while pregnant and lactating. Since the two programs offer exactly the same services to pregnant and lactating women, we did not expect any major differences in the growth of children in the first 6 months of age, but possibly thereafter. Before assessing these aspects of plausibility, we also examine changes from baseline in the prevalence of stunting, underweight, and wasting (see Figure 7.1). Baseline results are combined for both program groups because they did not differ at baseline. Using adjusted prevalences for baseline and final surveys in both groups, the results indicate that stunting decreased by 3.5 percentage points since baseline in the preventive communities, while it went 92 up very slightly in the recuperative communities. The prevalence of underweight went up by 2.8 percentage points in the recuperative communities, while it went down by 2.8 percentage points in the preventive communities. Wasting went up by.4.0 percentage points in the recuperative areas while it decreased slightly in the preventive areas. Figure 7.1 Prevalence of undernutrition among children 12-41 months at baseline (groups combined) and end line, by program group Stunting Underweight Wasting % 0 10 20 30 40 Baseline (combined) Recuperative (final) Preventive (final) Notes: Groups were not different at baseline. Random effects logit models adjusted for child age and sex were used to assess statistical significance of differences between program communities. P values are for differences between preventive and recuperative at final survey. Numbers in figure are adjusted prevalences derived from the random effects logit models. When looking at the magnitude of differences between program communities for the age range of children exposed to the intervention between 6 and 23 months of age, larger effect sizes are indeed observed for most indicators (Figures 7.2 and 7.3). Among the children 24-35 months at final survey, differences in mean anthropometric indicators between preventive and recuperative communities were larger than for the other age groups (differences in favor of the preventive group were +0.21 Z-scores for HAZ, +0.34 for WAZ, and +0.27 for WHZ). Differences in favor of the preventive group in the prevalence of underweight were also of larger magnitude in this age group compared to the sample as a whole (7.3 percentage points). For stunting, however, large differences were seen both in the 12-23 and the 24-35 month age groups (6.5 and 6.3 percentage points), but not in the older age group (36-41 months). In fact, in the older age group, stunting was lower in the recuperative areas compared to the preventive (but 93 Figure 7.2 Mean anthropometric outcomes, by child age and program group at final survey Child age (months) 12-23 24-35 36-41 Height-for-age z-score (HAZ) -2.0 -1.5 -1.0 -0.5 Recuperative (final) Preventive (final) 12-23 24-35 36-41 Weight-for-age z-score (WAZ) -2.0 -1.5 -1.0 -0.5 0.0 12-23 24-35 36-41 Weight-for-height z-score (WHZ) -2.0 -1.5 -1.0 -0.5 0.0 P<0.05 P<0.05 P<0.01 P<0.05 P<0.01 P=0.16 P=0.18 Notes: Random effects linear regression models adjusted for child age and sex were used to assess statistical significance of differences between program communities for each age group. P values are for differences between preventive and recuperative at final survey. Numbers in figure are adjusted means derived from the random effects models. 94 Figure 7.3 Prevalence of stunting, underweight, and wasting, by child age and program group at final survey 12-23 24-35 36-41 Stunting (%) 0 10 20 30 40 50 Recuperative Preventive 12-23 24-35 36-41 Underweight (%) 0 10 20 30 40 50 Child age (months) 12-23 24-35 36-41 Wasting (%) 0 10 20 30 40 50 P<0.1 P=0.1 P>0.2 P<0.1 P<0.05 P>0.2 P<0.05 P<0.1 P>0.2 Notes: Random effects logit models adjusted for child age and sex were used to assess statistical significance of differences between program communities for each age group. P values are for differences between preventive and recuperative at final survey. Numbers in figure are adjusted prevalences derived from the random effects logit models 95 nonsignificant, p>0.5). For wasting, larger differences were seen among the 12-23 month age group (4.7 percentage points) but not for the other age groups, where differences were similar to the overall differences in wasting between the groups. Since the periods of 12-23 and 24-35 months of age are those of greater nutritional vulnerability, these results are largely as expected. The level of utilization of the different program services in preventive and recuperative communities is discussed in Chapter 6. One point worth highlighting in the context of the impact of the program on nutritional status is the fact there were no differences between program groups in the mothers’ participation during pregnancy and lactation (see Table 6.2, Chapter 6). Differences in participation between the groups start when the child is 6 months of age, as expected, because by design the two programs have different eligibility criteria for children 6 months and older (age for the preventive group; and malnutrition for the recuperative group). Figure 7.4 shows the mean weight-for-age Z-scores of index children (12-41 months) and of their younger siblings (0-11 months). Although the children in the preventive group track above the children in the recuperative group at all ages from 0 until 41 months of age, the differences are stronger after about one year of age. This pattern was expected since mothers in the preventive and recuperative groups received exactly the same package of intervention until the child reached 6 months, and participation in both groups was similar. On average, children Figure 7.4 Weight-for-age Z-scores of index children (12-41 mo) and their younger siblings (0-11), by age and program group at final survey Adjusted mean WAZ by age and program group, for children 0-41 months of age at final survey Age groups (months) 0-5 6-11 12-17 18-23 24-29 30-35 36-41 Weight for age z-scores -1.6 -1.4 -1.2 -1.0 -0.8 -0.6 -0.4 -0.2 Preventive Recuperative Note: Numbers in figure are adjusted means derived from random effects regression models comparing the program communities and adjusting for child age and sex. 96 in the preventive approach entered the program at approximately 7 months of age, and thus, it is highly plausible that differences in children’s growth between the two approaches start becoming significant only after this age. A striking finding of the WAZ curves shown in Figure 7.4 is the steep drop in mean WAZ observed during the first year, in spite of the fact that more than half of the mothers in both program areas participated in the program during pregnancy and the first 6 months of lactation. Similar drops were also observed for HAZ (not shown). 7.5.4 Differences in morbidity between the two program groups Figures 7.5-7.8 present the differences between the preventive and recuperative groups in the prevalence of symptoms of infections. Overall, there were no consistent differences between the groups, except for fever, where the recuperative group had a consistently higher prevalence than the preventive group (the differences were statistically significantly different only among the 24-29 month old children, however). Figure 7.5 Percentage of children who had fever in the past two weeks, by age and program group (final survey) Child age (months) 12-17 18-23 24-29 30-35 >=36 All % with fever 0 20 40 60 80 100 Preventive Recuperative * Note: * p < 0.05 using separate random effects logit models for each age group, adjusting for child sex and comparing the program communities. 97 Figure 7.6 Percentage of children with cold/cough in the past two weeks, by age and program group (final survey) Child age (months) 12-17 18-23 24-29 30-35 >=36 All % with cold/cough 0 20 40 60 80 100 Preventive Recuperative Note: Statistical significance was tested using separate random effects logit models for each age group, adjusting for child sex and comparing the program communities. Figure 7.7 Percentage of children with diarrhea in the past two weeks, by age and program group (final survey) Child age (months) 12-17 18-23 24-29 30-35 >=36 All % with diarrhea 0 20 40 60 80 100 Preventive Recuperative Note: Statistical significance was tested using separate random effects logit models for each age group, adjusting for child sex and comparing the program communities. 98 Figure 7.8 Percentage of children with fast breathing in the past two weeks, by age and program group (final survey) Child age (months) 12-17 18-23 24-29 30-35 >=36 All % with fast breathing 0 20 40 60 80 100 Preventive Recuperative Note: Statistical significance was tested using separate random effects logit models for each age group, adjusting for child sex and comparing the program communities. 7.6 Summary of Results 7.6.1 Probability results ™ There were no differences between the two program communities at baseline in mean anthropometric indicators or in the prevalence of stunting, underweight, or wasting. ™ At final survey, children in preventive communities had higher mean HAZ, WAZ, and WHZ, and lower prevalence of stunting, underweight, and wasting than children in recuperative communities. All differences were statistically significant. ™ The differences in mean HAZ, WAZ, and WHZ (adjusted for child age and gender and clustering effect) in favor of the preventive group were +0.14, +0.24, and 0.24, respectively. For the prevalence of stunting, underweight, and wasting, differences in favor of the preventive group were 4, 6 and 4 percentage points, respectively. The magnitude of these effects is modest, but comparable to effects seen in other studies 99 that compared baseline and post-intervention or intervention and control group at the end of an intervention (Caulfield, Huffman, and Piwoz 1999; Swindale et al. 2004). 7.6.2 Plausibility results ™ Compared to baseline, children’s nutritional status appears to have deteriorated among the recuperative group, especially with regards to the prevalence of underweight and wasting; by contrast, wasting remained the constant over time in the preventive group, and both underweight and stunting decreased over time. These results suggest that the preventive approach may have helped mitigate the deleterious effects on childhood malnutrition of the economic and political crisis that occurred in Haiti during the study period. ™ Children who were exposed to the preventive approach for the entire period between 6 and 23 months (i.e., who were 24 to 35 months at final survey) benefited more from the intervention than children who were only partially exposed. This was particularly true for mean HAZ, WAZ, and WHZ, and underweight. For stunting, children 12-23 months old and 24-35 months old showed larger benefit, while for wasting, children 12-23 months old showed the largest benefit. ™ Analysis of younger siblings of index children (i.e., infants 0-11 months of age at the final survey) shows no difference between the preventive and recuperative groups - a result that was expected, given that the two programs offered exactly the same preventive services to mothers during pregnancy and the first 6 months of lactation. Neither program approach, however, succeeded in preventing the steep decline in mean anthropometric indicators during the first year of life. ™ No differences in morbidity symptoms were found between the preventive and recuperative groups, with the exception of fever, which was less prevalent among children 24-29 months of age in the preventive group. 7.7 Conclusions The study confirms that the preventive program approach, which targets food assistance and BCC to all children 6-24 months, is more effective at reducing undernutrition than the recuperative approach, which targets malnourished children younger than five years of age. The evidence of the greater effectiveness of the preventive over the recuperative program approach is strong from a probability perspective. It is also highly plausible, given the larger impact found among children fully exposed to the program, compared to those who were exposed only partially during the period of greatest nutritional vulnerability (i.e., when they were 6-23 months of age). The following three chapters explore some of the intermediary mechanisms that may be responsible for the greater effectiveness of the preventive compared to the recuperative approach. These include changes in maternal knowledge and practices related to child feeding and care (Chapter 8), improved household food security (Chapter 9), and other potential household-level socioeconomic benefits (Chapter 10). 100 101 8. IMPACT OF THE PROGRAM ON KNOWLEDGE AND PRACTICES22 8.1 Introduction This chapter describes the impact of the program on the anticipated outcomes of the behavior change communications activities, i.e., caregiver knowledge and practices. In examining the impact of the program on these outcomes, we attempt to track different stages leading to sustained behavior change, moving from knowledge to trial and adoption of recommended child-feeding and care practices, to childcare practices reported by mothers in the final survey. Throughout, we maintain a focus on the feeding and care practices promoted in the Mothers’ Club (MC) learning sessions described in Chapter 4, which constituted the main venue for delivery of the BCC. 8.2 Data and Variable Creation 8.2.1 Maternal knowledge We assessed caregiver knowledge related to infant and child-feeding practices using the knowledge questions from the baseline survey instrument as well as a more extensive knowledge test that was developed for the operations research (OR) study conducted in 2004. The questions from the baseline survey focused on knowledge about the ideal duration of breastfeeding, the appropriate introduction of complementary foods, and appropriate feeding frequency for children in different age groups. The knowledge test from the OR-2004 questionnaire was specifically designed to assess the respondent’s knowledge about topics taught at the MCs and Rally Posts (RPs). Thus, in addition to questions regarding infant and young child feeding, this knowledge test also included more general health-related questions such as prevention of HIV infection, danger signs in pregnancy, prevention of worms, hand-washing practice, etc. Using both of these sets of questions, we constructed a variety of scales to capture respondent knowledge about child feeding and care. These are described below. 8.2.1.1 Infant and Young Child-feeding Knowledge Test (same as in baseline questionnaire) The survey instrument included a module on appropriate timing of introduction of complementary foods and one on appropriate feeding frequency for infants and children of different ages. Both are described briefly below. Knowledge of timing of introduction of complementary foods. The questionnaire assessed respondents knowledge related to the introduction of liquids and foods from six different food groups (water/liquids, semisolids, staple foods, vegetables, eggs, and meats) to infants, using types of liquids and foods commonly fed to infants and young children in this area (Menon et al. 2002b). Respondents were asked when (in months after the birth of an infant) they thought it was appropriate to introduce each liquid or food. The data on the age of introduction of 22 Mary Arimond led the writing and analysis for sections pertaining to infant feeding practices in this chapter. Purnima Menon and Mduduzi Mbuya conducted analysis and writing pertaining to maternal knowledge, as well as awareness, trial, and adoption of recommended practices. 102 individual liquids and foods were then used to create a summary scale of the overall knowledge of a respondent about appropriate introduction of foods, using the current PAHO/WHO Guiding Principles on complementary feeding to define appropriateness (PAHO/WHO 2003). For each food group, the scoring distinguished between appropriate introduction (i.e., at between 6 and 8 months of age) and introduction that was either too early (before 6 months of age) or too late (9 months or older). Introduction of any of the foods in the food group at the appropriate time was assigned a score of 1, and introduction of any of the foods in the inappropriate window was assigned a score of 0. The scores for a total of six food groups were then added up to create a summary knowledge scale that ranged from a possible minimum of 0 to a maximum of +6. Details regarding the scoring are provided in Annex 8.1. Knowledge of appropriate feeding frequency. Six questions in the feeding knowledge questionnaire asked respondents about their knowledge of the appropriate frequency of feeding meals and snacks to infants and young children in three different age groups (6-8 months, 9-11 months, and 12-23 months). These data were combined to create an overall scale that assessed the knowledge of appropriate feeding frequency. The scoring of the variables was based on current age-specific recommendations on the frequency of feeding complementary foods to breastfed children between 6 and 23 months of age (PAHO/WHO 2003). The detailed scoring is presented in Annex 8.1. Based on this scoring, respondents whose answers for each age group matched or exceeded the currently recommended meal and snack frequency for that age group received a score of 1, while those whose responses indicated a lower frequency than the recommendations received a score of 0. The scale thus created went from a possible minimum of 0 to a maximum of +6. Overall feeding knowledge (weighted scale). The two scales described above— knowledge regarding introduction of new foods and knowledge of age-appropriate feeding frequency—were combined with information on knowledge about the optimal duration of breastfeeding to create a scale to assess overall feeding knowledge. The two 6-point scales were each divided by 3, to contribute up to 2 points each to the overall feeding knowledge scale; the overall scale ranged from 0 to +6. The respondents’ knowledge about the optimal duration of breastfeeding was assessed by asking then how many months they believed infants and young children should be breastfed. Responses to this question were also coded based on the current recommendations for feeding infants and young children, i.e., that children should be breastfed up to at least 24 months of age. Responses that were further away from 24 months received lower points than responses closer to 24 months, e.g., a response between 0 and 5 months received -2 points, while a response of 24 months and beyond received the maximum of +2 points. The points on breastfeeding knowledge were summed with the points on the two other weighted scales to create an overall feeding knowledge scale that ranged from a possible minimum of 0 to a possible maximum of +6. As with the previous feeding knowledge scales, the detailed scoring for this overall scale is presented in Annex 8.1. 103 8.2.1.2 General Nutrition and Health Knowledge Test - based on topics covered at MCs and RPs Box 8.1 presents the questions asked on the knowledge test related to the health and nutrition topics topics covered at the MCs and RPs. Responses were evaluated relative to the discussion of those topics at the MCs (or, where relevant, the RPs). Each variable was scored to contribute one point to an overall knowledge score. For most open-ended questions, we assigned one point on the question/topic if even one correct response was provided by the mother. For a few questions, certain responses were considered much more appropriate or important than Box 8.1 Knowledge test questions on topics covered at MCs and RPs a. Breastfeeding knowledge (maximum score: 8) - Until what age do you think you should breast feed your child? - How long after birth should a baby start breastfeeding? - What should a mother do with “first milk” or colostrum? - What should a mother if she thinks her baby is not getting enough BM? - Should infants < 6 be given water in addition to BM if the weather is hot? - What should the baby < 6 months be fed if mother needs to be away? - Can a mother who is not well fed produce enough breast milk? - What are the things you can do to increase milk production? b. Complementary feeding knowledge (maximum score: 9) - What are the special foods mothers could make to complement breast milk? - Will feeding a child enriched gruel in the evening cause indigestion? - How many times per day should a child 6-8 months old eat (meals and snacks)? - How many times per day should a child 9-11 months old eat (meals and snacks)? - How many times per day should a child ≥ 12 months old eat (meals and snacks)? - At meal times, how much food should a child 6-9 months old be offered? - At meal times, how much food should a child ≥ 12 months old be offered? - Should a one-year old child eat only the same foods as the rest of the family? - What are some of the things you can do to encourage young children to eat food? c. Knowledge about child illness (maximum score: 4) - Do children need an extra meal per day after they have been sick? - For how long do children need an extra meal after they have been sick? - What should you do when your child has diarrhea? o Oral rehydration salts (ORS) o Food and diet-related answers d. Knowledge about general health issues (maximum score: 5) - What can you do to protect a child against polio? - What can you do to protect a child against HIV/AIDS? - When should you wash your hands? - How can you protect a child from getting worms? - How can you make drinking water safe? e. Knowledge about malnutrition (maximum score: 2) - Why do you think children get malnourished? - How can we help malnourished children recuperate? 104 others; in such cases, the most appropriate response(s) was assigned a full score (1 point), and the less appropriate response(s) was assigned a partial score (0.5 point). And finally, for the question that asked about the use of the lactation amenorrhea method (LAM), only those respondents who mentioned all three necessary criteria for LAM were assigned a point. The coding of correct responses to the questions is shown in Annex 8.2. The scores for a total of 28 questions were then added up to create a summary knowledge scale that ranged from a possible minimum of 0 to a maximum of 28. The total score on the knowledge scale was also transformed into a variable that provided information on the proportion of the 28 questions the respondent had answered correctly. We also created more focused knowledge scales to capture respondent knowledge about specific topics, i.e., on breastfeeding, complementary feeding, feeding sick children, general health, and malnutrition (using the questions shown in Box 8.1, a through e). 8.2.2 Trial and adoption of key recommended practices The BCC strategy was developed comprehensively to improve overall infant and young child-feeding practices. At the same time, it promoted certain specific feeding practices and recipes that were developed based on the formative research and recipe trials described in Chapter 4 (Menon et al. 2002b). Based on the theory that behavior change occurs in stages (Prochaska and DiCelmente 1983), we used qualitative and quantitative approaches to understand the stages through which mothers exposed to new practices would have to progress for the program to have an impact, i.e., being aware of the new practice, then trying it, and finally, adopting the new practice. Thus, we first examined awareness, trial, and adoption of these key new practices using qualitative research methods in the operations research study described in Chapter 5 and in more detail in Menon et al. (2005). We then designed a survey module for examining awareness, trial, and adoption based on the results and insights from this qualitative research for inclusion in the final impact survey. Focusing on seven key recommended practices and new recipes (Box 8.2), we asked respondents who had been exposed to the MCs if they had ever heard about the practice at the MCs, whether they had ever tried it at home and if so, how often they had tried it. We also asked about reasons for never trying a practice or for not adopting it, defining “adoption” as doing the practice more than just 1 or 2 times. Furthermore, for those women who had heard about the practice at the MCs, we also asked if the practice had ever been demonstrated to them, since the qualitative research about trial and adoption showed that demonstrations at the MCs encouraged trial of the practice at home. For women who had never been exposed to the MCs, we asked the same sequence of questions about awareness, trial, and adoption, and reasons for not trying a practice or not practicing it more than a few times. However, instead of asking about whether the practice had been demonstrated to them in the MCs, we asked where they might have heard about the new practices recommended by the program. This latter question was intended to provide some ideas about the potential diffusion of the program messages outside of the MCs. 105 Box 8.2 From awareness to trial and adoption: Key practices included in the survey - Leaving expressed breast milk behind for baby when leaving the house - Using expressed breast milk in gruels - Adding an egg to the child’s portion of gruel - Preparing enriched gruel with beans and groundnuts - Preparing mashed plantain with added pumpkin - Feeding enriched gruel in the evenings - Feeding an extra meal for two weeks after the child recovered from an illness 8.2.3 Child-feeding and care practices 8.2.3.1 Indicators pertaining to the Guiding Principles for feeding infants and young children The material covered in the MC learning sessions closely followed current international “Guiding Principles” for infant and young child feeding (PAHO/WHO 2003; WHO 2005). Our baseline and final surveys were designed to capture information reflecting practices taught and promoted in MCs and summarized in the Guiding Principles (see Annex 8.3 for Guiding Principles for feeding breastfed (a) and non-breastfed (b) children). Table 8.1 lists the Guiding Principles, and related recommended practices, as well as data collected and indicators constructed to assess them. Practices related to some recommendations are not easily captured via maternal recall in simple surveys. For example, our survey did not include quantitative data collection on consistency and quantity of food offered/eaten, nor on energy and nutrient density of foods. Survey responses on recall of feeding and care practices may also be biased, when respondents overreport good practices. Data on hygiene practices are particularly susceptible to this type of bias (Ruel and Arimond 2002). For some practices that are difficult to capture via recall, proxy measures are identified in Table 8.1. 8.2.3.2 Constructed indicators for child feeding Results for most of the indicators listed above and in Table 8.1 are presented as simple percentages reflecting responses to single questions. A few exceptions (constructed indicators) are briefly described below. Exclusive breastfeeding (last 24 hours). The percent of children exclusively breastfed in the last 24 hours was calculated by considering responses to a series of questions asking caregivers if their child had been given any of a long list of liquids or solid food groups the previous day. If the child was reported to be still breastfeeding but not to consume any of the other liquids/foods listed, s/he was categorized as exclusively breastfed. For those so categorized, we cross-checked against other variables in the data set (recalled age of introduction of various liquids/foods, and frequency of feeding in the past 24 hours). In all cases, the three sets of questions were consistent. Note that exclusive breastfeeding in the last 24 hours is not equivalent to exclusive breastfeeding since birth (which is much harder to ascertain by recall). However, for purposes of comparing between groups, and assessing changes since baseline, 106 Table 8.1 Guiding Principlesa and recall data collected on recommended practices Guiding Principle – aspect of feeding Guidance/recommended practices discussed in MCs Indicators used in Haiti survey Duration of exclusive breastfeeding (BF) and introduction of complementary foods - Use good BF practices on day of child’s birth (initiate BF in first hour, give colostrum, practice exclusive BF) - BF exclusively for 6 mo; introduce complementary foods (CF) at 6 months of age while continuing to breastfeed % initiating BF within one hour of birth; % exclusively BF; % using colostrum % of children 0-5.9 mo exclusively BF in past 24 hours Recalled age of introduction of liquids and semi-solid and solid foods Maintenance of breastfeeding Continue frequent, on-demand BF until 2 years and beyond % BF in past 24 hour by age group Responsive feeding Feed infants directly, assist older children; if child refuses foods, use different methods of positive encouragement % reporting use of positive strategies (caressing, playing, offering other choices of food) when child refuses to eat Safe preparation and storage of foods Practice good hygiene and proper food handling; this includes hand washing and avoidance of baby bottles % “usually” using baby bottles Amount of complementary food needed “Ensure child energy needs are met.” Not measured because requires total energy intake from food and breast milk No indicator Food consistency Increase food consistency and variety as infant gets older; from 6 months on, infant can eat pureed, mashed, and semi-solid foods) No direct measurement of consistency; recipes promoted and practiced in Mothers’ Clubs were of appropriate consistency % children given semi-solid foods beginning at 6-6.9 mo % fed recipes promoted in MCs (past 24 hr, 3+ or 7+ times last wk) Meal frequency and energy density Increase number of times child is fed complementary foods as s/he gets older; BF children, feed: 2-3 times for 6-8 mo; 3-4 times for 9-23 mo; Non BF, feed: 3-4 times for 6-8 mo; 4-5 times for 9-23 mo Energy density: no direct indicators, but promoted recipes were formulated to be of adequate energy density % children fed at least minimum recommended number of meals of solid/semi-solid foods yesterday Mean number of meals; mean number of snacks % fed recipes promoted in MCs (past 24 hr, 3+ or 7+ times last wk) Nutrient content of complementary foods Feed children a variety of foods including nutrient-rich food groups (e.g., animal source foods and vitamin A-rich food) daily. Promoted recipes were formulated to be nutrient-dense % fed foods from 8 food groups yesterday Mean dietary diversity % fed recipes promoted in MCs (past 24 hr, 3+ or 7+ times last wk) Use of vitamin-mineral supplements or fortified products for infant and mother Use as needed for child; BF mothers may also need supplements In areas where vitamin A deficiency is prevalent or under-five mortality rate is > 50/1000, give high-dose vitamin A supplements twice annually % eating fortified WSB (past 24 hr, 3+ or 7+ times last wk); age first received WSB and number of months received % women who received prenatal iron and postnatal vitamin A supplements; % children who received vitamin A in past 6 mo Feeding during and after illness Increase fluid intake during illness, including more frequent BF. After illness, give more food, more often % giving more liquids during diarrhea; % giving the same (or more) food during diarrhea; % giving extra meals after diarrhea a Guiding Principles for feeding infants and young children are described in PAHO/WHO 2003 (for breastfed children) and WHO 2005 (for non-breastfed children). 107 exclusive breastfeeding in the last 24 hours is a useful measure and is widely used in nationally representative surveys such as the Demographic and Health Surveys. Fed minimum recommended number of meals of solid/semi-solid foods. Respondents reported the number of times during the morning, at noon, afternoon, evening, and night that children were fed solid and/or semi-solid foods. These were summed and the total compared to recommendations. For breastfed children aged 6-8.9 mo, 2-3 meals are recommended, plus snacks “as desired,” and for those aged 9-23.9 mo, 3-4 meals are recommended, plus snacks (PAHO/WHO 2003). These recommendations assume an energy density of at least 0.8 kcals/gram for complementary foods, and also assume that children are fed to gastric capacity at meals. Formative research done in the study areas confirmed that the energy density of commonly prepared complementary foods was adequate (Ruel et al. 2004). All promoted enriched recipes also had an energy density of 0.8 or greater. For non-breastfed children, 4-5 meals are recommended, again with 1-2 snacks as desired (WHO 2005). We constructed a dichotomous variable indicating whether or not each child received at least the minimum number of meals for his/her age and breastfeeding status. Dietary diversity. A dietary diversity score was calculated, identical to one used at baseline, and summing the following eight food groups: grains (and all gruels made from grain); roots and tubers; legumes; vitamin-A rich fruits and vegetables; other fruits and vegetables; dairy products (including infant formula); flesh foods and eggs (including meat, organ meat, poultry, fish, seafood, and eggs); and nuts. Each food group eaten yesterday provided 1 point, yielding a diversity score ranging from 0-8. For selected nutrient-dense foods, and for recipes promoted in MCs, we also created dichotomous indicators reflecting whether or not the child had received these foods at least 3 times in the previous week, and whether they had received them at least 7 times in the previous week. IYCF indicator: An IYCF indicator was created using data on breast feeding and key aspects of complementary feeding for children between 6 and 23 months of age. The indicator was modeled after the IYCF indicator developed for use in the Demographic and Health Surveys (Mukuria, Kothari, and Abderrahim 2006), with one adaptation. In our survey, data were not gathered on consumption of foods containing fats and oils, and therefore, this variable was not available for inclusion in the index. Nevertheless, an indicator was created that captured the three main components of the IYCF indicator: whether a child received breast milk or other calcium rich foods if not breast fed, whether the child was fed solid/semi solid foods a minimum number of times per day, and whether the child was fed an age-appropriate minimum number of food groups (dietary diversity). The dietary diversity variable was slightly different from the DHS IYCF indicator; in our IYCF indicator, we separated nuts from legumes (in effect substituting the variable on nut consumption for the variable on fats/oils consumption). Thus, the variable on dietary diversity that was created for the IYCF indicator had seven food groups, exactly the same number as the DHS IYCF indicator, but one of the variables was different between the two. 108 8.2.4 Other care practices: Preventive care, hygiene practices, and care during illness Families who participate in WV-Haiti programs are encouraged to care for their children by accessing RP preventive health-care services such as immunizations, and by using good hygiene practices. Parents are encouraged to seek help from the health agent or other medical professional when children are ill, and to use the ORS that is distributed at RPs when children have diarrhea. These caregiving and care-seeking practices are promoted within the larger MCHN program, and thus for all children in both preventive and recuperative areas. The following indicators were used to measure these other care practices: 1. Percent of children 12 mo and older who were fully immunized; 2. Percent of ill children (last two weeks) for whom medical advice or care was sought, by symptom (fever, respiratory symptoms, diarrhea); 3. Percent of children with diarrhea who were given ORS or homemade sugar-salt solutions. Child fully immunized. The percent of children fully immunized was constructed from information about individual vaccines received. This information was available from health cards and/or from mother’s recall. Information from these two sources is presented separately (with recall data reported only for those who did not have health cards) and is then combined for an overall estimate. Children are considered to be fully immunized if, at ages 12 mo and older, they have received the following vaccines: BCG, 4 polio, 3 DTP, and measles (WHO 2002). Hygiene scale scores. Spot observations of child and maternal cleanliness, as well as cleanliness of the interior and exterior of the house were used as proxy measures of hygiene practices. Observers assessed the cleanliness of the child’s hands, face, clothes, and hair, and rated each on a scale of 1-3. For children who were naked at the time of observation, the cleanliness of their body was assessed instead of cleanliness of clothes. The observer also noted yes/no for whether the child had an “unattended” runny nose. A child cleanliness scale was constructed using these variables to be identical to the scale used at baseline. The possible range for the scale was 4-13. A similar scale was constructed for maternal cleanliness (range 4-12), including four of the same items: cleanliness of hands, face, clothes, and hair, each scored from 1 to 3. Two scales were constructed for cleanliness of the area around the house (general appearance of the compound, was the compound swept, and were garbage, animal feces, or human feces absent from the compound), and the cleanliness of the interior of the house (general appearance, was the floor swept, were drinking water containers covered, and were dirty clothes absent). These two scales ranged from 1 to 7 (exterior) and 1 to 6 (interior). 109 8.3 Analysis The analyses presented in this chapter are meant to describe current practices. As the overall study was designed to compare the preventive and recuperative approaches, statistical tests are reported primarily for these comparisons, consistent with the study design and sampling. Most results are presented for index children (12-41 months of age), but for a few variables that are relevant mainly for younger children (e.g., exclusive breastfeeding during the first 6 months), we include results for the sample of younger siblings. Note, however, that the statistical power for this sample of younger siblings is much lower than for the index children because the sample size of younger siblings is only 341. For data on immunization coverage, care-seeking, and hygiene, baseline information is available only from 18 months onward, so for comparability we present this information for children aged 12-41 months. In order to account for the survey design, tests of differences in means and proportions were performed in STATA using xtreg (continuous variables) or xtlogit (dichotomous variables). This allows specification of the paired cluster design and provides the most appropriate standard error for these tests. P-values of less than 0.05 are considered to be significant. For some topics/variables, we also examined differences between program participants and nonparticipants using two measures of participation: (1) whether the mother had ever participated in the MCs or not; and (2) whether the child was currently in the program and therefore receiving food assistance commodities.23 These comparisons are presented mainly to provide additional insight about the plausibility of our key results; they are not part of our main analysis, which focuses on an “intent to treat” approach. Finally, comparisons are also made between breastfed and non-breastfed children for key relevant variables. Comparisons with baseline results are often presented in our results, but they are meant to be interpreted qualitatively—i.e., they are presented to provide a sense of the initial levels of the different variables assessed in the final survey. No statistical testing is done to compare baseline and final survey results. 8.4 Results 8.4.1 Impact of the program on maternal knowledge We examined differences between the program groups on a variety of maternal knowledge scales, capturing both overall maternal health and nutrition knowledge, and knowledge on specific groups of topics taught at the Mothers’ Clubs. Infant and Young Child-feeding Knowledge Test: At baseline, the program communities were not different on any of the knowledge variables. At the final survey, respondents in the preventive community scored higher than those in the recuperative group on knowledge tests 23 These participation variables are slightly different from the ones used in Chapter 6. This is because we viewed mother’s exposure to MCs as the key program exposure for changes in practices. For current participation, we wanted to assess the relationship between current receipt of food and selected practices. 110 related to feeding frequency and on the overall knowledge scale; differences were statistically significant, but of small magnitude (Table 8.2). Notably, results for the three knowledge tests for which data were also available at baseline were markedly higher at the final survey, particularly among respondents who had ever participated in the program (Table 8.3). Within program groups, the differences between respondents who had ever been exposed to the program and those who had not were large and statistically significant. Knowledge scores among those never exposed to the program were close to baseline values, while scores for those who had been exposed were higher. This was true for both program groups, which shows that exposure to the BCC strategy included in both program approaches had a large impact on maternal knowledge. General Nutrition and Health Knowledge Test of BCC topics: The knowledge scores based on a test of topics taught at the MCs are slightly higher (and statistically significant) among preventive community respondents for overall health and nutrition knowledge and knowledge pertaining to childhood illness. All other scores are almost identical between the two groups and not statistically different. Table 8.3 shows, as would be expected, that overall and topic-specific knowledge is quite different for mothers who never participated in the program compared to those who had participated. Although this was true for both program groups, differences between participants and nonparticipants were larger in the preventive group; participating mothers from preventive communities had higher overall knowledge scores and higher scores to the breastfeeding and complementary feeding knowledge tests (Table 8.3). Trends for proportion of knowledge questions answered correctly were the same as for the knowledge scores. Multivariate analyses show a statistically significant interaction between having ever participated in the program and program group (preventive versus recuperative) (Table 8.4). The significant interaction confirms findings from bivariate analyses described above that suggest that a larger effect of program participation on overall knowledge score is found in preventive compared to recuperative communities. 111 Table 8.2 Maternal nutrition and health knowledge, by program group Baseline Final survey Preventive Recuperative Overall (n = 748) (n = 752) (n = 1,500) Knowledge scores Mean SD Mean SD Mean SD Mean SD Infant and Young Child-feeding Knowledge Test Knowledge about appropriate introduction of foods (maximum score: 6) 2.4 1.7 4.9 1.7 4.9 1.7 Appropriateness of feeding frequency knowledge (maximum score: 6) 5.4 0.9 5.7 0.6 5.6** 0.7 Feeding knowledge, weighted (maximum score: 6) 4.3 0.7 5.4 0.7 5.3** 0.7 Overall Nutrition and Health Knowledge Test (BCC topics) Overall health and nutrition knowledge (maximum score: 28) n/a 20.1 2.9 19.8** 2.8 19.9 2.9 Breastfeeding knowledge (maximum score: 8) n/a 4.4 1.3 4.3 1.2 4.4 1.2 Complementary feeding knowledge (maximum score: 9) n/a 8.4 1.0 8.3* 1.0 8.3 1.0 Knowledge about child illness (maximum score: 4) n/a 2.4 0.9 2.3** 0.9 2.3 0.9 Knowledge about general health issues (maximum score: 5) n/a 3.3 1.0 3.2 1.0 3.2 1.0 Knowledge about malnutrition (maximum score: 2) n/a 1.7 0.5 1.7 0.5 1.7 0.5 Percentage of answers correct on each of the scores Percentage of answers correct on overall knowledge test n/a 71.7 10.6 70.6* 9.9 71.2 10.2 Percentage of answers correct on breastfeeding questions n/a 55.0 15.8 53.8 14.6 54.4 15.2 Percentage of answers correct on complementary feeding n/a 92.9 10.6 92.0* 11.3 92.5 11.0 Percentage of answers correct on child illness n/a 65.2 20.6 63.7 20.8 64.5 20.8 Percentage of answers correct on general health n/a 58.9 21.7 56.6** 23.0 57.8 22.4 Percentage of answers correct on malnutrition n/a 85.1 25.4 87.0 23.9 86.1 24.7 Note: * 0.05< p < 0.1; ** p < 0.05; *** p < 0.01. 112 Table 8.3 Maternal nutrition and health knowledge, by program group and participation (ever participated) Baseline Preventive Recuperative (n = 1,524) (n = 126) (n = 622) (n = 126) (n = 626) Nonparticipant Participant Nonparticipant Participant Nutrition and health knowledge Mean SD Mean SD Mean SD Mean SD Mean SD Infant and Young Child-feeding Knowledge Test Knowledge about appropriate introduction of foods (maximum score: 6) 2.4 1.7 3.4 2.3 5.2 a 1.4 4.1 2.2 5.1 a 1.5 Appropriateness of feeding frequency knowledge (maximum score: 6) 5.4 0.9 5.6 0.7 5.7 a 0.6 5.5 0.8 5.7 a 0.6 Feeding knowledge, weighted (maximum score: 6) 4.3 0.7 4.7 0.8 5.5 a 0.6 4.9 0.9 5.4 a 0.6 Overall Nutrition and Health Knowledge Test (BCC topics) Overall health and nutrition knowledge (maximum score: 28) n/a 16.6 3.5 20.8 a 2.2 17.9 3.4 20.2 a 2.5 Breastfeeding knowledge (maximum score: 8) n/a 2.7 1.5 4.7 a 0.9 3.3 1.4 4.5 a 1.0 Complementary feeding knowledge (maximum score: 9) n/a 7.5 1.3 8.5 a 0.8 7.9 1.3 8.4 a 0.9 Knowledge about child illness (maximum score: 4) n/a 1.9 0.9 2.4 0.8 2.1 1.0 2.3 0.9 Knowledge about general health issues (maximum score: 5) n/a 2.9 1.2 3.3 1.0 3.0 1.1 3.2 1.0 Knowledge about malnutrition (maximum score: 2) n/a 1.5 0.7 1.7 0.5 1.6 0.6 1.8 0.4 Percentage of answers correct on each of the scores n/a Percentage of answers correct on overall knowledge test n/a 59.2 12.6 74.3 a 8.0 63.9 12.2 72.0 a 8.8 Percentage of answers correct on breastfeeding questions n/a 34.2 18.4 59.3 a 11.2 41.6 18.0 56.4 a 12.4 Percentage of answers correct on complementary feeding n/a 83.7 14.9 94.8 a 8.4 87.4 14.6 92.9 a 10.3 Percentage of answers correct on child illness n/a 57.9 24.0 66.7 19.6 60.0 21.3 64.5 20.6 Percentage of answers correct on general health n/a 48.0 23.4 61.2 20.7 51.7 25.3 57.6 22.4 Percentage of answers correct on malnutrition n/a 74.2 33.5 87.3 22.9 81.5 31.5 88.2 21.9 a Difference between participants and nonparticipants (within program group) is significant at p < 0.05 (random effects regression analysis). 113 Table 8.4 Program impact on maternal nutrition and health knowledge (regression analysis) (1) Bivariate model (2) Main effects, adjusted (3) Main effects, ever participated (4) Interaction model, ever participated Knowledge Knowledge Knowledge Knowledge Program -0.31* -0.32* -0.34* 1.06** (0.15) (0.16) (0.14) (0.36) Respondent’s education (years) 0.12** 0.12** 0.12** (0.03) (0.03) (0.03) Partner’s education (years) 0.03 0.04 0.04 (0.03) (0.02) (0.02) Ever participated 3.31** 4.13** (0.20) (0.28) Ever participated x Program -1.66** (0.40) Constant 20.09** 19.86** 17.07** 16.37** (0.15) (0.17) (0.24) (0.29) Observations 1,500 1,250 1,250 1,250 Number of pairs 10 Notes: Standard errors in parentheses. * significant at p < 0.05; ** significant at p < 0.01. In summary, results of the impact of the two program approaches on maternal knowledge show that: For Infant and Young Child-feeding Knowledge Test: ™ There were no differences between program communities in maternal knowledge at baseline. ™ At the final survey, mothers from the preventive program communities had significantly higher scores on the overall feeding knowledge scale and the knowledge about feeding frequency tests than mothers from the recuperative program communities; all differences were of small magnitude, however. ™ Both preventive and recuperative groups showed markedly greater nutrition knowledge at final survey compared to baseline. ™ Attained knowledge at final survey was significantly higher among women who had participated in the program at some point in the previous 3 years, compared to those who had never participated. For the General Nutrition and Health Knowledge Test – based on MC and RP topics ™ Maternal knowledge about health and nutrition topics taught at the MCs was good, with respondents getting about 71% correct answers on the general nutrition and health knowledge test. 114 ™ Respondent knowledge about BCC topics was statistically significantly better in preventive communities compared to recuperative communities in overall scale as well as subscales related to breastfeeding and complementary feeding, but differences were of small magnitude. ™ More meaningful and significant differences in knowledge were seen between respondents ever exposed to the program versus those never exposed to the program. Differences between exposed and nonparticipant respondents were greater in the preventive communities than in the recuperative communities. 8.4.2 Trial and adoption of key recommended practices Figures 8.1a through 8.1g present results related to awareness, trial, and adoption of key recommended practices; Table 8.5 presents comparisons between preventive and recuperative communities in trial of recommended practices and the reasons reported for non-trial of the practices. Data are presented separately for women who had ever participated versus those who had never participated because data collection instruments were slightly different for these two groups of women. Although statistical significance is not depicted in Figures 8.1a through 8.1g, significant results are discussed below, and Annex 8.4 presents the same results in tabular format, with statistical significance indicated in the table. Differences between program groups among participants in awareness, trial, and adoption were statistically significant for almost all practices. Notably, for adoption rates, the differences between participants in preventive and recuperative groups were seen mainly for complementary feeding-related practices, with no significant differences in adoption of the use of expressed breast milk. This conforms to expectations because pregnant and lactating women were targeted in both program approaches, and participation rates were remarkably similar for pregnant and lactating women in the two program groups. Overall, the magnitude of differences in awareness, trial, and/or adoption between the preventive and the recuperative program are of a smaller magnitude than the differences between respondents who had ever participated versus those who had never participated in each of the program groups (also statistically significant for almost all of the practices). The overall pattern suggests that in both program groups, awareness, trial, and adoption are substantially higher among respondents who had ever participated in the program compared to those who had never participated. Practices that had been tried by at least half of the participant mothers were the use of expressed breast milk, adding an egg to the child’s gruel, feeding enriched gruel in the evening, preparing special recipes such as mashed plantain with pumpkin, and feeding an extra meal a day after an illness (Table 8.5). The practices that had been tried by only about one-third of participant respondents were adding breast milk to gruel, and preparing gruel with added beans and nuts. 115 Table 8.5 Differences in trial of recommended practices between preventive and recuperative groups and reasons for non-trial of practices, by program exposure Recommended practice Ever participated in MC Never participated in MC Prev. (N = 622) Recup. (N = 622) Prev (N = 122) Recup. (N = 130) Ever tried Ever tried % % Reasons for non-trial (among those who did not try the practice) % % Reasons for non-trial (among those who did not try the practice) Leaving expressed breast milk behind when going out 65.1 a b 59.3 a ™ Don’t leave home for long (~75%) 22.2 24.6 ™ Don’t know how to do it (~30%) ™ Don’t leave home for long (~50%) Adding breast milk to gruel 31.7 a b 24.0 a ™ Don’t know how to do it (~33%) ™ Afraid child won’t eat it (~10%) ™ It’s disgusting (~33%) 3.2 4.0 ™ Don’t know how to do it (~25%) ™ Afraid child won’t eat it (~15%) ™ It’s disgusting (~45%) Adding an egg to child’s gruel 72.5 b 53.7 a ™ Don’t know how to do it (~23%) ™ No ingredients (~25%) ™ Expensive ingredients (~20%) ™ Negligence/don’t care (~25%) 76.2 70.6 ™ Don’t know how to do it (~25%) ™ No ingredients (~28%) ™ Expensive ingredients (~25%) ™ Negligence/don’t care (~15%) Preparing gruel with beans and nuts 45.5 a b 35.9 a ™ Don’t know how to do it (~35%) ™ No ingredients (~235%) ™ Expensive ingredients (~20%) ™ Negligence/don’t care (~10%) 7.9 4.0 ™ Don’t know how to do it (~28%) ™ No ingredients (~23%) ™ Expensive ingredients (~25%) ™ Negligence/don’t care (~15%) Feeding enriched gruel in the evening 76.7 a b 67.1 a ™ No ingredients (~40%) ™ Expensive ingredients (~10%) ™ Negligence/don’t care (~15%) 14.3 15.9 ™ No ingredients (~50%) ™ Expensive ingredients (~12%) ™ Negligence/don’t care (~12%) Preparing mashed plantain with pumpkin 79.7 a b 67.3 a ™ Don’t know how to do it (~15%) ™ Pumpkin difficult to find (~47%) ™ Pumpkin is expensive (~13%) 36.5 29.4 ™ Don’t know how to do it (~12%) ™ Pumpkin difficult to find (~50%) ™ Pumpkin is expensive (~12%) Feeding an extra meal after illness 90.7 a b 84.0 ™ Child has no appetite (~35%) ™ Not enough food at home (~35%) ™ Not enough time (~10%) 77.8 81.4 ™ Child has no appetite (~35%) ™ Not enough food at home (~35%) ™ Not enough time (~10%) a Difference between participants and nonparticipants (within program group) is significant at p < 0.05 (random effects regression analysis). b Difference between program groups is significant at p < 0.05 for participants (random effects regression analysis). 116 Figure 8.1a From awareness to adoption: Leaving expressed breast milk behind when going out Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 120 Awareness Trial Adoption Participated Never participated Figure 8.1b From awareness to adoption: Adding breast milk to gruel Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 Awareness Trial Adoption Participated Never participated 117 Figure 8.1c From awareness to adoption: Adding an egg to child’s gruel Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 120 Awareness Trial Adoption Participated Never participated Figure 8.1d From awareness to adoption: Preparing gruel with beans and nuts Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 Awareness Trial Adoption Participated Never participated 118 Figure 8.1e From awareness to adoption: Feeding enriched gruel in the evening Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 120 Awareness Trial Adoption Participated Never participated Figure 8.1f From awareness to adoption: Preparing mashed plantain with pumpkin Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 120 Awareness Trial Adoption Participated Never participated 119 Figure 8.1g From awareness to adoption: Feeding an extra meal after illness Preventive-P Recuperative-P Preventive-NP Recuperative-NP Percent 0 20 40 60 80 100 120 Awareness Trial Participated Never participated As expected, trial of recommended practices was markedly lower among nonparticipating mothers. Only two of the recommended practices were reported by more than half of nonparticipant mothers: the addition of an egg to the child’s gruel and feeding an extra meal after an illness. Between 25% and 37% of nonparticipant mothers had tried leaving expressed milk behind for the child and preparing mashed plantain with pumpkin, but for the remaining practices, the trial rates were extremely low. This was likely driven - at least partly - by low awareness rates, which was observed among nonparticipants for most practices, except leaving expressed breast milk when mother is absent and feeding an extra meal after recovery from an illness. Patterns of adoption (defined as engaging in the recommended practice at least 3 or more times) followed the patterns of trial. On average, 2.5 practices had been adopted by participants in the preventive group, compared with 2.1 practices in the recuperative group and less than 1 practice among nonparticipants in the preventive group and 0.5 among nonparticipants in the recuperative group (p < 0.05 for all differences). Figures 8.1a to 8.1g (and Annex 8.4) present results that show the drop-off from awareness to trial and adoption, by exposure to the BCC program and by program approach. Results are shown for seven key practices promoted by the program, and demonstrate that depending on the practice, exposure to the program can either impact the translation from awareness to trial (e.g., for using expressed breast milk), or have a direct influence on awareness itself, as well as on subsequent trial and adoption. 120 For example, the use of enriched gruels in the evening was a completely new practice recommended by the program. In this case it is apparent that exposure to the program led to greater awareness of the practice among participants compared to nonparticipants; for a substantial proportion of participants, this led in turn to trial and adoption. On the other hand, for a practice where the difference in awareness between exposed and nonparticipant groups was not as large, e.g., the use of expressed breast milk, participation in the program appears to better enable the translation of knowledge into action among those exposed to it. Interestingly, nonparticipants were more likely to report trial than awareness of two practices (addition of an egg to the child’s gruel and feeding extra meals after an illness (Figures 8.1c and 8.1g)). The reasons for this are not immediately clear. Reasons for non-trial of practices: As demonstrated in the qualitative study done as part of the operations research in 2004 (Menon et al. 2005), the translation of awareness into trial of a recommended practice is influenced by a variety of social, cultural, and economic factors. Table 8.5 presents findings on the reasons reported for not trying the recommended practices. As was revealed in our previous qualitative work, the reasons for non-trial vary among practices. Since the reasons for non-adoption are largely the same as reasons for non-trial, we do not present those results separately. For practices that related to the use of expressed breast milk, the major reason for not trying the practice was that 75% of participant and 50% of nonparticipant mothers who had not tried the practice reported not leaving the house for long periods of time. Among the nonparticipant mothers, lack of knowledge was also mentioned as a barrier among 30% of those women who did not try the practice. Perceptions that the practice of using expressed breast milk was unclean were a deterrent to trial of adding breast milk to gruel among over a third of participant mothers, and 45% of nonparticipant mothers who had not tried the practice. Very few nonparticipant mothers had ever tried adding breast milk to gruel. For those practices that required the use of resources to purchase nutrient-rich foods, it is apparent that economic conditions and access to ingredients were considerable barriers to moving from awareness to trial. For example, mothers reported not trying recipes that encouraged them to add an egg to the child’s gruel or to use pumpkin along with mashed bananas because they did not have the ingredients at home, or because the ingredients were expensive. Similarly, they reported not feeding the child enriched gruels in the evening or giving an extra meal after the child recovered from an illness because of inadequate resources at home. In addition to resource constraints, poor child appetite was mentioned as a reason for not trying to feed an extra meal per day to the child. An interesting reason for the lack of trial of recommended practices is “negligence,” which is reported by about 10% of women who did not try recommended practices. The word “negligence” is translated literally from the Creole word and since it is a self-reported reason by respondents, it is not clear if it should be taken in the literal sense of the word in English. Without a deeper understanding of the meaning of this word in Haitian culture, we interpret it to mean that these women who reported “negligence” as a reason for not trying a practice simply did not pay attention to the recommendations, or were not motivated to try them at home. Further examination of the characteristics of women who report “negligence” as a reason for not trying practices, combined with future qualitative research, could help explain this construct. 121 Lastly, it should be noted that there were no differences in reasons for not trying recommended practices between those who had ever participated and those who had never participated in the program. A difference was seen only in the case of using expressed breast milk, where about a third of nonparticipant mothers said they had not tried the practice because they did not know how to do it. In summary, results of the impact of the two program approaches on trial and adoption of key practices show that: ™ For most key practices, respondents in preventive program areas were more likely to report awareness, trial, and adoption than were respondents in recuperative areas. In most cases, however, differences were of relatively small magnitude. ™ Differences between those exposed to the program and those not exposed (in each area) were much larger than differences between program areas. ™ Reasons for non-trial and non-adoption of practices were practice-dependant and ranged from economic and sociocultural to child-focused reasons. 8.4.3 Impact of intervention on infant and young child-feeding and care practices Child-feeding and care practices were measured by maternal recall in the final survey. One important caveat of this approach to assessing the impact of a BCC program on practices is the possibility that mothers systematically report what they have learned, which results in an overestimate of the true impact of the program on behavior change. Although this type of error is inevitable, the main thrust of our study is not on assessing change in practices since baseline, but rather on comparing the impact of two different program approaches on outcomes at the end of the study. As in previous chapters, the presentation of results in this section emphasizes differences between preventive and recuperative program communities. While mention is made of baseline results and “changes” since baseline, differences between baseline and final survey should be interpreted as qualitative information - and are not tested for statistical significance. Early infant feeding practices At baseline, there were no differences between program communities in maternal report of early feeding practices (early initiation of breastfeeding; exclusive breastfeeding on the first day; using colostrum). These recommended practices, however, were reported by a relatively small percentage of mothers—less than half of the mothers reported early initiation of breastfeeding and exclusive breastfeeding on the first day, and approximately two-thirds reported giving colostrum to the newborn infant (Table 8.6 and Figure 8.2). In the final survey, the percentage of mothers who reported optimal early infant feeding practices was much higher than at baseline in both the preventive and recuperative program communities, and no differences between the program communities were found for early initiation of BF (65%) and exclusive BF on the first day (90%). There was a small difference in the proportion reporting that they gave colostrum, with more mothers in the recuperative areas reporting this practice compared to preventive areas (88% versus 84%; P < .05). 122 Table 8.6 Child-feeding and care practices, by program group and time of survey Final Survey Feeding/care practice Baseline Preventive Recuperative All children Early child-feeding practice (index child) (n = 1,602) (n = 748) (n = 752) (n = 1,500) - Initiated breastfeeding within 1 hour 36.4 64.6 65.8 65.2 - Only breast milk on first day 47.1 90.0 90.4 90.2 - Gave colostrum 63.9 84.4* 88.3* 86.3 Early child-feeding practices (younger siblings 0-11 mo)) (n = 431) (n = 159) (n = 185) (n = 344) - Breastfed within 1 hour 48.2 84.1 81.9 83 - Only breast milk on first day 55.8 98.1 93 95.3 - Gave colostrum 73.3 95 94.1 94.5 Exclusive breastfeeding in past 24 hour (if < 6 mo) (n = 255) (n = 100) (n = 129) (n = 229) 42.0 93 91.5 92.1 Child age when liquids foods were introduced (index child) - % who started liquid at 6-8.9 mo 22.2 78.7 78.9 78.8 - % who started semi-solid food at 6-8.9 mo 30.9 83.4 82.4 82.9 - % who started solid food at 6-8.9 mo 42.5 67.9 69.1 68.5 - % who started meat at 6-8.9 mo 15.9 65.9 63.4 64.7 - % who started eggs at 6-8.9 mo 38.1 73.4 60.3 71.9 Maintenance of breastfeeding (index child) - % still breastfeeding at 12-17 mo 87.2 89.8 89.8 89.8 - % still breastfeeding at 18-23 mo 36.2 47.1 44.9 46.1 - % still breastfeeding at 24-42 5.4 6.1 4.9 5.5 Responsive Feeding (index child) - Child normally feeds himself (12-41 mo) 83.9 62.0* 68.1* 65.0 - 12-23 mo 57.3 34.9 39.5 37.0 - 24-42 mo 93.5 85.9 87.6 86.8 - Mothers who mention 2+ positive strategies 23.6 64.9* 59.7* 62.3 - Caress him/her 53 55.1 66.5 60.7 - Play with him/her 2.2 25.1 12.4 18.8 - Offer other foods 6.5 4.7 4.5 4.6 - Mothers who mention negative strategies - Force him/her 35.2 14.1 14.8 14.4 Safe preparation and storage of complementary foods - use of baby bottles - % mothers who used baby bottles 39.9 17.5 16.2 16.9 Breastfed children - 12-17 months % fed minimum no. of meals 38.2 60.8 54.2 57.6 Mean no. of meals 2.4 2.8 2.6 2.7 Mean no. of snacks 1.8 2.1 1.9 2.0 - 18-23 months % fed minimum no. of meals 53.9 66.7 56.6 62.4 Mean no. of meals 2.6 2.9 2.6 2.8 Mean no. of snacks 2.2 2.2 2.3 2.2 - 24-42 months % fed minimum no. of meals -- -- -- -- Mean no. of meals 2.6 2.7 2.8 2.7 Mean no. of snacks 2.0 2.8 1.7 2.3 Non-breastfed children (n = 1,373) (n = 475) (n = 509) (n = 984) - 12-17 mo % fed minimum no. of meals 0.0 (n=17) 14.3 10.5 12.5 Mean no. of meals 2.7 2.9 2.7 2.8 Mean no. of snacks 2.0 2.4 2.6 2.5 - 18-23 mo % fed minimum no. of meals 14.1 17.3 12.1 15.0 Mean no. of meals 2.7 2.8 2.8 2.8 Mean no. of snacks 2.5 2.4 2.2 2.3 (continued) 123 Final Survey Feeding/care practice Baseline Preventive Recuperative All children - 24 mo and older % fed minimum no. of meals -- -- -- -- Mean no. of meals 2.6 2.7 2.7 2.7 Mean no. of snacks 2.3 2.3 2.1 2.2 Feeding patterns - evening meal (index child) - Will wake sleeping child for evening meal 20.6 18.6 19.7 19.1 Percent of children who consumed selected food groups in previous 24 hrs (index child) - Cereals 96.3 98.0 97.1 97.5 - Roots, tubers, starchy vegetables 35.0 33.2 37.1 35.2 - Legumes 75.8 87.3 84.2 85.7 - Vitamin A-rich vegetables 95.1 72.2* 67.3* 69.7 - All other fruits and vegetables 49.9 38.4*** 30.3*** 34.3 - Milk and formula 18.9 27.2** 21.1** 24.1 - Meat, fish and egg 88.3 62.5*** 53.6*** 58.1 - Nuts 48.3 23.0 24.1 23.5 Mean dietary diversity (index child) - 12-17 mo 4.8 4.4* 4.0* 4.2 - 18-23 mo 5.0 4.5 4.2 4.3 - 24-35 mo 5.1 4.4 4.2 4.3 - 36-41 mo 5.1 4.4 4.1 4.2 - All index children: 12-41 mo 5.1 4.3*** 4.1*** 4.3 Percent of children who consumed selected animal source foods (index child) - Eggs 42.6 20.7* 16.2* 18.5 - Chicken 25.1 7.8 5.6 6.7 - Fish and seafood 65.8 34.7 30.2 32.4 - Beef and pork 64.9 28.4*** 21.0*** 24.7 - Heart and liver 18.3 5.0 4.1 4.5 Frequency of consumption of nutrient-rich foods (index child) - Food/groups consumed 3 or more times in the last 7 days - Eggs 8.4 14.3 12.1 13.2 - Flesh food 34.8 72.6*** 60.1*** 66.3 - Vitamin A-rich orange/red fruits/vegetables 67.2 47.3 46.8 47.0 IYCF indicator - Score Not available 2.12* 1.99* 2.1 - % meeting minimum recommendations for all 3 practices Not available 43.0* 36.3* 39.8 Consumption of WSB (only among index children currently in program) (n = 280) (n = 106) (n = 320) - Gruel made with WSB in past 24 hr n/a 53.0 48.1 51.7 - Gruel made with WSB 3+ times in past 7 d n/a 58.6 58.5 58.5 - Gruel made with WSB 7+ times in past 7d n/a 5.4 6.6 5.7 - Other food made with WSB in past 24 h n/a 16.7 17.9 17.1 - Other food made with WSB 3+ times in past 7 d n/a 16.0 17.9 16.5 - Other food made with WSB 7+ times in past 7d n/a 0.7 0.9 0.8 Vitamin and Mineral Supplements (index child) - % children receiving vitamin A capsule in last 6 mo 30.2 51.6* 45.7* 48.7 - % women who received postpartum vitamin A 7.84 63.5*** 54.1*** 58.8 - % women who received prenatal iron supplements 65.3 82.3 84.0 83.2 Feeding during and after diarrhea (index child) - % who gave more liquid 52.5 50.0 44.8 47.4 - % who gave more semi-solids/solids 6.4 30.6 28.2 29.4 - % who gave extra meal after recovery n/a 66.7* 55.7* 61.4 124 Figure 8.2 Percent of mothers who reported optimal early feeding practices during first day of life, by program group and baseline and final survey—Index child BF within 1 hour Only BM on 1st day Fed colostrum Percent 0 20 40 60 80 100 Baseline (both) Preventive (final) Recuperative (both) * * Statistically significant at p < 0.05. Since many of our index children were born before their mother was exposed to the program BCC, we also look at whether early infant practices had improved for their younger sibling (see Table 8.6 and Figure 8.3). We found no differences between program groups, but younger siblings from the final survey were much more likely to have received optimal early infant practices than those from the baseline survey. As was expected, younger siblings were also more likely than index children to have been breastfed within the first hour (83% versus 65% of index children), and to have been given colostrum (95% versus 86%). These findings suggest that improved maternal knowledge may indeed have translated into improved practices related to early infant feeding practices. Exclusive breastfeeding up to 6 months of age is a key practice and one that received great emphasis in the MC’s learning sessions. This practice was assessed only among younger siblings, because our index children were between 12-41 months of age. At baseline, 42% of mothers with infants younger than 6 months of age reported exclusively breastfeeding the previous day, with no difference between program areas. At the time of the final survey, 92% of women with an infant younger than 6 months reported exclusively breastfeeding in the previous 24 hours, again with no differences between program areas (Table 8.6). While these results, as noted, could reflect women’s knowledge of the “desirable” response, this is unlikely because the categorization of children into the “exclusive breastfeeding” category required that mothers respond consistently to several feeding practices questions (see variable creation in methods section 8.2). For instance, mothers were asked how many times the infant had been fed solids/semi-solids the previous day (at different times during 125 the day); they were also asked about a long list of liquids and foods that their infant might have consumed yesterday, and about the timing of introduction of various foods in their infant’s diet. Responses were consistent across all these questions. Figure 8.3 Percent of mothers who reported optimal early feeding practices during first day of life, by program group and baseline and final survey—Younger siblings BF within 1 hour Only BM on 1st day Fed colostrum Percent 0 20 40 60 80 100 120 Baseline (both) Preventive (final) Recuperative (final) * Difference between preventive and recuperative at final survey is statistically significant (p < 0.05). We also explored whether the proportion of infants reported to be exclusively breastfed fell off sharply in the first 6 mo, as had been the case at baseline. In the final survey, this was not the case (see Figure 8.4). Median duration of breastfeeding, as assessed by Kaplan-Meier survival tables,24 was 5.9 months and did not differ between program areas. Timing of introduction of liquids, and semi-solid and solid foods Mothers were asked to recall when they had first given their child various liquids and foods. Specifically, they were asked to recall at what age they had first given their infant liquids other than breast milk, semi-solid foods, solid foods, meat products, and eggs. A large proportion of women reported introducing liquids and semi-solid foods at the recommended age of 6-8.9 months (e.g., 79% for liquids, 83% for semi-solid foods; 69% for solid foods), with no difference between the preventive and recuperative program areas. Compared to baseline, a much lower percentage of mothers reported introducing liquids and semi-solid foods earlier than at 6 months of age (21% for liquids and 17% for semi-solid foods); at baseline 81% reported giving liquids early and 73% reported giving semi-solids early (Table 8.6; Figure 8.5). For some 24 Survival tables allow analysis of “censored” data, where all observations can be incorporated into the analysis whether or not the “event” (in this case, move from exclusive to nonexclusive breastfeeding) has occurred. 126 infants, meat (29%) and eggs (21%) were introduced late (after 9 mo), with fewer children (7%) given these foods too early. The percentages of mothers who introduced these nutrient-rich foods at the recommended age (6-8.9 months) were much higher in the final survey compared to baseline (Table 8.6). Figure 8.4 Comparing baseline and final sample exclusive breastfeeding in past 24 hours for infants < 6 months: Kaplan-Meier survival functions (preventive/recuperative groups combined) Baseline Final survey Maintenance of breastfeeding Table 8.6 shows the proportion of index children still breastfed, by age group. The pattern is similar in preventive and recuperative areas, and is also quite similar to the pattern seen at baseline, with most children being breastfed up to approximately 17 months, followed by a sharp decline thereafter. Responsive feeding Responsive feeding is a complex behavior and one that is difficult to capture in simple surveys. In the survey, respondents were asked if their child ate by him/herself, or if s/he was fed. The appropriateness of a child feeding him/herself is clearly related to age: as children develop into their second and third years, it is expected that children will gain experience with feeding themselves, but supervision by an adult caregiver is important to ensure that the child eats adequate amounts of food. Mothers were also asked if they would take any action when 127 Figure 8.5 Maternal recall of timing of introduction of liquids and foods, at baseline and final survey (preventive/recuperative groups combined Child age (months) 0-.9 1-1.9 2-2.9 3-3.9 4-4.9 5-5.9 6-8.9 >=9 Cumulative Percent 0 20 40 60 80 100 120 Other liquids (final) Semisolids (final) Solids (final) Meat (final) Eggs (final) Other liquids (baseline) Semisolids (baseline) Baseline Final 128 the child refused to eat. If they responded “yes,” they were prompted for three actions or strategies they might use when the child refused. Women in recuperative communities were slightly more likely to report that their child fed him/herself (68% versus 62% in preventive areas) (Table 8.6). Conversely, women in preventive communities were slightly more likely to be able to name two or more positive actions that they took when the child refused to eat (65% versus 60% in recuperative areas). There were, however no significant differences between program communities in the percentage of mothers who reported using any given positive or negative strategy when the child refused to eat. Compared to baseline, final survey results suggest positive changes in recommended practices related to responsive feeding. The proportion of children in both program communities who were reported to feed themselves was lower in the final survey compared to baseline. The proportion of women who reported using two or more positive strategies (caressing, playing, offering different food) increased from 24% to 62%, while the proportion reporting using one or more negative strategy decreased markedly (forcing the child decreased from 35% at baseline to 14% at the final survey). Safe preparation and storage of complementary foods - Use of baby bottles Our survey did not include data on the safe preparation and storage of complementary foods. However, we did gather data on the use of baby bottles. We found no difference between preventive and recuperative communities in the use of baby bottles at the time of the final survey (about 17% in both groups). This was substantially lower than the proportion of mothers who reported using baby bottles at baseline (40%). Feeding frequency Infants and young children need to be fed frequently throughout the day, due to their small gastric capacity and their high energy and nutrient requirements. Non-breastfed children must be fed more often than breastfed children because they do not benefit from the energy and nutrients contained in breast milk. Current recommendations for feeding frequency are that breastfed children 6-8 months old should receive 2-3 meals of complementary foods/day and 9- 23 months old, 3-4 meals a day (PAHO/WHO 2003). Non-breastfed children should receive one additional meal a day, and thus the recommendation is 3-4 meals/day for 6-8-month-old children and 4-5 meals a day for 9-23-month-old children. There are no specific recommendations for children 24 months and older, who are expected to consume the family diet; results for this age group are more difficult to interpret. At baseline, frequency of feeding was similar between the two program groups for breastfed children 12-41 months old (Table 8.6). The same was true at the final survey. Although children in the preventive program communities were slightly more likely to have met the minimum recommended number of meals between 12-23 months of age and had consumed, on average, a slightly higher number of meals, the differences between program groups were not statistically significant. Compared with baseline, a much higher proportion of 12-17-month-old breastfed children in both program groups had consumed the minimum recommended number of meals (58% compared to 38% at baseline) (Figure 8.6) and the average number of meals had increased by 0.3 and 0.2 among 12-17 and 18-23-month-old children, respectively. 129 Figure 8.6 Percent of breastfed children having received complementary foods minimum recommended times yesterday, by age, program group, and time of survey 6-8 mo 9-11 mo 12-17 mo 18-23 mo Percent 0 20 40 60 80 100 Baseline (both) Preventive (final) Recuperative (final) Younger siblings Index children * Differences between preventive/recuperative statistically significant (p < 0.05). Analysis of younger siblings (children 6-12 months of age) showed similar findings: i.e., no significant difference between program groups at final survey and a generally larger proportion of children having been fed the minimum recommended number of times compared to baseline (not shown). Non-breastfed children were much less likely than breastfed children to have received the minimum recommended number of meals (Figure 8.7). Overall, 86% of non-breastfed children aged 12-23 months were reported to have had solid/semi-solid foods25 fewer than the recommended 4 times the previous day, with no change since baseline (87%), and no difference between program areas. The number of meals at different ages also increased only slightly between baseline and the final survey among non-breastfed children (average increase of 0.1 meal). The results should be interpreted with caution because the data on number of meals does not include information on consumption of milks other than breast milk. However, only 23% of non-breastfed children were reported to have had dairy products the previous day. An additional caveat is the contribution of snacks to daily energy intake, which could be more important for non-breastfed than breastfed children. This information is not available from our survey data, but data on the number of snacks suggest a slightly higher number of snacks among non￾breastfed 12-23-month-old children compared to breastfed children. The learning sessions did not emphasize different feeding frequencies for non-breastfed children, and this could have influenced the results for this subset of children. 25 About 28 percent of children between 12 and 23 months of age were not being breastfed any longer. 130 Figure 8.7 Percent of non-breastfed index children fed complementary foods a minimum recommended times, by age, program group, and time of survey Age groups (months) 12-17 mo 18-23 mo Percent 0 2 4 6 8 10 12 14 16 18 20 Baseline (both) Preventive (final) Recuperative (final) 0% Note: Percent of non-breastfed children fed minimum number of meals was 0% at baseline for 12- 17-month-old children. * Differences between preventive/recuperative statistically significant (p < 0.05). In the formative research conducted early in this project (Menon et al. 2002b), it was noted that young children were rarely fed in the evening, for several reasons. Both lack of food, and cultural beliefs that late feeds could lead to indigestion were cited as reasons for this. The importance of giving children a meal late in the day was one program message. However, meal patterns remained similar to baseline in that respect (Table 8.6), with few children being fed in the evening. Only approximately one-fifth of mothers at baseline and at the final survey reported waking a sleeping child to feed an evening meal, without any difference between program groups. Information from a 24-hour recall suggests an even lower percentage - 10% - of young children being fed an evening meal (not shown), again, without any differences between program groups. Nutrient content of complementary foods: Dietary quality and diversity A diverse diet that includes micronutrient-rich food groups can help ensure adequate intakes of all nutrients for infants and young children. There were no differences at baseline between the two program groups in the proportion of children who had consumed different food groups, or in mean dietary diversity. Results of the final survey (Table 8.6; Figure 8.8) show that children in the preventive areas were more likely than those in recuperative areas to have eaten vitamin A-rich fruits and vegetables, other fruits and vegetables, dairy, and other animal-source foods (meat, fish, and eggs) (all had p < 0.05) on the previous day. Consumption of cereals, roots/tubers/starchy vegetables, and nuts did not differ between program communities at final 131 survey. Mean dietary diversity was consistently greater among preventive compared to recuperative program children, but differences were statistically significant only for the 12-17- month age group and for all ages grouped combined. Figure 8.8 Percent of index children who consumed selected food groups in previous 24 hours, by program group, and baseline and final survey VA fr/veg Other fr/veg Dairy Meat/fish/egg Percent 0 20 40 60 80 100 Baseline (both) Preventive (final) Recuperative (final) * * * * * Differences between preventive/recuperative statistically significant (p < 0.05). Overall dietary diversity and the proportion of children consuming fruit and vegetables and meat/fish/eggs were lower at the final survey compared to baseline (Table 8.6; Figure 8.8). This may reflect the deteriorating economic situation, increased food prices and continued extreme food insecurity in the region during the survey period (see Chapter 9 on food security). Because eggs and flesh foods are high in a number of bioavailable micronutrients, we also looked more closely at these food groups (Table 8.6). In the final survey, index children in preventive areas were more likely to have had eggs, and beef or pork than children from recuperative program areas. Children in the preventive program communities were also more likely to have consumed flesh foods 3 times or more in the previous 7 days (73%) than were children in the recuperative group (60%). Finally, the WV-Haiti program had developed and promoted a series of recipes based on donated food commodities (wheat-soy blend (WSB)) complemented with nutrient-rich locally available foods. Among current program beneficiaries26 at the time of the final survey, 26 This analysis is done only among children who are current program beneficiaries because children not currently in the program do not receive WSB. 132 approximately half of the children had consumed a WSB-gruel on the day prior to the survey and 17% had consumed a complementary food preparation containing WSB and other foods with no difference between the two program areas (Table 8.6). We also assessed frequency of consumption of these foods over the past week; 59% of mothers reported having given the child a WSB-gruel and 17% gave another preparation containing WSB three or more times in the previous week, again, with no difference among participants from the preventive versus the recuperative areas. Energy and nutrient-dense foods are particularly important for non-breastfed children because they do not benefit from all the nutrients contained in breast milk. In spite of their great need for nutrient-dense foods, we found that non-breastfed index children were not more likely than breastfed children to have had any of the animal source foods in the previous 24 hours, nor were they more likely to have had these foods more frequently over the past week (results not shown). IYCF indicator The score on the IYCF indicator was slightly higher and statistically significantly different in the preventive compared to recuperative approach (2.13 versus 1.99) (see Table 8.6). The proportion of children between 6 and 23 months of age for whom the 3 practices met the minimum recommendations was also higher in the preventive group (43% versus 36%, p < 0.05 using random effects logit regression models to compare program groups). Food consistency As noted above, food consistency was not directly observed or measured. However, several indirect indicators related to food consistency were measured. Results related to recipes promoted in MCs are presented above: in addition to being nutrient-dense, these recipes were designed to help mothers prepare foods of appropriate consistency for their child. As reported above, a large proportion of children were fed these recipes on a regular basis while participating in the program, and there were no differences between the groups. Second, infants should be offered pureed, mashed, and semi-solid food starting at 6 months of age. The percentage of children first offered semi-solid food at 6-6.9 months serves as an indicator of both appropriate timing and consistency of food. As reported above, overall, 83% of mothers of index children reported first giving semi-solid foods to index children at 6-6.9 months, without any difference between the two program groups. However, as noted previously, it is unclear whether these results truly represent improved practices or whether they reflect major improvements in maternal knowledge. Use of fortified foods and vitamin-mineral supplements Fortified foods and vitamin-mineral supplements can help ensure nutrient adequacy, especially where access to and intakes of animal-source foods are limited. In our study area, we determined that fortified commodities such as CSB and WSB could help meet micronutrient needs for children in the second year of life and beyond, but for infants 6-11 mo, even these fortified commodities were insufficient to allow infants to meet their daily iron and zinc 133 requirements (Ruel et al. 2004). Therefore additional micronutrient supplements may be required. Results related to intake of fortified WSB were presented above and showed no major differences between program groups when children were in the program. Traditional supplementation with vitamin A capsules (children and mothers) and with iron (mothers during pregnancy) was part of the range of services offered in both program areas. These services, however, were not offered only to program participants; they were available at the Rally Posts for the whole community. Coverage with vitamin A capsules, both for index children in the last 6 months and for mothers immediately postpartum, was higher in the preventive than in recuperative areas, though differences were not large (differences of 6-9 percentage points in favor of the preventive group) (Table 8.6). Coverage with iron supplements during pregnancy was high in both program areas and did not differ between areas. Compared to baseline, overall coverage for women had improved dramatically, particularly for vitamin A capsules. At baseline, coverage with iron pills during pregnancy was 65% compared to 83% in the final survey. At baseline, only 8% had received a postpartum vitamin A capsule, as compared to 59% in the final survey. Finally, vitamin A capsule coverage for index children also increased dramatically, from 30% at baseline to 49% in the final survey, but still falls far short of universal coverage even among children who had ever participated in the program (52%, not shown). Feeding during and after diarrhea Recommended practices for feeding during and after diarrhea include giving more liquids and continuing to offer solid food while the child is ill, and giving an extra meal each day once appetite returns (as the child recovers). Results of the final survey show that overall, approximately half of mothers report giving more liquids, and 30% offering more semi-solid or solid foods when their child was ill (Table 8.6). There were no differences between program areas. For liquids, these results are very similar to baseline, but for semi-solid and solid foods, a marked increase over baseline is observed (6% at baseline compared to 30% at final survey). After the child recovered, 61% reported that they were able to give their child an extra meal each day, for a number of days ranging from 1-15 (median 7). Mothers in preventive areas were more likely to report giving an extra meal to the child during convalescence (67% versus 56% in recuperative areas). 8.4.4 Impact of interventions on preventive care, care during illness, and hygiene Immunization Approximately one-quarter (27%) of the index children were fully immunized - based on their health card or maternal recall - at the time of the final survey, with no differences between program groups (Table 8.7). Although these numbers are higher than at baseline (11%), they reflect very low coverage of immunization. Even among children who had participated in the 134 program at some point in the previous three years, the percent of fully immunized children was very low (31% for preventive and 26% for recuperative). These results, however, are consistent with the program’s reported problems with supply of vaccines (see Chapter 5). Table 8.7 Preventive care, care during illness, and hygiene, by program group and time of survey, for index children 12 to 42 months of agea Final survey Feeding/care practice Baseline Preventive Recuperative All children (n = 1,462) (n = 550) (n = 564) (n = 1,114) Immunization status (n = 1,468) (n = 551) (n = 565) (n = 1,116) - Fully immunized (according to card) 12.9 28.9 25.0 26.8 - Fully immunized (from recall) 6.6 27.2 28.9 28.0 - Fully immunized (card or recall) 10.5 28.5 25.7 27.1 Curative health-care-seeking Sought treatment when ill with: - fever 82.2 66.9* 57.9* 62.0 - cough 81.0 51.4 49.2 50.3 - fast breathing 80.9 67.2 68.5 67.9 - diarrhea 72.0 45.0 36.8 40.9 Used ORS when child had diarrhea (denominator: children who had diarrhea in previous 2 weeks) 40.0 45.0 46.8 45.9 Used home-made sugar-salt-solution when child had diarrhea 10.0 27.2 19.9 23.5 Used either ORS or homemade sugar-salt-solution when child had diarrhea 45.4 41.4 40.9 41.2 Hygiene scales (index child) - Overall child cleanliness score (mean); max = 13 7.7 8.6 8.5 8.5 - Maternal cleanliness score (mean); max = 12 9.8 8.4 8.3 8.4 - House interior cleanliness score (mean); max = 6) 4.8 4.5 4.6 4.5 - House exterior cleanliness score (mean); max = 7) 5.8 5.4 5.6 5.4 * Significant at p ≤ 0.05 (comparison of preventive and recuperative areas; results from xtlogit for dichotomous variables). a Baseline data for these variables were only available for children aged 18 mo and older; final data are shown for the same age group for comparison. Health seeking practices for fever, respiratory infections, and diarrhea, and use of oral rehydration solution (ORS) for diarrhea Table 8.7 also presents results on health-care-seeking practices for basic childhood illnesses such as fever, upper respiratory infections, fast breathing, and diarrhea. There are no striking differences between program groups for care-seeking for any of the illnesses. Compared to baseline, care-seeking for all four illnesses was markedly lower. Since health-care service provision had improved since baseline (with the availability of the WV MCHN program services), the lower care-seeking at the follow-up survey could indicate either lower severity of illness or poor availability of economic resources to use services. However, since most WV 135 services were available free of cost, the latter was less likely to of an explanation for lower rates of care-seeking. Use of ORS for diarrhea was also not different between the two program groups, and there was little difference from baseline. However, in the final survey women were more likely to report having used a home-made sugar-salt solution for diarrhea. Hygiene We collected data on proxies for hygiene practices, which could be assessed by spot￾check observations (see description of method in section 8.2). Results show no difference in any of the hygiene scales between program groups (Table 8.7). In summary: ™ At baseline there were no differences between program groups in feeding practices, care-seeking, or proxies for hygiene practices. ™ In the final survey, there were no differences between program groups in early feeding practices (initiation of BF and feeding of colostrum). Large improvements were seen since baseline. ™ Similarly, there were no differences between program groups in the timing of introduction of liquids and complementary foods, but large improvements were seen since baseline, with a majority of women reporting initiating feeding of both liquids and semi-solids at approximately 6 months of age. ™ There were no differences between program groups in continued breastfeeding up to 24 months, and no differences since baseline either. Haitian women’s breastfeeding practices are generally positive, with children being breastfed until between 18 and 24 months of age. ™ Slightly more young children in the preventive group received assistance while eating than in the recuperative group; there was a moderately large increase in this practice over baseline. ™ Use of baby bottles was not different between program groups, but rates of use were halved since baseline. ™ There were no differences between program groups in the frequency of feeding, but meal frequency for children in the vulnerable age of 12-17 months of age had increased since baseline. ™ There were no differences between program groups in the practice of feeding the child an evening meal, and no difference since baseline either. ™ Diet quality was slightly better in the preventive group overall, as assessed by several proxies. Children in preventive areas were more likely to have had nutrient-rich food 136 groups, including eggs and beef/pork in the previous day or week, and mean dietary diversity was higher. The overall consumption of ASF was lower than at baseline, possibly due to the economic crisis in Haiti, which had led to increased food prices over the study period. ™ Scores on the IYCF indicator – which combines information on breastfeeding, feeding frequency, and dietary diversity - were slightly higher in the preventive group than in the recuperative group. ™ Vitamin A supplementation rates were higher in the preventive group, with large differences seen since baseline. ™ Feeding practices during and after diarrhea were also better in the preventive group, particularly for reported increases in meal frequency after an episode of diarrhea. Large improvements were also seen since baseline. ™ Immunization rates were not different between the program groups, but had almost tripled since baseline. However, rates remain very low. ™ There were no differences in markers of hygiene practices between program groups. ™ Care-seeking for fever, cough, fast breathing, and diarrhea were not different between program groups. Care-seeking rates were lower at the follow-up survey than at the baseline survey. ™ Use of ORS when the child had diarrhea did not differ between program groups and had not changed markedly since baseline; use of home-prepared solutions had, however, increased. 8.4.5 Comparisons between participants and nonparticipants on child-feeding and care practices As noted in the methods section, additional analyses for some key outcomes were done to compare child-feeding and care practices between participants and nonparticipants, within the preventive and recuperative program communities. The purpose of this analysis was to assess whether participation in the program was associated with a meaningful change in practices. Except when specified otherwise, the indicator used for participation here is “ever participated” in the program (and therefore exposed to the BCC delivered in the MCs). A summary of the differences by participation is presented in Table 8.8, and detailed results are presented in Annexes 8.5 to 8.12. We provide here a brief summary of the practices where marked differences were found between participant and nonparticipant mothers. 137 Table 8.8 Summary of findings on behavior change outcomes Preventive vs. Recuperativea Change compared to baselineb Difference between ever participated vs. never participatedc Knowledge Child feeding knowledge tests - Timely introduction of foods - d ↑ Large diff - Feeding frequency Prev > (small diff.) ↑ Large diff - Feeding knowledge (weighted) Prev > (small diff.) ↑ Large diff Overall nutrition and health knowledge test (BCC topics) Prev > (small diff.) n/a - Breastfeeding knowledge - n/a Large diff - CF knowledge Prev > (small diff.) n/a Large diff - Child illness Prev > (small diff.) n/a - - General health issues - n/a - - Malnutrition - n/a - Awareness, trial and adoption of key practices - Awareness Prev > (small diff.) n/a Large diff - Trial Prev > (small diff.) n/a Large diff - Adoption Prev > (small diff.) n/a Large diff Care and feeding practices Early infant practices BF within 1 hour - Large ↑ Large diff. EBF on 1st day - Large ↑ Large diff. Gave colostrum Recup > Large ↑ Large diff. Timing of introduction of liquids and foods - Large ↑ Large diff. Continued BF - - - Responsive feeding Prev > (for helps child feed) ↑ More positive strategies (prev.) Use of baby bottles - ↓ by half Large diff. Meal frequency - ↑ (for 12-17 mo, BF) > if child currently in program Evening meal - - - Dietary diversity (DD) - Food group consumed Prev > for nutrient-rich foods (small differences) ↓ for nutrient-rich foods - - Mean dietary diversity Prev > at 12-17 mo ↓ Small diff. in some age groups - Animal source food consumed Prev > for eggs, beef/pork ↓ by half Small diff. - Consumption of WSB while in program - n/a Large diff. (as expected) IYCF indicator - Score Prev > (small diff) Not available No diff in prev.; partic Not available No diff in prev.; partic for VA Large diff. Large diff. Feeding during and after diarrhea Prev > for extra meals after recovery Large ↑ in giving more food during diarrhea Large diff for extra meals after recovery (continued) 138 Preventive vs. Recuperativea Change compared to baselineb Difference between ever participated vs. never participatedc Preventive care, hygiene, and curative care seeking Immunization: fully immunized - Large ↑ Diff. only in prev. Hygiene scores - ↓ Small diff. in recup for child cleanliness Curative care seeking - ↓ Small diff for fever Use of ORS when child has diarrhea - - No diff. Use of home-made SSS when child has diarrhea - ↑ Diff. only in prev. a Only statistically significant differences reported for differences between preventive and recuperative program groups. b Only large differences are reported – no statistical testing. c Only large differences are reported – no statistical testing. d “-” means no difference; “n/a” means not applicable. Early infant feeding practices and timing of introduction of liquids and semi-solid and solid foods In both program groups, a much larger proportion of mothers who had been exposed to the BCC intervention reported optimal early infant feeding practices compared to nonparticipant mothers (Annex 8.5). The same was true for the timing of introduction of liquids and semi-solid and solid foods. Among nonparticipant mothers, 30% reported giving liquids by 2 months and 56% by 6 months, compared to those who had been exposed to the MCs (4% reported giving liquids by 2 months and 14% did so before 6 months). Nonparticipant mothers were also more likely to have introduced semi-solid foods before the age of 6 months (close to 50%) compared to participant mothers (approximately 10%) (Annex 8.6). Use of baby bottles There were large differences in the use of baby bottles by program participation, with high use of baby bottles among mothers who had never been exposed to the program compared to those who had. In the preventive group, 46% of nonparticipants reported using baby bottles compared to 12% among participants. In the recuperative group, differences were of smaller magnitude but showed a similar pattern: 24% among nonparticipants compared to 15% among participants. Meal frequency Children whose mothers had ever participated in the MCs were no more likely to be fed (at least) the recommended minimum number of times the day before the survey. However, if the child was currently receiving a food ration, the mean number of meals was higher both for 12-23-month-old children and for younger siblings (6-11 months old). This, however, was not true for older children. Most strikingly, non-breastfed children aged 12-23 months were nearly 3 times as likely to be given at least the minimum recommended number of meals (four) if they were currently receiving a food ration (23% versus 8% for those not receiving a ration) (Annex 8.7). These results suggest that food assistance, which at least temporarily improves household food security, may have short-term beneficial effects on frequency of feeding for non-breastfed 139 children. This is particularly important because as discussed above, very few non-breastfed children were fed as frequently as recommended. Nutrient content of complementary foods: Dietary quality and diversity There were few differences in diet quality related to program participation, when participation was defined as “ever participated.” Children whose mothers were never exposed to the program were much less likely to have consumed eggs in the previous day, especially in the preventive program areas (Annex 8.9); eggs were specifically recommended in the recipes promoted and demonstrated at the MCs. Also, as could be expected, the use of fortified foods—which in this population was largely limited to WSB—was much more common among current program participants (in both program groups) than among nonparticipants (Annex 8.11). This is expected as WSB is only available through food assistance programs in Haiti. Overall dietary diversity was also slighter higher among participant compared to nonparticipant children in both program groups in certain age groups (children aged 18-23 months in preventive areas, and children aged 24-29 months in recuperative areas) (Annex 8.12). IYCF indicator: Mothers who had been exposed to the program in the preventive group had better scores, and were more likely to have met minimum recommendations for all 3 practices, than mothers who were exposed in the recuperative group. There was little difference between exposed and nonexposed mothers in the preventive group. In the recuperative group, however, the IYCF indicator scores and the percent meeting minimum recommendations were higher among nonexposed mothers than among exposed mothers. This could be due to reverse causality; children who are fed poorly are more likely to be malnourished and thus be eligible for enrollment in the recuperative program (i.e., be “exposed”). Use of vitamin-mineral supplements. Mothers who had been exposed to the program were much more likely to have received prenatal iron supplements and postpartum vitamin A in both program groups. Children were also more likely to have received vitamin A supplements in the previous 6 months (Annex 8.13). These findings suggest that, although these services are available at RPs for the entire population, they appear to be used more by women who have participated in the MCs, and their children. Feeding during and after diarrhea. Preventive area mothers who had participated in the MCs were more likely to offer more extra food to their child during a diarrheal episode. Participating mothers in both areas were more likely to feed their child an extra meal after recovery from diarrhea (Annex 8.14). Preventive care, care during illness, and hygiene When compared to nonparticipants, immunization rates were higher among participants in the preventive group (Annex 8.15). There were no differences in proxies for hygiene practices between participants and nonparticipants in either program group, except for child cleanliness where participants in the recuperative group had better scores than nonparticipants (results not 140 shown). Care-seeking for fever, cough, fast breathing, and diarrhea were also not very different between participants and nonparticipants. However, use of ORS when the child had diarrhea was higher by about 10 percentage points among participants when compared to nonparticipants but differences were not significantly different. Use of salt-sugar solution (SSS) was marginally significantly higher (p = 0.06) among participants in the preventive group but not in the recuperative group. In summary: ™ There were large differences in early feeding practices between ever-participants and never-participants, i.e., initiation of BF and feeding of colostrum. ™ There were large differences between ever-participants and never-participants in the timing of introduction of liquids and complementary foods. ™ There were no differences between ever-participants and never-participants in continued breastfeeding up to 24 months. ™ Use of baby bottles was significantly lower among ever-participants compared to never-participants. ™ Meal frequency was higher for children who were currently enrolled in the program; non-breastfed children currently receiving rations were much more likely to be fed with recommended frequency. ™ There were no differences between ever-participants and never-participants in the practice of feeding the child an evening meal. ™ Among children 6-23 months of age, there were no differences between ever￾participants and never-participants in the preventive group on the IYCF indicator. Among the recuperative group, however, ever-participants had lower scores on the IYCF indicator than never-participants. ™ Consumption of fortified WSB was markedly higher among ever-participants, as expected. ™ Vitamin A supplementation rates were higher among ever-participants (both women and children) than among never-participants. Women who had participated were also more likely to have received iron supplements during pregnancy. ™ Feeding practices during (preventive area) and after diarrhea (both areas) were different between ever-participants and never-participants. ™ Immunization rates were also different between ever-participants and never￾participants, but only in the preventive group. 141 ™ There were no differences in markers of hygiene practices between ever-participants and never-participants in either program group, except for child cleanliness where participants in the recuperative group had better scores than never-participants. ™ Care-seeking for fever, cough, fast breathing, and diarrhea were not very different between ever-participants and never-participants. ™ Use of ORS when the child had diarrhea was higher among ever-participants than never-participants but differences were not significant. Use of salt-sugar solution (SSS) was marginally significantly higher among ever-participants. 8.5 Conclusions This chapter presented a wide range of results related to behavior change outcomes. The BCC activities, delivered through the MCs (and described in Chapters 3, 4, and 5) comprised the primary program inputs aimed at influencing behavior change. These inputs were expected to lead to improvements in maternal knowledge and skills and thereby to improved infant feeding practices (see program theory description in Chapter 2). Differences in practices between program groups were expected due to two specific aspects of program design: (1) the timing of delivery of messages regarding complementary feeding, which was designed to be provided before the child reaches complementary feeding age for the preventive group; and (2) the duration of exposure to the BCC, which was longer for the preventive (up to 18 months) compared to the recuperative group (maximum 9 months). As highlighted in Chapters 5 and 6, both of these critical design characteristics were achieved and mothers in preventive areas were thus more likely to receive the education/BCC in a timely fashion and to be exposed to the messages for a longer duration than mothers in the recuperative areas. In addition to the differences in timing and duration of exposure, the results on program participation also showed greater program participation rates after the child was 6 months old in preventive compared to recuperative areas (Chapter 6), thus suggesting an additional mechanism by which the preventive program may have had a greater impact on behavior change than the recuperative program. The results presented here demonstrate that the program pathway and the expectations based on program design were generally achieved as intended, and differences between program groups were consistent with expectations based on program targeting and exposure. For instance, there were no differences between the groups in knowledge and practices related to early breastfeeding and exclusive breastfeeding. This was expected, given that exposure to the program was similar in both groups during pregnancy and the first 6 months of lactation. By contrast, differences in favor of the preventive group were observed in several of the complementary feeding behavior change outcomes; again this was expected, given that the preventive approach was intended to provide greater intensity and more timely delivery of education outcomes. In general, however, the magnitude of differences between the groups was quite small for most behavior change outcomes. Although overall differences between program groups were generally small, differences since baseline in both groups were quite large; this suggests that the program was successful in raising overall child-feeding and care knowledge and practices in the communities where it operated. Differences between participants and nonparticipants were also large and statistically 142 significant for many behavior outcomes. Participants fared generally much better than nonparticipants for most BCC outcomes, from knowledge, trial, and adoption to the full spectrum of reported child-feeding practices. This was true in both the preventive and recuperative program groups, and can be interpreted as reflecting programmatic success. It also suggests that the process used to develop the BCC program, the technical content and staff development intrinsic to the program, was successful, as was the mechanism used to deliver the BCC program (i.e., the small peer-group approach). Finally, since improvements in practices were seen for younger siblings in both program groups, the results also suggest that the positive benefits of a solidly designed and well-implemented BCC program extend beyond immediate effects. It will be important to continue to monitor knowledge and practices related to infant and young child feeding in these Haitian communities to evaluate the true long-term impact of the investments in the development of such a strategy. 143 9. IMPACT OF THE PROGRAM ON FOOD SECURITY 9.1 Introduction This chapter discusses the impact of the WV-Haiti MCHN program on food security outcomes, particularly those outcomes related to food access (rather than utilization or availability). The WV MCHN program, by virtue of providing food rations to households and supportive learning environments (via Mothers’ Clubs), could directly impact household resources such as food security, and such impacts could be greater in the preventive than in the recuperative program communities, since program benefits are provided for longer durations and to a large proportion of households. This chapter therefore examines differences between the two program groups on household food security outcomes. It also compares the food security indicators at baseline in 2002 and at the final survey in 2005. 9.2 Data and Analysis 9.2.1 Data and variables The final impact survey gathered data on two dimensions of household food insecurity: (1) Household experiences related to food insecurity. Data were collected on 11 types of food-insecurity experiences. These experiences of food security covered a range from less extreme to more extreme, e.g., “cooking without beans” (a preferred food), which was at the less extreme end of the experiences, compared to “going to bed hungry,” which was considered a more extreme experience of food insecurity. Data on these 11 individual experiences related to food insecurity were used to develop a composite food-insecurity scale that summarized the information from the individual variables into a meaningful composite measure. The scale was constructed by summing the total number of food-insecurity-related experiences that a household had faced in the past 30 days. All the variables included contributed a maximum of 1 point to the scale, and variables with multiple categories were recoded so that the highest category contributed 1 point, the lowest category, 0 points, and categories in between contributed between 0 and 1 point. Thus, the scale ranged from a minimum of 0 to a maximum of 11, where households with a score of 11 would have experienced extreme levels of food insecurity. The distribution of the food-insecurity experiences scale was divided into terciles to create three groups that represented low, moderate, and severe food insecurity. The detailed scoring of the variables that were included in the scale is presented in Annex 9.1. The reliability of the household food-insecurity experiences scale was tested using Cronbach’s alpha. The scale had a reliability of 0.77. (2) Months of inadequate household food provisioning (MIHFP). The MIHFP measure was adapted from an indicator developed by FANTA (Bilinsky and Swindale 2005). Respondents were asked to indicate whether they had enough food for household consumption in each individual month in the previous 12 months. Responses were on a 3-point scale and ranged 144 from “Yes, enough food,” to “No, not enough at all.” Responses to these questions were summed up to create two summary variables: Months of inadequate household food provisioning (MIHFP): This summary variable captured the number of months that a respondent reported any food inadequacy in the household. In order to create it, the 3-point scale was recoded such that any kind of insufficiency (either “not enough at all” or “somewhat enough”) was coded “yes.” Although this could overestimate the number of months of food inadequacy, we used this coding because it is closer to the responses we would have received from respondents had we used the original FANTA yes/no response sets for these questions (Bilinsky and Swindale 2005). Severity of food inadequacy: A summary variable was created by adding up responses on the 3-point scale for all 12 months in the MIHFP measure. A higher score indicated more severe food inadequacy. The distribution of the score for the entire survey sample was also divided into terciles to enable categorical comparisons. The terciles were labeled “low,” “medium,” and “high,” and the proportion of households in the “high” group was compared between program approaches. 9.2.2 Analysis A systematic stepwise approach focusing on differences between program communities at the time of the final survey was used for the analysis. It included the following steps: 1) First, we assessed the unadjusted difference between program communities using random effects regression methods that adjusted for the paired design. 2) Next, the difference between program communities was examined after adjusting for common confounding factors like respondent’s education level and partner’s education level, depending on the outcome of interest. 3) Third, we examined benefits of program participation by separating participants and nonparticipants, and examining the differences between participants and nonparticipants in the two program groups. We also note any relevant differences between nonparticipants in each of the program groups. In examining differences between program participants and nonparticipants within program group, we used two measures of participation: (1) whether a household/mother had ever received food rations; (2) whether a household/mother was receiving program benefits at the time of the survey. We evaluated the significance of differences between participants and nonparticipants using interaction terms between participation and program group, evaluating separately the interactions between “ever participation” and program group and between “current participation” and program group. 4) Finally, where possible, we draw informal comparisons with the baseline results to evaluate the extent of change from baseline. In general, in evaluating the differential influence of the program approaches on these outcomes of interest, we do not adjust for any variables that could be influenced by the program. Adjusting for such variables would underestimate the impact of the program. 145 9.3 Results 9.3.1 Impact of the Program on Household Food Insecurity Descriptive results for the three measures of food insecurity - household food insecurity experiences scale, number of months of food inadequacy, and severity of food inadequacy scale - are presented in Table 9.1. Note that there were no differences in food insecurity at baseline and, therefore, the baseline results are presented for the two programs combined. Table 9.1 shows that preventive program communities generally fared better in terms of household food security at the time of the final survey than the recuperative communities, where food insecurity was consistently higher. Table 9.1 Household food insecurity, by program and time of survey Baseline Final survey (2005) (2002) Both groups Preventive Recuperative Both groups (n = 1,514) (n = 748) (n = 752) (n = 1,500) Food security characteristics Variable Mean (SD) Mean (SD) Mean (SD) Household food insecurity experiences (scale) fdinsec 7.8 (1.8) 7.7 (2.4) 7.9 (2.3)* 7.8 (2.3) Number of months of food inadequacy Mo_insuf n/a 8.8 (2.8) 9.2 (2.7)* 9.0 (2.8) Severity of food inadequacy sevinsuf n/a 25.7 (5.4) 26.5 (5.4)* 26.1 (5.4) * p < 0.05 (differences tested using unadjusted random effects regression models). Compared to baseline, food insecurity continues to be extremely high in these communities: scores on the food insecurity experiences scale for both groups combined are almost identical between the baseline survey in 2002 and the combined final survey in 2005. An examination of the individual household food insecurity experiences included in the household food insecurity experiences scale (Table 9.2) shows that the preventive program likely better mitigated some of the more severe food insecurity experiences. Overall, the prevalence of experiences reflecting severe food insecurity in this context, i.e., respondents going to bed hungry, children going to bed hungry, and cooking the same food every day, are significantly lower in the preventive group than in the recuperative group. They are also lower in 2005 compared to 2002 (both groups combined), suggesting that, overall, the program has likely relieved some of the more severe food insecurity experiences. The impact of the program on food insecurity experiences appears to be driven by current participation in the program. Comparing current program participants within program group with nonparticipants suggests that current participants are slightly less food insecure than nonparticipants, and especially so among preventive communities (Table 9.3 and Figures 9.1 - 9.3). Regression models that include current participation show that the magnitude and significance of the program approach variable is diminished with the introduction of the current participation variable (Table 9.4 - 9.6). This is true for all three measures of food insecurity. 146 Table 9.2 Selected household food insecurity experiences, by program and time of survey Baseline Final survey Both groups Preventive Recuperative Both groups Food security characteristics Variable (n = 1,514) (n = 748) (n = 752) (n = 1,500) (%) (%) (%) Cooked with less beans Q713 98.0 97.6 98.0 97.8 Cooked without any beans Q711 96.4 94.5 94.9 94.7 Ate less food because of hardship Q708a 89.7 87.0 88.3 87.7 Worried about not having enough food Q719a 88.2 86.5 89.2 87.9 Cooked without herring head Q715a 87.1 77.5 77.4 77.5 Ate same food day after day Q716 85.0 64.6 72.2** 68.4 Went to bed hungry Q717a 83.9 73.8 78.5* 76.1 Children went to bed hungry Q718a 75.5 59.9 66.8** 63.3 Cooked less because of lack of fuel Q709a 50.2 48.5 48.5 48.5 * p < 0.05 (differences tested using unadjusted random effects regression models). Table 9.3 All household food insecurity measures at final survey, by participation (ever and current) Preventive Recuperative Non￾participant Participant Non￾participant Participant Food security characteristics Variable Mean SD Mean SD Mean SD Mean SD (1) Ever participated (n = 126) (n = 622) (n = 130) (n = 622) Household food insecurity experiences (scale) fdinsec 7.8 2.2 7.6 2.4 7.9 2.5 8.0 2.3 Months of inadequate household food provisioning (MIHFP) Mo_insuf 8.9 2.9 8.8 2.8 9.2 2.8 9.2 2.7 Severity of food inadequacy sevinsuf 25.8 5.6 25.8 5.4 26.6 5.6 26.5 5.3 (2) Current participation (n = 423) (n = 325) (n = 621) (n = 131) Household food insecurity experiences (scale) fdinsec 7.9 2.3 7.4 2.4 8.0 2.3 7.7 2.3 Number of months of food inadequacy Mo_insuf 9.2 2.7 8.4 2.9 9.2 2.7 9.1 2.7 Severity of food inadequacy sevinsuf 26.3 5.3 25.0 5.4 26.6 5.4 26.3 5.4 147 Figure 9.1 Household food insecurity experiences score, by program group and current participation at final survey Program group preventive recuperative HH food insecurity experiences score 0 2 4 6 8 10 Non-participant HH Currently participating ** p=0.08 Figure 9.2 Prevalence of severe food insecurity (based on terciles of the household food insecurity experiences score), by program group and current participation at final survey Program group Preventive Recuperative % of HH classified as highly food insecure 0 10 20 30 40 Non participant HH Currently participating ** 148 Figure 9.3 Months of inadequate household food provisioning (MIHFP), by program group and current participating at final survey Program group Months of inadequate HH food provisioning (MIHFP) preventive recuperative 1 3 5 7 9 Non-participant HH Currently participating ** Table 9.4 Program impact on household food insecurity experiences at final survey (regression analysis) (1) Bivariate model (2) Main effects, adjusted (3) Main effects, ever participated (4) Interaction model, ever participated (5) Main effects, currently participating (6) Interaction model, currently participating Household food insecurity experiences score Program 0.27* 0.28* 0.28* 0.15 0.15 0.13 (0.12) (0.13) (0.13) (0.33) (0.14) (0.16) Respondent’s education (years) -0.11** -0.11** -0.11** -0.11** -0.11** (0.03) (0.03) (0.03) (0.03) (0.03) Partner’s education (years) -0.07** -0.07** -0.07** -0.07** -0.07** (0.02) (0.02) (0.02) (0.02) (0.02) Ever participated -0.06 -0.13 (0.18) (0.25) Currently participating -0.48** -0.51** (0.15) (0.19) Ever participated and Program 0.15 (0.36) Currently participating and Program 0.09 (0.30) Constant 7.68** 8.06** 8.10** 8.17** 8.26** 8.28** (0.11) (0.12) (0.19) (0.24) (0.13) (0.14) Observations 1,500 1,250 1,250 1,250 1,250 1,250 Number of pairs 10 Notes: Standard errors in parentheses. * significant at 5%; ** significant at 1%. 149 Table 9.5 Program impact on months of inadequate household food provisioning (MIHFP) at final survey (regression analysis) (1) Bivariate model (2) Main effects, adjusted (3) Main effects, ever participated (4) Interaction model, ever participated (5) Main effects, currently participating (6) Interaction model, currently participating Months of inadequate household food provisioning (MIHFP) Program 0.34* 0.45** 0.45** 0.30 0.33* 0.14 (0.14) (0.16) (0.16) (0.39) (0.16) (0.19) Respondent’s education (years) -0.11** -0.11** -0.11** -0.11** -0.11** (0.03) (0.03) (0.03) (0.03) (0.03) Partner’s education (years) -0.02 -0.02 -0.02 -0.02 -0.02 (0.03) (0.03) (0.03) (0.03) (0.03) Ever participated -0.04 -0.13 (0.22) (0.30) Currently participating -0.45** -0.71** (0.18) (0.22) Ever participated and Program 0.17 (0.43) Currently participating and Program 0.69 (0.36) Constant 8.83** 8.98** 9.02** 9.10** 9.18** 9.29** (0.19) (0.18) (0.26) (0.32) (0.20) (0.21) Observations 1,500 1,250 1,250 1,250 1,250 1,250 Number of pairs 10 10 10 10 10 10 Notes: Standard errors in parentheses. * significant at 5%; ** significant at 1%. Table 9.6 Program impact on severity of months of inadequate household food provisioning (MIHFP) at final survey (1) Bivariate model (2) Main effects, adjusted (3) Main effects, ever participated (4) Interaction model, ever participated (5) Main effects, currently participating (6) Interaction model, currently participating Severity of months of inadequate household food provisioning (MIHFP) Program 0.75** 0.96** 0.96** 0.55 0.76* 0.41 (0.27) (0.30) (0.30) (0.76) (0.31) (0.36) Respondent’s education (years) -0.27** -0.27** -0.27** -0.27** -0.27** (0.06) (0.06) (0.06) (0.06) (0.06) Partner’s education (years) -0.00 -0.00 -0.00 -0.00 -0.00 (0.05) (0.05) (0.05) (0.05) (0.05) Ever participated 0.04 -0.20 (0.41) (0.59) Currently participating -0.75* -1.23** (0.34) (0.42) Ever participated and Program 0.49 (0.83) Currently participating and Program 1.30 (0.70) Constant 25.76** 26.02** 25.99** 26.19** 26.35** 26.56** (0.44) (0.43) (0.56) (0.66) (0.45) (0.49) Observations 1,500 1,250 1,250 1,250 1,250 1,250 Number of pairs 10 10 10 10 10 10 Notes: Standard errors in parentheses. * significant at 5%; ** significant at 1%. 150 Figure 9.4 maps the responses to the questions related to months of food insecurity experienced by households, by month. It shows a consistent pattern of a slightly lower percentage of households reporting insufficient food at each month in the preventive group, but differences are of small magnitude. As with the previous analyses, however, Figure 9.4 also shows that, in fact, a large proportion of households report food inadequacy throughout the year, ranging from a low of around 50% in January to a high of around 90% in May and June. This pattern corresponds to the cropping and harvest season in the Central Plateau of Haiti. The results confirm the severity of food inadequacy and food insecurity in these Haitian rural communities, while demonstrating the potential for food assistance programs to make a difference. Figure 9.4 Proportion of households with insufficient food in each month of the calendar year, by program group (derived from the months of inadequate household food provisioning (MIHFP) measure) Proportion of households with insufficient food in year preceding the survey 0 10 20 30 40 50 60 70 80 90 100 Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Month % of food insufficient households Preventive Recuperative * * ** ** ** * 0.05 90% of children were taken to RP) but RP use patterns are age dependent. Fewer older children taken to preventive area RPs. N/A N/A 285 Outcome Baseline differences Overall difference between program communities at final survey and improvements since baselinea Difference by program participation a CURRENT PARTICIPATION EVER PARTICIPATED Enrollment in food assistance and BCC package N/A: Program had not started at baseline - Participation rates are same in program groups for pregnant and lactating women (58% and 63%). - Current enrollment of child beneficiaries is higher in preventive (37% vs. 14.1%). - More children in preventive areas were ever exposed to the program (73% vs. 28%) N/A N/A HOUSEHOLD RESOURCES FOR CARE Household characteristics None Overall, no major differences in most characteristics. Household durable goods and productive asset ownership slightly higher in preventive group. Household asset ownership same as baseline in preventive, but lower than baseline in recuperative. Overall: No differences between households in construction, number of rooms, home ownership, sources of drinking water or household assets. Greater household size among current participants compared to nonparticipant. Within program group: Preventive: Number of household assets higher among current participants in preventive. No difference for other household characteristics Recuperative: No differences in participant-nonparticipant comparisons. Overall: No differences between households in construction, number of rooms, home ownership, sources of drinking water. Household size larger among participants in both groups. Within program group: No differences in participant￾nonparticipant comparisons in either group. Food security Food security experiences None Greater food insecurity among recuperative group Severe food insecurity higher among recuperative No improvements since baseline, and overall food insecurity is very high among both groups Overall: Lower food insecurity among current participants. Within program group: Difference in food insecurity between current and nonparticipants larger in preventive versus recuperative. Overall: No difference. Within program group: No differences in participant￾nonparticipant comparisons between groups. 286 Outcome Baseline differences Overall difference between program communities at final survey and improvements since baselinea Difference by program participation a CURRENT PARTICIPATION EVER PARTICIPATED Months of household food insufficiency Not assessed at baseline Preventive communities better off in terms of months of food insufficiency and severity of insufficiency. Preventive program somewhat protective of food insufficiency in most severely food insufficient months. Overall: Fewer months of food insufficiency among current participants Within program group: Greater/more significant difference between participants and nonparticipants in preventive. Overall: No difference. Within program group: No differences in participant￾nonparticipant comparisons between groups. Severity of months of household food insufficiency Not assessed at baseline Preventive communities better off in terms of months of food insufficiency and severity of insufficiency. Preventative program somewhat protective of food insufficiency in most severely food insufficient months. Overall: Fewer months of food insufficiency among current participants. Within program group: Greater/more significant difference between participants and nonparticipants in preventive. Overall: No difference. Within program group: No differences in participant￾nonparticipant comparisons between groups. CAREGIVER RESOURCES FOR CARE General characteristics None Overall, no differences between groups in caregiver education, partner’s education, marital status, employment status, BMI, asset ownership by respondents, control over household purchases, involvement in decisionmaking, material and financial support. Work patterns slightly different between groups (but nonsignificant). Respondents in preventive communities work more outside home, spend more time outside. Overall: Higher social support for participants, higher rates of employment among participants, and longer duration away from home. Within program group: Larger difference in social support between participants and nonparticipants in preventive. More pregnant women among current beneficiaries in recuperative compared to preventive or nonparticipant recuperative (27 % vs. 15%). Overall: Higher rates of pregnancy among nonparticipants than participants. Better communication with spouse among participants. Slightly higher asset ownership among participant respondents. Slightly higher proportion of partnered women among participants. Within program group: No differences in participant￾nonparticipant comparisons between groups. 287 Outcome Baseline differences Overall difference between program communities at final survey and improvements since baselinea Difference by program participation a CURRENT PARTICIPATION EVER PARTICIPATED Caregiver knowledge Baseline scales (introduction of foods, feeding frequency, overall knowledge) None Knowledge higher among P v. R. Knowledge scores improved overall from baseline to final. Overall: Overall significant differences between participants and nonparticipants in both groups. Within program group: Differences between participants and nonparticipants greater in P v. R. Overall: Greater improvements in nutrition knowledge from baseline among participants. Large difference between those who had ever been exposed to the program and those who had never been exposed. Within program group: Differences between participants and nonparticipants greater in P v. R. BCC topic specific scales (overall, BF, complementary feeding, child illness, general health, malnutrition) Not assessed at baseline Slightly better in P communities v. R communities. Overall: Differences between current participants and nonparticipants stat sig. for overall knowledge, BF, and complementary feeding. Marginally significant for childhood illness. Within program group: BF knowledge significant higher in P In R., sig. better overall knowledge, BF, and complementary feeding knowledge among participants. Overall: Large differences between participants and nonparticipants Within program group: More significant differences among participants and nonparticipants in P vs. R, especially for overall knowledge score. Mental and physical well-being None Preventive area caregivers better off on 4 women’s well-being measures. Slight improvement since baseline on mental stress and other well-being measures Overall: Current participation associated with better self-rated health, lower mental stress and lower time stress. Within program group: Differences between participants and nonparticipants larger in preventive areas. Overall: No differences between respondents who had ever been exposed to the program and those who had not. IMPACT ON CARE PRACTICES Awareness, trial, and adoption of recommended practices Not assessed at baseline Awareness, trial, and adoption better in preventive compared to recuperative for complementary feeding related practices. No difference for breast-feeding related practices (as expected by design) Drop off from awareness to trial to adoption dependent on participation and type of behavior being recommended. Not evaluated Overall: Knowledge and adoption for all practices is higher in participants v. nonparticipants. Trial was higher among participants for all practices except adding a beaten egg to a child’s gruel trial. Within program groups: No difference between program groups in patterns of awareness, trial, and adoption by participation. 288 Outcome Baseline differences Overall difference between program communities at final survey and improvements since baselinea Difference by program participation a CURRENT PARTICIPATION EVER PARTICIPATED Breastfeeding Initiation of BF None Program groups are very similar. Improved practices since baseline. 69% initiate BF w/in 1st hr; 88% gave colostrum; only 9% reported giving pre-lacteals. Younger siblings more likely BF in 1st hr vs. index children (83% v. 65%), more likely to been given colostrum (95% v. 86%). N/A Overall: Sig. > percent of participants BF w/in 1hr, gave colostrum, and did not give pre￾lacteals compared to nonparticipants. Within program groups: Differences between participants and nonparticipants were large for each practice and not different between program groups. Exclusive BF None Program groups are very similar. EBF improved since baseline: among children < 6mo old at final survey--92% exclusive BF in last 24h, compared to 47% at baseline. N/A; too few nonparticipants in this age subgroup (< 6 mo) N/A; too few nonparticipants in this age subgroup (< 6 mo) Duration of any BF None Similar BF pattern in P & R group at final. No differences since baseline. Most children breastfed to 18mo, sharp decline in second half of child’s second year. Comparison between current participants and nonparticipants is confounded by large differences in age distribution; differences within 6-mo age groups NS with low statistical power, except more children 18- 23 mo were BF among current partic in R (68%) than among nonpartic in R (38%). Differences within 6-mo age groups are NS with low statistical power No evidence that participation (current or ever) compromised continued breastfeeding Complementary feeding Introduction of foods None No difference between program groups. Large improvements since baseline; 86% delayed introduction of other liquids until at least 6 mo; 77% of women reported first giving semi-solid food to index children at 6-6.9 mo N/A Overall: Early introduction of liquids and semi-solids much more likely to be reported among nonparticipants v. participants. Feeding frequency None No meaningful differences between P & R in frequency of feeding. Breastfed children 12-23 mo much more likely to receive minimum recommended number of meals (46% received 3+ meals) compared to non-breastfed (only 14% received recommended 4+ meals) Overall: No significant differences by current participation status; for non￾breastfed children 12-23 mo, frequency of feeding and likelihood of receiving minimum recommended number of meals higher among current participants (23%) vs. nonparticipants (8%). Overall: No significant differences 289 Outcome Baseline differences Overall difference between program communities at final survey and improvements since baselinea Difference by program participation a CURRENT PARTICIPATION EVER PARTICIPATED Feeding patterns None Small difference between groups; children in P slightly more likely to have received evening meal yesterday than children in R. Not much improvement since baseline and only 11% index children received evening meal on day before survey. 19% of mothers reported they wake index child for evening meal. Overall: No differences Overall: No differences Dietary diversity None Slightly higher in P than in R; lower than baseline in both areas. Overall: Slightly higher among current participants in both areas. Within program groups: Much higher for currently participating children 6-11 mo in preventive group (4.2 vs. 2.5 food groups for nonparticipants) Higher among participants in some age groups (18-23 in P and 24-29 in R) Nutrient rich foods/new recipes None P more likely than R to have eaten Vit A rich fruits and veg, other fruit and veg, dairy, and other animal-source foods yesterday. Children in P more likely to have had eggs, beef and pork; 63% in P group vs. 41% in R group had recommended recipes >/= 3x/wk, 31% v. 15% had them 7x or more in last week (P v. R). Decline since baseline in consumption of animal source foods No major differences except younger siblings (6-11 mo) of current participants in P more likely to have legumes (72% vs. 44% of nonparticipants) Overall: Nonparticipants less likely to have eaten several nutrient-dense animal source food groups. Within program groups: Preventive: Index children who were participants much more likely to have had flesh foods (meat poultry fish) 7 or more times in the last week (36% of those ever participating vs. 8% of those never participating). 290 Outcome Baseline differences Overall difference between program communities at final survey and improvements since baselinea Difference by program participation a CURRENT PARTICIPATION EVER PARTICIPATED Use of fortified foods and vitamin-mineral supplements None P more likely to have ever received WSB and to be currently receiving WSB. Mean age at receiving WSB was 7.7mo in preventive vs. 13.6 mo in recuperative areas; P areas received rations for more months (11.7 v. 7.5) Vit A supplementation for index children in last 6 mo and for postpartum mothers higher in preventive group. Vit. A capsule coverage increased from 29% to 49%. Iron supplement coverage increased 65% to 83% at final survey. Postpartum Vit. A supplementation increased from 8% to 59% at final survey and higher in preventive group (71%). Overall: Participants much more likely to have consumed WSB yesterday, by design. Within program groups: More participants in preventive group consumed WSB, compared to recuperative, because of greater participation rates. N/A: Only currently participating children receive WSB Participants in both P & R much more likely to have received vitamin A (children and mothers) and somewhat more likely to receive prenatal iron (mothers). Feeding during and after diarrhea None No difference by program group in reporting of giving more liquids, offering the same or more solid foods. After recovery, 61% reported able to give child an extra meal/d. P group more likely to give extra meal (67% v. 56%). Overall: Current participants more likely to report giving an extra meal after diarrhea (76% vs. 54%) Overall: Those who had ever participated were much more likely to report giving an extra meal (66% vs. 37%) Responsive Feeding None Fewer children reported to feed themselves than at baseline, and slightly fewer in P than in R. At final, more women could name positive strategies for coping when child refuses food. 65% named 2+ strategies; this was slightly higher in P than in R. Fewer women reported negative strategies than at baseline, with no difference between P & R. N/A Overall: Mothers who had participated were more likely to report taking action when child refused to eat (both areas). Within program groups: Women in P who had participated were more likely to know 2+ positive strategies. Immunization and care-seeking during illness None Immunization: increased coverage for full immunization since baseline, but coverage remains low (27%), with no difference between program areas. Decrease since baseline in care-seeking during illness, no major differences between P & R. Overall: No differences in immunization or care-seeking related to current participation. Overall: No major differences in immunization or care-seeking. Within program group: Higher rates of full immunization among participants in P, but not in R.