Prepared by: TANGO International Commissioned by: World Vision 2 ENDLINE SURVEY FOR EMERGENCY FOOD SECURITY PROJECT (EFSP) WORLD VISION INTERNATIONAL DECEMBER 2020 2 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com DISCLAIMER The opinions expressed are those of the evaluation team and do not necessarily reflect those of the World Vision. Responsibility for the opinions expressed in this report rests solely with the authors. Publication of this document does not imply endorsement by World Vision of the opinions expressed. ACKNOWLEDGEMENTS The evaluation team would like to acknowledge the contribution of World Vision staff at headquarters and in Bangladesh. Special thanks to Henrik Sturve, Dorance Cooper, Diana Ceci, Noreen Lue, Sisay Wagnew and Asif Ahmed. Thank you to the project participants and respondents who contributed to this study. 3 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Acronyms and Abbreviations ANC Antenatal care BDT Bangladesh Taka CFW Cash for Work CiC Camp in Charge EFSP Emergency Food Security Program EOPS End of project status FCS Food Consumption Score FGD Focus Group Discussion GBF Grameen Bikash Foundation HDDS Household Dietary Diversity Score HH Household HHS Household Hunger Score IGA Income generating activity IYCF Infant Young Child Feeding KII Key Informant Interview M&E Monitoring and Evaluation MCHN Maternal and Child Health and Nutrition MEAL Monitoring, Evaluation, Accountability and Learning NGO Non-governmental organization ODK Open Data Kit PDM Post Distribution Monitoring PIRS Performance Indicator Reference Sheets PNC Postnatal care PPS Probability Proportional sample to Size rCSI Reduced Coping Strategies Index USD United States Dollar WASH Water, Sanitation, and Hygiene WV World Vision 4 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com CONTENTS ACRONYMS AND ABBREVIATIONS......................................................................................3 1. EXECUTIVE SUMMARY.........................................................................................................7 1. INTRODUCTION................................................................................................................. 10 A. BACKGROUND ON EFSP............................................................................................................. 10 B. OBJECTIVES OF ENDLINE EVALUATION............................................................................... 10 2. METHODOLOGY................................................................................................................. 11 A. SAMPLING STRATEGY................................................................................................................... 11 QUANTITATIVE DESIGN ................................................................................................................. 11 QUALITATIVE DESIGN...................................................................................................................... 12 B. DATA MANAGEMENT AND ANALYSIS.................................................................................. 12 DATA COLLECTION AND ENTRY .............................................................................................. 12 DATA QUALITY CONTROL........................................................................................................... 12 DATA ANALYSIS ................................................................................................................................. 13 C. TEAM TRAINING............................................................................................................................. 13 D. LIMITATIONS .................................................................................................................................... 13 3. FINDINGS................................................................................................................................ 14 A. PROJECT ACHIEVEMENTS AGAINST TARGETS................................................................... 14 IMPROVED FOOD SECURITY......................................................................................................... 15 NUTRITION .......................................................................................................................................... 17 COPING STRATEGIES........................................................................................................................ 21 B. EFFECTIVENESS AND RELEVANCE OF FRESH FOOD VOUCHER MODALITY AND IMPACT ON LOCAL MARKETS...................................................................................................................................... 22 IMPACT ON LOCAL MARKETS...................................................................................................... 23 C. EFFECTIVENESS OF COMPLEMENTARY INTERVENTIONS.............................................. 24 MULTIPURPOSE TRAINING CENTERS........................................................................................ 24 5 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com IMPROVED LIVELIHOODS ............................................................................................................... 25 COMMUNITY COOKING & LEARNING CENTERS (CCLCS).............................................. 26 D. IMPACT OF COVID-19................................................................................................................... 30 E. STRENGTHS, CHALLENGES AND LESSONS LEARNED..................................................... 31 STRENGTHS AND BEST PRACTICES ........................................................................................... 31 CHALLENGES ....................................................................................................................................... 32 LESSONS LEARNED............................................................................................................................ 34 4. CONCLUSIONS.................................................................................................................... 35 5. APPENDIX............................................................................................................................... 37 5.1 SAMPLE SIZE CALCULATION........................................................................................................ 37 5.2 SAMPLE WEIGHTS FOR STRATA AND CLUSTERS ................................................................ 37 5.3 TEAM TRAINING AGENDA............................................................................................................ 38 6. ANNEX..................................................................................................................................... 40 ANNEX 1 QUANTITATIVE SURVEY TOOL..................................................................................... 40 ANNEX 2 QUALITATIVE SURVEY TOOL......................................................................................... 40 ANNEX 3 QUALITATIVE DATA ENTRY MATRICES..................................................................... 40 ANNEX 4 INCEPTION REPORT........................................................................................................... 40 ANNEX 5 QUANTITATIVE DATA ...................................................................................................... 40 ANNEX 6 DETAILED EFSP TOPLINE INDICATOR ANALYSIS TABLES .................................. 40 ANNEX 7 QUALITATIVE DATA........................................................................................................... 40 ANNEX 8 EFSP MONITORING AND EVALUATION FRAMEWORK....................................... 40 6 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com TABLES Table 1: Project Indicators..............................................................................................................................................................7 Table 2: Quantitative Sample Size.............................................................................................................................................. 12 Table 3: Project Indicators........................................................................................................................................................... 14 Table 4: Improved Food Security and Nutrition Indicator Values..................................................................................... 19 Table 5: Improved Livelihood Indictors.................................................................................................................................... 25 Table 6: MATERNAL AND CHILD HEALTH DECISION MAKING............................................................................... 29 FIGURES Figure 1: Food Consumption Score........................................................................................................................................... 16 Figure 2: Prevalence of households with moderate or severe Household Hunger Scale (HHS) score.................... 16 Figure 3: Percentage of households with improved dietary diversity score (HDDS) ................................................... 17 Figure 4: percentage of beneficiaries who feel it is important to source and utilize nutritious food, specifically for women of child bearing age......................................................................................................................................................... 18 Figure 5: percentage of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices)............................................................................................................................................. 20 Figure 6: Percentage of households utilizing coping strategies in the past 12 months.................................................. 22 Figure 7: Impacts of COVID-19 on Households in the Past 12 Months........................................................................... 30 7 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com 1.Executive Summary Purpose and Objective of Endline Evaluation. TANGO completed the Baseline evaluation for the EFSP project in December 2019. This endline evaluation is designed to evaluate the achievements of the activity in relation to the goal, objectives, results, and targets as well as the effectiveness and relevance of the fresh food vouchers modality and complementary interventions to achieve activity outcomes. This is supported by identification of best practices, lessons learned, strengths, and challenges in the activity design. This endline evaluation also includes an evaluation of the activities impact on local markets, and how it affected certain groups of interest (women and men; youth). Methods. This mix-method assessment included data collection through a quantitative household survey and qualitative key informant interviews and focus group discussions with refugee beneficiaries. Data was not collected in host communities. The design was based on the project’s monitoring and evaluation framework in coordination with World Vision. 666 household surveys were completed out of the total sample size of 680, with a non-response rate of 2.1 percent. Twenty focus group discussions and twenty key informant interviews were completed. TANGO reviewed uploaded data on a daily basis, and ensured data quality through daily in-briefs and de-briefs with research teams during data collection. Data analysis triangulated quantitative, qualitative, and field observation data to ensure a high level of detail and diversity of perspectives are incorporated into the report. Indicator Values. Based on point estimates, of the 8 total topline indicators, 4 met end of project targets, 3 exceeded the end of project targets, and one did not meet the end of project target.1 Table 1: Project Indicators Indicator 2019 Baseline 2020 Endline End of Project Target Status at Endline Purpose 1: Increased consumption of diverse and quality foods that meet the nutritional requirements of vulnerable households in Refugee Camps and Host Communities % Poor (0-21) Food Consumption Score (FCS) 0.6% 0.6% 0.6% met % Borderline (>21 to 35) FCS 14.9% 14.5% 14.4% met % Adequate (>35) FCS 84.5% 84.9% 85% met Intermediate Outcome 1.1: Improved access to and consumption of diverse and nutritious foods for 34,112 refugee HHs and 5,229 host community HHs directly impacted by the refugee influx. Prevalence of households with moderate or severe Household Hunger Scale (HHS) score. 7.2% 1.7% 5.5% exceeded % of households with improved dietary diversity score (HDDS) % HHs that consume more than the average number of food groups 41.4% 53.9% 50% exceeded 1 Targets “met” indicates that the point value estimates have achieved the target, but are less than 95% confidence that the true population is above the target. Targets “exceeded” indicates that point value estimates have exceeded the target with 95% confidence interval. Further detail is included in the “Project Achievements Against Targets” section of this report. 8 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com % of beneficiaries who can name 3 key nutrition messages (IYCF, NC, CD) 17.1% 48.3% 75% not met % of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices) 57.1% 82.9% 75% exceeded Intermediate Outcome 1.2: Enhanced Refugee and Host Community HHs’ capacities to sustainably access to nutritious and diversified food % of HHs who have more than two sources of income as a result of project intervention 5.6% 0.0% Not applicable for refugees. The project scope is not increasing income generation for refugees but for hosts only- The support for refugees is to engage in micro home gardening for household consumption % of households engaged in income generating activities 0.3% 2.3% target setting for IGA only for host community Purpose 2: Capacity of Refugees and Host Communities to withstand future shocks enhanced Reduced Coping Strategies Index (rCSI) % HHs above the mean 41.4% 39.4% 38.5% met Conclusions. Despite the unanticipated impacts of COVID-19 and the related government lockdown and restrictions which affected program implementation, EFSP practiced effective adaptive management to address evolving needs and opportunities, and achieved intended program objectives for targeted refugee households in Cox’s Bazar District, Bangladesh. The FFV modality is found to have been highly effective, and even within the context of variable food prices and restricted movement due to COVID-19, refugee beneficiaries food security improved since baseline. All three indicators (Food Consumption Score, Household Hunger Scale, and Household Dietary Diversity Score) used to measure food security indicate a positive direction of change indicating households are eating a wider diversity of foods and are more food secure than at baseline. Voucher entitlement for beneficiaries were based on two family size categories (1-7 members and 8+). The vouchers did not provide an adequate amount of food to meet the needs of families with eight or more members. Nearly all respondents reported awareness of the Corona Virus/COVID-19 and nearly 40 percent reported being directly impacted by COVID-19. The primary coping strategy to cope with the impacts of COVID-19, as reported by over 75 percent of households, is to receive food aid or assistance from an NGO (including food/cash-for-work). Interviews with beneficiaries found that households feel the impact of COVID-19 was not as severe as it would have been, had they not had access to FFV. Overall, the average reduced coping strategies index (rCSI) score increased from baseline to endline, indicating that more severe coping strategies are being used at endline as compared to baseline. This is in alignment with expected impacts of COVID-19 and related restrictions. At endline, more households reported 9 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com needing to borrow food or rely on help for food access, reduced adult consumption, and reduced number of meals eaten per day. The effectiveness of the complementary interventions, multipurpose training centers (MTC) and community cooking and learning centers (CCLCs) was made possible due to the efficient adaptive management of EFSP staff and donor support. Programming shifted to a remote model as much as possible. EFSP staff successfully adapted MTC programming to produce cloth masks for personal protection equipment (PPE). This reclassified the MTCs as a lifesaving activity, which allowed them to reopen in a limited capacity in alignment with COVID-guidelines. Lead mothers were utilized as a focal point for program information dissemination, which successfully allowed for CCLC program messaging to still be communicated through communities from lead mothers through their neighbors and social networks. 55 out of 62 planned CCLCs were established. CCLCs were adapted to include smaller class sizes which allowed for women to still utilize the kitchens in a limited capacity. ***** 10 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com 1.Introduction A. Background on EFSP The overall goal of the Emergency Food Security Program (EFSP) is to improve food security and nutrition status of Rohingya refugees and vulnerable host community members in Cox’s Bazar District, Bangladesh. EFSP is a two-year emergency food assistance project funded by United States Agency for International Development (USAID). EFSP started on 1st September 2019. For programming in refugee camps, the project continued for 12 months, while for the host community project activities will continue for 24 months. The overall goal of the project is to improve the food security and nutrition status of Rohingya refugees and vulnerable host community members and has two main purposes; Purpose 1: Improved access to and consumption of diverse and nutritious foods for 34,112 refugee HHs; while also addressing the short-term food security needs and strengthening the medium-term livelihoods recovery of 5,229 of the most vulnerable host community HH’s directly impacted by the refugee influx. • Intermediate outcome 1.1: Improved access to and consumption of diverse and nutritious foods for 34,112 refugee HHs and 5,229 host community HHs directly impacted by the refugee influx • Intermediate outcome 1.2: Enhanced Refugee and Host Community HHs’ capacities to sustainably access nutritious and diversified food Purpose 2: Capacity of Refugees and Host Communities to withstand future shocks enhanced. • Intermediate outcome 2.1: Community assets and infrastructure restored in Host communities through CFW. • Intermediate outcome 2.2: Community savings groups strengthened • Intermediate outcome 2.3: Social cohesion reinforced between host community and refugee HHs The main interventions included unconditional, restricted commodity value-based fresh food voucher transfers. 34,112 HHs were targeted for selection to benefit from the fresh food voucher (FFV) modality. Specific consideration is given to households with pregnant or lactating mothers, children under the age of 5, female headed, containing family members with disability or chronic illness, and/or without any HH member currently paid to volunteer for an aid organization or participating in cash for work. Commentary activities included the establishment of Community Cooking & Learning Centers (CCLCs) and Multipurpose Training Centers (MTC). B. Objectives of Endline Evaluation TANGO completed the Baseline evaluation for the EFSP project in December 2019. This endline evaluation is designed to evaluate the (1) achievements of the activity in relation to the goal, objectives, 11 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com results, and targets as well as (2) the effectiveness and relevance of the fresh food vouchers modality and complementary interventions to achieve activity outcomes. This is supported by identification of best practices, lessons learned, strengths, and challenges in the activity design. This endline evaluation also includes an evaluation of the activities impact on local markets, and how it affected certain groups of interest (women and men; youth). This endline evaluation only includes data collected within the refugee camps. 2.Methodology A. Sampling Strategy Quantitative design Although the baseline was conducted in both host communities and refugee camps, the endline only collected data in refugee camps. Utilizing a two-stage cluster sampling procedure, the clusters were selected using Probability Proportional sample to Size (PPS) procedure. The two-stage sampling involved the selection of sample sub-blocks (clusters) across the 13 project working camps. Equal number of samples were drawn randomly from the project participants sampling frame in a sampled cluster. The PPS sampling procedure and equal cluster sample size ensured that the participating households in the different clusters had the same chance to be selected. EFSP project participating households were the primary selection units of the overall sampling frame. 34 clusters (sub-blocks) were selected across the camps using the PPS sampling procedure. Twenty project participating households from each of the selected 34 clusters were selected randomly from the sampling frame of project participating households. The table below shows the summary of the proposed revised sample size and sampling approach: Items Refugee No. of EFSP project participants 77,733 households No. of camps 13 camps Sample size 680 No. of PPS sampled cluster (sub-blocks) 34 Cluster Sample size 20 Total, 666 response cases are included in the data set at endline, as presented in Table 2, below. 12 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Table 2: Quantitative Sample Size Qualitative design Qualitative data was collected via 20 key informant interviews (KIIs) and 20 focus group discussions (FGDs), equally distributed across the same strata used for the household survey. The sampling strategy for the FGDs was purposive and focused on identifying members from ‘typical’ households to participate in the sessions. In addition, TANGO included at 1-2 FGDs per strata that focused on ‘positive deviants’ to understand the available/potential resilience capacities that can be utilized by participants and may be an area of focus for the World Vision EFSP program. TANGO also completed KIIs with key program staff at office and field levels to triangulate qualitative findings and get additional insight into the implementation process. B. Data Management and Analysis Data Collection and Entry For quantitative data, electronic data capture devices, equipped with ODK (Open Data Kit) software were used for data collection. The survey tool is included in Annex 1. The use of mobile devices and an electronic questionnaire form improved data quality by allowing data validation rules and consistency checks to be integrated in the data collection process. Mobile devices reduced the data entry burden as data were entered immediately at the interviewer level and records are uploaded to a cloud server using the built-in internet connectivity of the devices. Qualitative data was collected through FGDs and KIIs using semi-structured tools outlining key topics and sub-topics included in Annex 2. Interviews were completed by a two-person team consisting of a facilitator and note-taker. All qualitative data was captured using matrices included in Annex 3. Data Quality Control Quality control was assured through in-field supervision and real time data quality review. Quantitative (ODK uploads) and qualitative data (emailing of the data matrices completed in or translated into English) were submitted to TANGO at least every 2 days, depending on connectivity. Feedback was provided regularly to the field teams through WhatsApp/email/telephone to support real-time data quality assurance. During data collection, the World Vision survey coordinator and supervisors organized daily in-briefs and de-briefs with quantitative and qualitative survey teams which focused on feedback on data quality and team members’ performance based. During debriefings, the World Vision team supervisors and the Baseline Endline # % # % Sample Size 350 680 Response cases 331 94.6% 666 97.9% Non-response cases 19 5.4% 14 2.1% 13 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com survey coordinator reviewed specific questions/topics in completed tools for consistency, reliability and accuracy before uploading data to TANGO. The survey coordinator and team supervisors also utilized additional data quality monitoring controls including re-interviewing a selected number of households and verification that non-response households are unavailable. The survey coordinator provided daily progress reports to TANGO via email, identifying challenges encountered and how these were solved in line with the established endline protocols developed by TANGO. TANGO, in turn, provided regular feedback on the quality of uploaded data and any irregularities in the interviewer logs Data Analysis Quantitative data was analyzed using Stata 15.1. Qualitative data was analyzed using MS Excel topline matrices. The team adopted a strategy of triangulation by examining the issues through various lenses and different perspectives, including data collected through baseline/endline surveys, KIIs and FGDs. The endline report includes triangulated findings against baseline data from both quantitative and qualitative data. C. Team Training TANGO conducted a four-day quantitative and qualitative team training through a combination of classroom training, practice and a field test. Trainings for qualitative and quantitative teams were conducted in parallel with real-time coordination between the two TANGO trainers to ensure consistent and accurate knowledge transfer across the overall baseline team. The detailed training schedule is included in Appendix 5.3. The training covered technical, logistical, and leadership components. Technical aspects included interview and body language techniques, handling tablets and data management, and data quality control measures. The training was designed to guide the team on supervisor and enumerator roles and responsibilities, rules, behaviors and ethics (including gender sensitivity), respondent selection, detailed review of the survey, and direct observation tools and mock interviews/role playing. The training also covered aspects of interviewer and respondent bias, how to establish informed consent, and build essential trust with respondents and key informants. D. Limitations i. The baseline sample was 350 households for the camps, whereas, the endline sample for the same group was 680. Although the sample sizes are different between the two rounds, the 95% confidence intervals for all indicators were found to be similar for both baseline and endline samples. This implies that the variability of the indicators is similar in the two survey rounds. Therefore, the smaller sample size at baseline does not impact the statistical validity of results. ii. This endline evaluation did not include a control or comparison group (host community households) which limits the interpretation of the effect of project interventions. This is mitigated by focusing analysis and interpretation on refugee households only. 14 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com iii. Training for qualitative researchers and quantitative enumerators had to be done virtually due to COVID-19 travel restrictions limiting the consultant’s ability to enter Cox Bazar. This was mitigated by having an in-person facilitator collaborate with EFSP staff to provide hands-on training and start￾up support in coordination with TANGO staff utilizing the remote-support model. A field test was included in training, which allowed for observation of field practices and appropriate feedback for enumerators prior to actual data collection. This ensured a high level of preparedness for field teams. 3.Findings This section is organized around the endline evaluation objectives and includes subsections on: achievements of the activity in relation to the goal, objectives, results, and targets; effectiveness and relevance of the fresh food vouchers modality; impact on local markets; effectiveness of complementary interventions to achieve activity outcomes; best practices, lessons learned, strengths, and challenges in the activity design. This endline evaluation also includes an evaluation of how program activities affected certain groups of interest (women and men; youth) integrated into each section. A. Project Achievements against Targets Overall, the EFSP project met or exceeded all but one of the project indicators. The following table gives a high-level overview of the project indicator values at baseline and endline, measured against the intended targets. End of project targets were revised in alignment with recommendations from the baseline evaluation. Details for each value are included in the descriptive sections below. The indicators included in this endline were agreed upon between TANGO and World Vision during the inception phase, based on the EFSP monitoring and evaluation framework, included in Annex 8 of this report. Based on point estimates, out of the 8 total topline indicators, 4 met end of project targets, 3 exceeded the end of project targets, and one did not meet the end of project target. Targets “met” indicates that the point value estimates have achieved the target, but with less than 95% confidence that the true population is above the target. Targets “exceeded” indicates that point value estimates have exceeded the target with 95% confidence interval. Targets “not met” have not been met with 95% confidence interval. This is presented in Table 3 below. Table 3: Project Indicators2 Indicator 2019 Baseline 2020 Endline End of Project Target Status at Endline Purpose 1: Increased consumption of diverse and quality foods that meet the nutritional requirements of vulnerable households in Refugee Camps and Host Communities % Poor (0-21) Food Consumption Score (FCS) 0.6% (-0.30, 1.60) 0.6% (-0.10, 1.40) 0.6% met % Borderline (>21 to 35) FCS 14.9% (7.60, 22.20) 14.5% (10.7, 18.3) 14.4% met 2 95% confidence interval reported in parenthesis under point estimates 15 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com % Adequate (>35) FCS 84.5% (77.0, 92.0) 84.9% (80.8, 89.0) 85% met Intermediate Outcome 1.1: Improved access to and consumption of diverse and nutritious foods for 34,112 refugee HHs and 5,229 host community HHs directly impacted by the refugee influx. Prevalence of households with moderate or severe Household Hunger Scale (HHS) score. 7.2% (1.20, 13.2) 1.7% (0.50, 2.90) 5.5% exceeded % of households with improved dietary diversity score (HDDS) % HHs that consume more than the average number of food groups 41.4% (31.1, 51.7) 53.9% (47.8, 60.1) 50% exceeded % of beneficiaries who can name 3 key nutrition messages (IYCF, NC, CD) 17.1% (9.73, 24.6) 48.3% (35.8, 60.8) 75% not met % of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices) 57.1% (34.8, 79.3) 82.9% (77.4, 88.5) 75% exceeded Intermediate Outcome 1.2: Enhanced Refugee and Host Community HHs’ capacities to sustainably access to nutritious and diversified food % of HHs who have more than two sources of income as a result of project intervention 5.6% (2.87, 8.42) 0.0% (0.0, 0.0) Not applicable for refugees. The project scope is not increasing income generation for refugees but for hosts only- The support for refugees is to engage in micro home gardening for household consumption. % of households engaged in income generating activities 0.3% (-0.39, 1.07) 2.3% (0.30, 4.20) target setting for IGA only for host community Purpose 2: Capacity of Refugees and Host Communities to withstand future shocks enhanced Reduced Coping Strategies Index (rCSI) % HHs above the mean 41.4% (27.9, 54.9) 39.4% (31.0, 47.8) 38.5% met The only target that was not met is the percentage of beneficiaries who can name three key nutrition messages for Infant Young Child Feeding (IYCF), nutrition counselling (NC), and cooking demonstrations (CD). Particularly in the context of COVID-19 and disruptions in planned activities and associated group implementation models, the target of 75 percent for specific messaging is high. The target of beneficiaries who can name at least three key nutrition messages based on the seven maternal and child health and nutrition (MCHN) practices is exceeded by 7.9 percent. Improved Food Security Food security increased for refugee beneficiaries from baseline to endline. To measure refugee food security status in alignment with project purpose one: “Increased consumption of diverse and quality foods that meet the nutritional requirements of vulnerable households in Refugee Camps and Host Communities”, three food security measures are utilized; the Food Consumption Score (FCS), Household Hunger Scale (HHS), and Household Dietary Diversity Score (HDDS). In conjunction, these measures allow an overall understanding of the food security status of the sample. 16 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com The Food Consumption Score (FCS) is a composite score based on dietary diversity, food frequency, and the relative nutritional importance of different food groups. To collect FCS data, a questionnaire is used to ask respondents about the frequency of their households' consumption of nine food groups over the previous seven days. To calculate the FCS, the consumption frequencies are summed and multiplied by the standardized food group weight, and households are then be classified into three groups based on their weighted scores - poor, borderline, or acceptable - using the World Food Program's recommended cutoff points.3 The data is presented in Table 4. At baseline, the FCS indicated that the majority of refugee (84.5 percent) respondents had acceptable food consumption score. This increased slightly to 84.9% at endline. Relatively few (0.6 percent at baseline and endline) reported poor FCS. The average Food Consumption Score (FCS) increased from 45.8% at Baseline to 47.4% at Endline. This would indicate that respondents are consuming foods from a variety of food groups with relative frequency. FCS scores did not vary greatly between male and female headed households at Baseline or Endline. The Household Hunger Scale (HHS) is a food deprivation scale that measures the percent of households experiencing hunger. To collect data for this indicator, the person in the household in charge of food preparation is asked about the frequency with which three events were experienced by any household member in the last four weeks: 1. No food at all in the house 2. Went to bed hungry 3. Went all day and night without eating. 4 The HHS found that the majority (92.8 percent baseline, 98.3 percent endline) of respondents had experienced no or little hunger in the last four weeks. The data is presented in Table 4. The prevalence of households with moderate or severe Household Hunger Scale (HHS) score decreased from 7.2% at Baseline to 1.7% at Endline. 3 USAID Food for Peace. February 2019. Indicators for Emergency Program Performance Indicator Reference Sheets. P. 7. 4 USAID Food for Peace. February 2019. Indicators for Emergency Program Performance Indicator Reference Sheets. P. 12. 0.6% 14.9% 84.5% 0.6% 14.5% 84.9% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % Poor (0-21) Food Consumption Score (FCS) % Boderline (>21 to 35) FCS % Adequate (>35) FCS Percentage of households with poor, borderline, and adequate Food Consumption Score (FCS) Baseline 2019 Endline 2020 Figure 1: Food Consumption Score 92.8% 7.2% 98.3% 1.7% 0% 20% 40% 60% 80% 100% % No/little hunger % moderate/severe hunger Prevalence of households with moderate or severe Household Hunger Scale (HHS) score 2019 Baseline 2020 Endline Figure 2: Prevalence of households with moderate or severe Household Hunger Scale (HHS) score. 17 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com The Household Diet Diversity Score (HDDS) is meant to provide an indication of household economic access to food, thus items that require household resources to obtain, such as condiments, sugar and sugary foods, and beverages, are included in the score. Individual scores are meant to reflect the nutritional quality of the diet. HDDS includes an aggregation of 12 total food groups, including: cereals; white tubers and roots; vegetables; fruits; meat; eggs; fish and other seafood; legumes; nuts and seeds; milk and milk products; oils and fats; sweets, spices, condiments and beverages. There are no established cut-off points in terms of number of food groups to indicate adequate or inadequate dietary diversity for the HDDS. Because of this it is recommended to use the mean score or distribution of scores for analytical purposes and to set program targets or goals.5 More than half of refugee (41.4 percent at baseline, 53.9 percent at endline) respondents had above the average number of food groups (5.2 food groups at baseline, 5.7 food groups at endline). Very few refugees’ households (0.9 percent) had more than eight food groups on average at baseline. At endline, this increased to 5.8 percent of refugee households having more than eight food groups on average. The data is presented in Table 4. All food security indicators met or exceeded the planned project targets. Nutrition The nutrition status of refugee households participating in the EFSP program has improved since baseline. Nearly all beneficiaries reported they feel it is important to source and utilize nutritious food, specifically for women of child bearing age (95.0 percent endline, 63.2 percent baseline), children under five (95.4 percent endline, 74.3 percent baseline), and for women, children, or both (96.5 percent endline, 76.3 percent baseline). This is illustrated in Figure 4, below. 5 FAO. 2010. Reprint 2013. Guidelines for Measuring Household and Individual Dietary Diversity. Prepared by Gina Kennedy, Terri Ballard and Marie Claude Dop Nutrition and Consumer Protection Division, Food and Agriculture Organization of the United Nations. ISBN 978-92-5-106749-9. 0.9% 41.4% 5.8% 53.9% 0% 20% 40% 60% 80% 100% % with >8 food groups % with >average food groups Percentage of households with improved dietary diversity score (HDDS) 2019 BASELINE 2020 ENDLINE Figure 3: Percentage of households with improved dietary diversity score (HDDS) 18 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Figure 4: percentage of beneficiaries who feel it is important to source and utilize nutritious food, specifically for women of child bearing age Interviews with project staff and beneficiaries support these findings. FGDs within the camps found that awareness of nutrition and the importance of eating a variety of fresh vegetables, fruits, milk and eggs increased. However, increased awareness did not necessarily translate into increased consumption; across the data households reported food prices increased due to COVID-19 and related restrictions on movement. Vegetable prices were specifically cited as being too high for many households. FGDs reported: “Everyone in the camp wants to eat nutritious food but cannot afford it due to lack of money”. The fresh food vouchers (FFV) were cited in several FGDs as being the primary source of nutritious foods in the last year. Some FGDs referenced kitchen gardens started with seed received from EFSP programming as a source for additional fresh vegetables, however, the production was limited and the primary source was FFV. FGDs reported that the seeds distributed by EFSP were limited. KIIs with program staff show that the second round of seed distribution did not take place due to COVID-19 programming changes. 63.2% 74.3% 76.3% 95.0% 95.4% 96.5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% A.Women of child bearing age B.Children U5 C.Women or children or both % of beneficiaries who feel it is important to source and utilize nutritious food, specifically for women of child bearing age 2019 BASELINE 2020 ENDLINE 19 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Table 4: Improved Food Security and Nutrition Indicator Values INDICATOR SURVEY ROUNDS 2019 BASELINE 2020 ENDLINE SIG. Value 95% CI DEF Value 95% CI DEF 1. % of households with poor, borderline, and adequate Food Consumption Score (FCS) Average Score (FCS) 45.8 (43.8, 47.9) 1.54 47.4 (45.5, 49.3) 1.84 % Poor (0-21) 0.6% (-0.30, 1.60) 0.99 0.6% (-0.10, 1.40) 1.19 % Borderline (>21 to 35) 14.9% (7.60, 22.20) 1.73 14.5% (10.7, 18.3) 1.37 % Adequate (>35) 84.5% (77.0, 92.0) 1.74 84.9% (80.8, 89.0) 1.45 2. Prevalence of households with moderate or severe Household Hunger Scale (HHS) score. Average Score (HHS) 0.27 (0.09, 0.45) 2.01 0.12 (0.08, 0.16) 1.29 * % No/little hunger 92.8% (86.8, 98.8) 1.95 98.3% (97.1, 99.5) 1.15 * % moderate/severe hunger 7.2% (1.20, 13.2) 1.95 1.7% (0.50, 2.90) 1.15 * 3. % of households with improved dietary diversity score (HDDS) Average Score (HDDS) 5.2 (4.82, 5.51) 1.95 5.7 (5.48, 6.00) 1.89 *** % with >8 food groups 0.9% (-0.13, 1.92) 0.92 5.8% (3.50, 8.20) 1.27 *** % with >average food groups 41.4% (31.1, 51.7) 1.75 53.9% (47.8, 60.1) 1.57 ** n 331 666 7. % of beneficiaries who have received key nutrition messages IYCF 27.5% (15.4, 39.6) 1.30 74.0% (66.0, 82.0) 0.99 *** Nutrition counselling 35.9% (22.9, 48.8) 1.29 72.4% (60.7, 84.1) 1.42 *** Cooking demonstration 18.1% (11.0, 25.1) 0.88 53.2% (41.7, 64.7) 1.25 *** 8. % of beneficiaries who can name 3 key nutrition messages (IYCF, NC, CD) 17.1% (9.73, 24.6) 0.94 48.3% (35.8, 60.8) 1.36 *** n 108 124 9. % of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices) 57.1% (34.8, 79.3) 2.86 82.9% (77.4, 88.5) 1.36 ** 1. At least 4 ANC check-up during pregnancy 42.5% (29.9, 55.1) 1.62 56.9% (48.0, 65.8) 1.65 * 2. First PNC visit with 72 hours of delivery 21.9% (13.8, 30.0) 1.24 43.6% (36.8, 50.3) 1.24 *** 3. More food intake during pregnancy 51.6% (31.3, 71.9) 2.59 88.1% (82.5, 93.8) 1.60 *** 4. More day-time rest during pregnancy 45.8% (26.6, 65.0) 2.46 81.4% (73.7, 89.0) 1.80 *** 5. Baby should put to mother's breast immediately/within one hour after delivery 74.4% (60.3, 88.5) 2.06 67.4% (62.2, 72.7) 1.03 6. Up to 6 months of age the baby should be exclusively breastfed 34.8% (19.8, 49.8) 2.00 44.3% (37.6, 51.0) 1.24 7. From 6 months of age the baby should start with solid/semi-solid food 16.0% (7.6, 24.4) 1.45 20.4% (15.4, 25.5) 1.14 10. % of beneficiaries who feel it is important to source and utilize nutritious food, specifically for women of child bearing age. A.Women of child bearing age 63.2% (42.3, 84.0) 2.75 95.0% (91.3, 98.8) 1.59 *** B.Children U5 74.3% (59.8, 88.8) 2.11 95.4% (92.0, 98.7) 1.45 *** C.Women or children or both 76.3% (62.9, 89.8) 2.01 96.5% (94.1, 98.9) 1.20 *** n 190 349 20 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com The percentage of respondents who could name three key nutrition messages was measured in two different ways. First, the percentage of beneficiaries who reported they specifically received key nutrition messages on Infant Young Child Feeding (IYCF), nutrition counselling, and cooking demonstrations. At endline, 74.0 percent (27.5 percent at baseline) reported receiving information on IYCF, 72.4 percent (35.9 percent at baseline) received nutrition counselling, and 53.2 percent (18.1 percent at baseline) participated in cooking demonstrations. Less than half (48.3 percent at endline, 17.1 percent at baseline) of respondents could name three key nutrition messages specific to IYCF, nutrition counselling or cooking demonstration programming. Second, beneficiaries were asked to name at least three of the seven Maternal Child Health and Nutrition (MCHN). Overall, 82.9 percent were able to name at least three of the seven key nutrition messages, compared to 57.1 percent at baseline. This exceeds the end of project target of 75.0 percent of households. This is illustrated in Figure 5, below. Figure 5: percentage of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices) 57.1% 42.5% 21.9% 51.6% 45.8% 74.4% 34.8% 16.0% 82.9% 56.9% 43.6% 88.1% 81.4% 67.4% 44.3% 20.4% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices) 1. At least 4 ANC check-up during pregnancy 2. First PNC visit with 72 hours of delivery 3. More food intake during pregnancy 4. More day-time rest during pregnancy 5. Baby should put to mother's breast immediately/within one hour after delivery 6. Up to 6 months of age the baby should be exclusively breastfed 7. From 6 months of age the baby should start with solid/semi-solid food % of beneficiaries who can name 3 (at least 3) key nutrition messages (based on the knowledge of 7 MCHN practices) 2020 ENDLINE 2019 BASELINE 21 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Coping Strategies When livelihoods are negatively affected by a shock or stress, households may adopt various mechanisms (strategies) to cope with reduced or declining access to food as a result of the shock/stress. The Reduced Coping Strategies Index (rCSI) is a proxy indicator of household food insecurity that is based on a list of behaviors (coping strategies). The index reflects both the frequency of each behavior (i.e., how many times the coping strategy was used by any member of the household) and severity (i.e., how serious the strategy). The rCSI is based on a list of five food-related coping strategies that the household used in the seven days prior to the survey. The rCSI raw scores are calculated by multiplying the frequency with which a behavior was used by a universally used severity weight, then summing the weighted scores for each coping strategy. 6 In its simplest form, monitoring changes in the rCSI score indicates whether household food security status is declining or improving. There are no universal thresholds for rCSI, but the higher the rCSI, the more severe the coping is applied by a household. The rCSI raw scores are calculated by multiplying the frequency with which a behavior was used by a universally used severity weight, then summing the weighted scores for each coping strategy. The maximum raw score for the rCSI is 56, i.e., a household that used all five strategies every day for the last 7 days would have a raw score of 56. Higher rCSI indicates a worse food security situation and vice versa. INDICATOR SURVEY ROUNDS 2019 BASELINE 2020 ENDLINE SIG. Value 95% CI DEF Value 95% CI DEF 6. Reduced Coping Strategies Index (rCSI) Mean 5.2 (4.20, 6.16) 2.51 6.8 (5.34, 8.24) 2.48 * Median 4.5 - - 4.0 - - Standard Deviation 3.29 - - 7.43 - - Minimum - - - 0.0 - - Maximum 19.0 - - 41.0 - - % above the mean 41.4% (27.9, 54.9) 2.30 39.4% (31.0, 47.8) 2.18 The mean rCSI score was 5.2 at baseline and 6.8 at endline. This indicates that more severe coping strategies were utilized by endline as compared to baseline. This is in alignment with expected impacts on households in the camp experiencing the impacts of COVID-19 and related restrictions. Figure 6 illustrates the percentage of households utilizing different coping strategies in the past 12 months. More households reported needing to borrow food or rely on help for food access, reduced adult consumption, and reduced number of meals eaten per day at endline. 6 USAID Food for Peace. February 2019. Indicators for Emergency Program Performance Indicator Reference Sheets. P. 10. 22 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Figure 6: Percentage of households utilizing coping strategies in the past 12 months B. Effectiveness and Relevance of Fresh Food Voucher Modality and Impact on Local Markets The primary activity of the EFSP program is the provision of fresh food vouchers (FFV). These vouchers were unconditional, restricted commodity value-based fresh food vouchers transfers for a 10 months provision period. With the support of Camp in Charge (CiC) and camp stakeholders (this includes site management, UNHCR, CiC and WFP), 34,112 HHs were selected to receive FFV. Specific consideration was given to households with pregnant or lactating mothers, children under the age of 5, female headed, containing family members with disability or chronic illness, and/or without any HH member currently paid to volunteer for an aid organization or participating in cash for work. Internal project monitoring reports that 100% of the 34,112 targeted households did receive FFV, which was confirmed by beneficiary feedback from qualitative data. Interviews with EFSP staff and participants show that the fresh food vouchers were highly effective in providing food for households residing in the camps. This is supported by the Post Distribution Monitoring Report (PDM) for Fresh Food Voucher Distribution report prepared by the MEAL team in July 2020. The PDM survey reported that 90% of beneficiaries are satisfied with the type and quality of foods provided. Key informant interviews (KIIs) with key project staff support this. The EFSP FFV included 14 food items; green chilies, dry red chili, onion, lemons, eggs, green leafy vegetables/spinach, garlic, potato, pumpkin, salt, dry fish, sugar, turmeric powder and chili powder. The PDM survey reported 10% of beneficiaries reported dissatisfaction with food quality. Interviews with project staff and beneficiaries found dissatisfaction with food quality was primarily on dried fish. FGDs with beneficiaries and KIIs with field staff indicate that fresh fish would have been preferred by recipients. KIIs indicate that program staff worked with vendors regarding the dried fish quality, and felt the quality improved as a result. KIIs with program staff found that EFSP held monthly vendor meetings, which 61% 25% 29% 17% 4% 60% 57% 28% 23% 12% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Rely on less expensive or less preferred food Borrow food or rely on help Limit/reduce portion size at meal time Reduce adult consumption Reduce number of meals eaten per day Percentage of households utilizing coping strategies in the past 12 months Baseline Endline 23 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com allowed for timely information sharing and resolution of issues such as food quality throughout the program period. The FY20 Q3 internal monitoring report stated: “Regular meetings with FFV vendors ensured quality food and adherence to policies on protection, safeguarding, conflict of interest, code of conducts, price monitoring, and complaint and feedback reporting”.7 Interviews with beneficiaries and project staff found that the amount of food provided was not sufficient to last a full month for an entire household. The PDM survey found that 71% of food received did not last the full month.8 Vouchers were provided in two sizes, one for small family households including one to seven members, and one large family households with eight or more members. The small household voucher was a value of 805 Taka, and the large family voucher 1,170 Taka. Interviews show that neither voucher size was adequate to provide sufficient food for an entire household for a full month. This is in alignment with coping strategy findings, which show that at endline more households reported needing to borrow food, reduced adult consumption, and reduced number of meals eaten per day at endline. Furthermore, beneficiaries and staff reported that it is not possible to keep fresh foods for a full month in refugee households, often with limited storage options. It is recommended that in the future fresh food be delivered two times per month to ensure fresh food is available in sufficient quantities without spoilage. FFV implementation began soon after the program officially started, and functioned as designed until March 2020 when COVID-19 caused mobility restrictions and in some instance’s full lockdown of the camps. FFV remained an essential service for refugee households. Interviews with project staff and beneficiaries found that FFV continued throughout COVID-19 and beneficiaries still had access to fresh food via the program. The FFV became even more crucial for refugee households during the lockdown period, as FGDs reported that even those households who had previously managed to find work outside of the camp were no longer able to do so, further limiting incomes and food access. FFV was originally designed to allow recipients to choose their household food items, however, due to COVID-19 health and safety precautions, the program shifted to providing pre-packaged food for beneficiaries. This was meant to prevent over-crowding and exposure risk for vendors and beneficiaries. Interviews with staff and the PDM findings indicate that the pre-packaged food items were based on the food consumption patterns identified in previous project monitoring. Overall, the FFV modality is found to have been highly effective in ensuring refugee access to a variety of fresh foods, even during COVID-19 related restrictions. Impact on Local Markets The FFV modality is found to not have had a negative impact on local markets. KIIs with project staff and local vendors support this. The original design had been to work with vendors further outside of the 7 WV Bangladesh-72DFFP19GR00058 EFSP FY20 Q3 Report Submitted September 25 2020 8 Post Distribution Monitoring Report for Fresh Food Voucher Distribution report. World Vision Bangladesh. MEAL Team. July 2020. 24 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com community. However, local leadership pushed for more local sourcing of vendors. EFSP staff shifted to “hyper local” sourcing of vendors within host communities as close to camps as possible. Identification of vendors was done in collaboration with local CiCs, camp admin, and local leaders in alignment with World Vision requirements and guidelines. Although KIIs with staff find this provided some logistical difficulties, it resulted in improved relationships with the host communities because it gave local vendors the opportunity to generate additional income and recoup losses associated with the influx of refugee arrivals. Food item issues, such as a price increase for onions in November and December of 2019, for example, were discussed during the monthly meetings with local vendors to ensure prices remained competitive and in alignment with market monitoring reports. C. Effectiveness of Complementary Interventions Multipurpose Training Centers The original EFSP design included six Multipurpose Training Centers (MTC) (4 for women and 2 for men) across three camps, serving 4,320 beneficiaries (2,880 women and 1,440 men) between the ages of 18-30. In order to build social cohesion between Rohingya and host communities, the project was designed to include host community individuals for up to 10 percent of participants. Given the limited training options allowed by the Government, the project planned to include culturally appropriate and feasible modules such as sewing and embroidery for women’s centers, and mat and basket weaving and solar equipment repair for men’s centers. MTCs were designed to have a layered impact by being placed in areas where Cooking and Learning Centers were already established and vouchers would be distributed. KIIs with EFSP staff found that during the inception period a shift in government staff led to changes in CiC staffing, and required a longer than expected approvals process before the MTCs could be established in the pre-identified areas. MTCs were inaugurated in January 2020, and functioned as planned for two months. During that time MTC participants were identified and sensitized to the program components. MTC centers closed in March 2020 due to GoB (Government of Bangladesh) COVID-related guidelines. EFSP staff successfully adapted programming to produce cloth masks for personal protection equipment (PPE). This reclassified Summary: Effectiveness of Fresh Food Vouchers  The FFV modality is found to have been highly effective in ensuring refugee access to a variety of fresh foods, even during COVID-19 related restrictions.  FFV was a relevant intervention, as food access is a limitation within the camps for refugee households.  Overall, beneficiaries were satisfied with FFV. Many reported it was the primary or only source of fresh foods. 10% of beneficiaries reported dissatisfaction with food quality, specifically of the dried fish. Interviews show that neither voucher size was adequate to provide sufficient food for an entire household for a full month.  The FFV modality is found to not have had a negative impact on local markets. 25 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com the MTCs as a lifesaving activity, which allowed them to reopen in a limited capacity in alignment with COVID-guidelines. New procedures were established to ensure small classroom sizes, proper disinfection and handling, and integrated COVID-19 messaging. Being strategically positioned on the border of two camps as well as the host community, the 3 MTCs were able to re-engage 16 refugee and 8 host community women who were previously enrolled in sewing training to learn this skill as an IGA.9 Mask production was done in coordination with ISCG/Food Security Sector initiatives to provide free blanket distribution of masks to adults 18-59 years living in the camps.. Some FGDs with participants find that this was a well-received activity, although relatively few individuals were able to participate due to COVID-restrictions. These activities largely benefitted women, as they were directly participating in the mask making activities. KIIs and FGDs also report that some men and boys were able to participate in solar panel trainings, although this was limited. After MTCs reopened in April 2020, 6-12 participants were in a classroom at one time, rather than the 100+ originally planned. The social cohesion aspect of the MTC programming was not possible to continue after March 2020, as EFSP staff in alignment with CiC and GoB recommendations decided not to have host and refugee communities interacting with each other. Thus, MTC activities after March 2020 focused on refugee participants and only included a small proportion of host community members. Overall, MTC activities were not implemented as originally designed due to the impact of COVID-19 and related restrictions on movement and activities within the camps. This limited the intended impact. The adaptive management of program activities and rapid reprogramming allowed for established MTC locations and participants to still benefit from activities as much as possible within the context of COVID￾19. The MTC component of the program was as effective as possible within the current context. Improved Livelihoods KIIs with staff and Annual Monitoring Indicator and Targeting data shows that the project scope did not include increasing income generation for refugees, but only for the host community population. Table 5 shows the improved livelihood indicators for the refugee population at baseline and endline. Table 5: Improved Livelihood Indictors 9 WV Bangladesh-72DFFP19GR00058 EFSP FY20 Q3 Report Submitted July 30, 2020 6 INDICATOR SURVEY ROUNDS 2019 BASELINE 2020 ENDLINE SIG. Value 95% CI DEF Value 95% CI DEF 4. % of HHs who have more than two sources of income as a result of Average income sources 0.47 (0.39, 0.56) 1.25 0.86 (0.73, 0.98) 3.01 *** >2 income sources 5.6% (2.87, 8.42) 1.04 0.0% (0.0, 0.0) 26 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Refugee households reporting engaging in some kind of income generating activity increased from 0.3 percent at baseline to 2.25 percent at endline. This increase is largely seen in poultry rearing (0.3 percent baseline, 2.11 percent endline). 0.15 percent of refugees reported engaging in “other” income generating activities, which includes Papaya, Potato, and winter melon (chal-kumra). The proportion of refugees reporting they had no source of income decreased from 99.7 percent at baseline to 97.7 percent at endline. At endline, 0.0 percent of refugees reported having more than two income sources. Interviews with project staff and beneficiaries find that the primary income sources available to refugees are outside of the camps. FGDs widely reported that due to COVID-19 restrictions on movement, accessing work opportunities outside of the camp was even harder than before March 2020. FGDs reported that camps are surrounded by barbed wire fences, and the GoB did not give any permission for refugees to leave the camps. For the few who are able to obtain work, FGDs report that there is wage discrimination between locals and Rohingya. For example, a few FGDs estimated that wages for the host community members are 500-600 Taka and 300-400 Taka for the Rohingyas. Within the camps, FGDs report that the primary work opportunity is with NGOs. Even before COVID-19 job opportunities with NGOs were limited and there were more individuals available for work than jobs. After March 2020 and camp access was further restricted work opportunities within the camps. FGDs report that individuals are restricted from working even in another block within the camp. Community Cooking & Learning Centers (CCLCs) The original EFSP design planned to establish 62 Community Cooking & Learning Centers (CCLCs) in 6 camps to serve as a gathering point for 3,100 women to benefit from support such as cooking demonstrations and nutrition awareness training; training and tools to grow nutritious vegetables around their shelter; personal and family hygiene practices including hand washing demonstrations; Disaster Risk Reduction (DRR) awareness and strategies, including monsoon season risks, cyclone preparedness and fire safety; awareness sessions on safe child care practices, preventing sexual gender based violence (SGBV), and promoting gender empowerment. The community kitchens were planned to provide enrolled women with the opportunity to cook one hot meal for their family per day in a safe environment with other women. project intervention 5. % of households engaged in income generating activities disaggregated by gendered HH type and by type of activities ALL 0.3% (-0.39, 1.07) 1.06 2.25% (0.30, 4.20) 1.68 * A.Goat rearing - - - - - - B.Poultry rearing 0.3% (-0.39, 1.07) 1.06 2.11% (0.28, 3.93) 1.61 C.Dry fish production - - - - - - D.Tailoring - - - - - - E.Weaving - - - - - - F.Handicraft - - - - - - G.Others (Specify) - - - 0.15% (-0.15, 0.45) 0.99 H.None 99.7% (99.0, 100) 1.06 97.7% (95.8, 99.7) 1.68 n 331 666 27 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com KIIs with EFSP staff found that the approvals process delayed the establishment of CCLCs. Internal project reports show 55 CCLCs were established in 6 camps. 10 WV decided in consultation with FFP to cease the remaining construction of seven new planned CCLCs due to long delays in the approvals process. 11 Schools, madrasas, and learning centers were closed due to COVID-19 at the same time, which included the CCLCs. Closure restrictions began in March 2020 and were not lifted until June 2020. Interviews with project staff report that CCLCs were not able to reopen in alignment with MTCs, because MTCs were providing emergency response support (via mask production for PPE) while CCLCs were not identified as an emergency service. In order to continue support to the enrolled women, EFSP staff utilized the lead mothers who had already completed many learning sessions through the CCLCs prior to March 2020. CCLC staff mobilized 126 lead mothers over the phone, and continued sharing training messaging.12 Lead mothers were positioned to continue disseminating information to the community while staff access to the camp was limited. Lead Mothers were mobilized to provide COVID-19 awareness messaging in their immediate communities until regular groups could resume including nutritious cooking demonstrations. The lead mother network successfully disseminated information through the community on COVID-19 awareness and prevention, health and hygiene, nutrition and DRR. Lead mothers also successfully shared important information with beneficiaries about which services would still be available, including FFV. The intended programming of the CCLCs continued through the lead mothers until closures lifted in June 2020. The lead mothers helped spread important program messaging about hygiene and nutrition, however, this primarily benefited women as the lead mothers would speak directly to other women. In June CCLCs in five of six camps resumed cooking and learning sessions. After June 2020, new procedures were put in place to reduce the number of beneficiaries cooking at any given time, spacing of stoves, and women preparing food in their homes and only doing the final cooking in CCLCs to reduce their time outside their homes. 13 FGDs with beneficiaries report that access to the CCLC to cook hot meals is a highly appreciated service, and allows households to consume “more delicious” meals and allows households to consume more hot meals and reduce expenditures on cooking fuel. This service primarily benefitted women, as they prepared the meals and had the opportunity to participate in the cooking training sessions. Messaging on health and hygiene during COVID-19 is reported by EFSP staff and beneficiaries to have been crucial, particularly during the lockdown period. FGDs found that refugee households took hygiene practices very seriously as a way to prevent the spread of COVID-19. FGDs reported that WVB's NFI program provided them with 30 soaps for each household, which allowed them to follow the hygiene 10 WV Bangladesh-72DFFP19GR00058 EFSP FY20 Q3 Report Submitted July 30, 2020 11 WV Bangladesh-72DFFP19GR00058 EFSP FY20 Q3 Report Submitted July 30, 2020 12 WV Bangladesh-72DFFP19GR00058 EFSP FY20 Q3 Report Submitted July 30, 2020 13 WV Bangladesh-72DFFP19GR00058 EFSP FY20 Q3 Report Submitted July 30, 2020 28 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com guidelines. This includes washing hands with soap and water after visiting the toilet, before cooking, and when coming into the house from outside. Children are also encouraged to wash hands with soap before eating. FGDs also find that refugees have been sensitized to wear masks when leaving their homes. FGDs report that due to sensitization on the importance of child and maternal health, there is greater understanding of the unique nutrition and hygiene needs of pregnant women and children. Decision making practices for child and maternal health and nutrition is included in Table 6. Overall, the percentage of households reporting that a spouse and partner jointly make decisions for child health and nutrition increased from 33.3 percent at baseline to 60.7 percent at endline. The percent of households reporting spouses and partners make decisions jointly for pregnant and lactating women (PLW) health and nutrition also increased from 33.0 percent at baseline to 60.6 percent at endline. This is in alignment with KII and FGDs which show that through information received from EFSP, refugee households are more understanding of the benefits and importance of joint decision making. Overall, CCLC activities continued as much as possible and were as successful as possible within the current context. The training and utilization of Lead Mothers allowed for pre-positioned localized support to disseminate community messaging when WV staff were unable to access beneficiaries due to COVID-19 mobility restrictions. Summary: Effectiveness of Complimentary Interventions  MTC and CCLCs are found to be relevant complementary interventions. Programming was efficiently adapted appropriately in response to COVID-19 and maintained relevance within the unexpected context.  The establishment of MTC and CCLCs was delayed due to lengthy approvals processes, which resulted in limited time for programming as originally designed before COVID-19 related closure restrictions began.  MTCs adapted programming to provide emergency response support (via mask production for PPE).  To continue programming despite CCLC closure, the lead mother network was successfully mobilized to disseminate information through the community on COVID-19 awareness and prevention, health and hygiene, nutrition and DRR.  Interviews with MTC and CCLC participants show overall satisfaction with the activities. Interviews indicate that there is strong interest for more individuals to be able to participate in MTC and CCLC programming. Interviews indicate participants especially valued the cooking classes and the ability to cook hot meals at the center.  MTC and CCLCs are found to have been as effective as possible within the current context. 29 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Table 6: MATERNAL AND CHILD HEALTH DECISION MAKING BACKGROUND CHARACTERISTICS SURVEY ROUNDS 2019 Baseline 2020 Endline # % # % MODULE –E: HEALTH AND NUTRITION (MATERNAL AND CHILD HEALTH DECISION MAKING) Respondent: Mother of Children U5 Yourself 30.8% 13.7% % make decisions for child health and nutrition Spouse 37.0% 22.3% Yourself and spouse/partner jointly 32.2% 62.8% Yourself and others jointly 0.0% 1.2% Others (specify) 0.0% 0.0% n 180 255 Respondent: Father of Children U5 Yourself 11.4% 10.7% Spouse 33.8% 33.5% Yourself and spouse/partner jointly 54.9% 54.9% Yourself and others jointly 0.0% 1.1% Others (specify) 0.0% 0.0% n 9 93 Respondent: All (Farther, Mother) Yourself 29.9% 12.9% Spouse 36.9% 25.3% Yourself and spouse/partner jointly 33.3% 60.7% Yourself and others jointly 0.0% 1.1% Others (specify) 0.0% 0.0% n 189 348 Respondent: Mother of Children U5 Yourself 23.8% 12.2% Spouse 43.8% 23.6% % make decisions for PLW health and nutrition Yourself and spouse/partner jointly 32.5% 62.3% Yourself and others jointly 0.0% 2.0% Others (specify) 0.0% 0.0% Respondent: Father of Children U5 Yourself 33.6% 10.7% Spouse 22.9% 31.2% Yourself and spouse/partner jointly 43.6% 55.9% Yourself and others jointly 0.0% 2.3% Others (specify) 0.0% 0.0% n 9 93 Respondent: All (Farther, Mother) Yourself 24.3% 11.8% Spouse 42.8% 25.6% Yourself and spouse/partner jointly 33.0% 60.6% Yourself and others jointly 0.0% 2.1% Others (specify) 0.0% 0.0% n 189 348 30 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com D. Impact of COVID-19 Although not an explicit area of focus for this evaluation, data was collected on the impact of COVID￾19. COVID-19 and related GoB restrictions and closures had significant impacts on program activities as designed. The World Health Organization (WHO) declared COVID-19 a global pandemic on 11 March 2020. The GoB declared national lockdown through mid-April. EFSP staff adapted and established new standard operating procedures (SOPs) for every program integrating updated GoB and Sectoral guidance, and multilateral advocacy to support the rapid return of EFSP program. 98.3 percent of households responded they are aware of Corona Virus (COVID-19), and 32.8 percent of households said their household was directly impacted by Corona Virus in the last 12 months. The primary impacts of COVID￾19 on households was reduced income (66.0 percent), reduced social interactions with relatives and friends (62.3 percent) and reduced ability to buy items to meet households’ basic needs (46.8 percent). Less than half of households reported reduced mobility (39.6 percent), reduced support from NGOs or GoB (37.4 percent) or loss of labor (36.5 percent). Figure 7: Impacts of COVID-19 on Households in the Past 12 Months In the months following COVID-19, 83.8 percent of households reported a slight decrease on household income, and 9.7 percent reported a severe decrease. 64.1 percent reported household food consumption decreased slightly, and 7.8 percent reported a severe decrease in food consumption in the 0.5% 36.5% 66.0% 37.4% 39.6% 0.0% 62.3% 0.5% 0.0% 0.0% 46.8% 1.9% 0.5% 0.5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% A.Loss of life B.Loss of labor C.Reduced income D.Reduced supports from Govt. or NGOs E.Reduced mobility F.Loss of livestock G.Reduced social interactions with relatives and friends H.Loss of HH assets (including distress sales) I.HH member migrated J.HH was displaced K.Reduced ability to buy basic needs (e.g. food, cloth –… L.Huge medical cost for treatment of family member M.Other N.Don’t know Household Impacts of COVID-19 in the past 12 months 31 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com months following COVID-19. 14 The primary coping strategy households reported to cope with the impacts of COVID-19 is to receive food aid or assistance from an NGO (including food/cash-for-work) (77.7 percent). This is followed by taking children out of school (20.2 percent) and reducing household food consumption (19.3 percent). This is in alignment with FGD findings, which show that household’s impact of COVID-19 was not as severe as it would have been had they not had access to FFV. E. Strengths, Challenges and Lessons Learned The following section outlines the strengths and best practices, as well as challenges faced by EFSP. Strengths and Best Practices A. Adaptative Management and COVID Response In late March 2020, EFSP staff implemented team-by-team risk assessments and adapted programming to be implemented remotely as much as possible. Activities that required in-person training or large groups were modified to align with COVID-19 guidelines. New SOPs were developed, allowing for continuation of the majority of activities (after being adapted) throughout the lockdown period and ongoing COVID-related movement restrictions. This efficient adaptive management was a significant strength of the program. Internal reports and KIIs with key staff find that EFSP teams collaborated effectively with other Livelihoods Working Group partners, which were then able to adapt the newly established SOPs for their own programming. This positioned WV and EFSP as a leader in the sector response to COVID-19, as well as more broadly in terms of livelihood strategy and implementation models. B. Relationships with the CiCs EFSP prioritized relationship with CiCs as the focal point of communication and camp authority. All programming was completed in coordination with the CiC. This was a strength of the program and recommended best practice. Close positive relationships with the CiC benefitted implementation as it helped activities complete the approvals process and allowed for efficient reprogramming of activities to accommodate COVID-19 guidelines and restrictions. Strong relationships and timely coordination with government stakeholders, CiC authorities and other actors in the food security sector is instrumental to get approvals, ensure efficient timing of activities and problem solve during the implementation. C. Sourcing of vendors and market analysis The supply chain team originally anticipated sourcing vendors further out from the community, however, after feedback from local leadership, EFSP shifted to hyper local resourcing of vendors. 14 Survey questions asked “In the months following Corona Virus/COVID-19, how severe was the impact on household INCOME?” and “In the months following Corona Virus/COVID-19, how severe was the impact on household FOOD CONSUMPTION?” to measure the impact of COVID-19 and related restrictions on households in the camps. 32 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com This led to positive relationship building with the host community. EFSP help vendor meetings once per month which allowed for regular feedback to vendors and rapid improvements as needed. Market analysis were conducted every three months throughout implementation, monitoring any impacts of FFV on local markets and ensuring standardized fair prices for all vendors. This level of price monitoring was particularly crucial during COVID-19, as prices fluctuated significantly during the lockdown period. This is a recommended best practice. D. Close coordination with Donor Interviews with key staff show that throughout programming, strong advocacy and support at higher levels was a key factor in ensuring the rapid adaptation to the shifting context of COVID-19 was possible. Program staff were able to openly communicate current status and ideas for reprogramming directly with the donors, as well as within sectoral and government levels. This allowed for donor understanding of field realities, and increased flexibility in adapting from initial proposed programming design. This is a suggested best practice moving forward. E. Early onboarding of key staff Internal documentation and interviews with key staff show that although not all staff were present in country at the start of programming (COP arrived February 2020, MEL lead in December 2019), weekly meetings with proposed key staff and department heads began with the formal program start in August 2019. This allowed for early onboarding of key staff, and establishment of focal points for program components. Similarly, the head of the Food Security Sector arrived in October 2019, and staff completed effective orientation meetings early on and established a supportive relationship which allowed for positive collaboration at the sector level. Challenges The major external factors (apart from COVID-19 as discussed in section 3D of this report) that influenced or impacted on the implementation of the camps interventions was primarily the camp and GoB approvals process, followed by coordination with the sector, timing and targeting of evouchers, vendor sourcing, and staffing. A. Camp and GoB Approvals Process A major challenge since baseline was the approvals process. Approvals for activity implementation were required every six months, which became a key bottleneck for implementation. The GoB has specific requirements and restrictions on activities that can be implemented within camps. Particularly after a re-staffing at the local government levels early into implementation, the approvals process was significantly delayed, causing the construction and inauguration of CCLCs to not be completed until January 2020. Although this challenge was somewhat mitigated by positive relationships and collaboration with CiCs, the program ultimately did not have control over the approvals process and had to adjust program implementation timing accordingly. 33 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com B. Coordination with Sector The lack of harmonization between sector actors was a challenge. Although the partners collaborated to identify catchment areas, these shifted after WFP rolled out the e-voucher first. This led to EFSP needing to reregister and sensitize recipients in new catchment areas. In January 2020, the partners discussed blanket distribution versus targeted distribution. EFSP programming remained a blanket distribution model, although this was not consistent across the sector. Due to a lack of a standardized/harmonized targeting approach in the Food Security Sector, different organizations applied different voucher values for entitlement. Some partners provided a higher rate than EFSP, which required additional sensitization of new EFSP beneficiaries to the FFV benefits. Program staff collaborated with camp admins to ensure this process went as smoothly as possible. It is recommended that further harmonization of targeting and voucher benefits take place across the sector. C. Timing and Targeting of Evouchers The original program design included an outline of a rough catchment area with intention to provide the same intervention over 10 months, however, this wasn’t the case for evouchers. As evouchers rolled out FFV rolled in to ensure continuous coverage, which made the implementation schedule of FFV dependent on the evoucher schedule. E-vouchers were only rolled out for a few months before moving on to the next groups, which presented coordination challenges. Although EFSP still achieved 100% of the FFV coverage target, staff spent significant time readjusting the catchment area, re￾registering beneficiaries, orienting staff, etc. Interviews with staff find that the evoucher schedule was not consistent, and block by block level coordination was required between WFP, the sector, camp and local leaders within communities. World Vision successfully adapted and accommodated changes in the evoucher schedule to ensure continuity with FFV and prevent gaps in coverage. D. Vendor Sourcing Although the successful sourcing of hyper local vendors is a strength of the program, this did present several challenges. First, the camps included in the catchment are geographically spread out, which meant once work closed out in one camp and shifted implementation to another, it was not possible to use the same vendor. The team ensured enough vendors were sourced to ensure coverage across different camp vendors. Staff also had to ensure the identified vendors understood that the length of time they would work with the project would be variable. This also presented logistical challenges, as staff had to coordinate with 30+ individual vendors. E. Staffing Some staff positions were not designed to be 100% effort on EFSP only. Finance support and supply chain support were shared roles; for example, managing a 50/50 or 25/75 role. This presented difficulty for EFSP staff to get the level of effort required when needed, causing some internal delays. 34 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com Lessons Learned One of the key activities included in the program design was a lessons learned workshop. This workshop was planned to be held upon completion of camps interventions in order to identify best practices, challenges and draw lessons and recommendations for operation team for future similar interventions. Although it was not possible to convene all relevant stakeholders in the same workshop due to COVID￾19 precautions, the EFSP MEAL team conducted learning sessions with key project stakeholders and beneficiaries in July and August 2020, and a separate workshop with EFSP and WV staff and donors in August 2020. Key learnings were aggregated into a Camps Lessons Learned Report in September 2020.15 The lessons learned highlighted in that report are triangulated with KIIs and FGDs completed through this endline evaluation. Lessons learned are organized by program modality, below. Fresh Food Voucher Modality • Strong coordination and active participation of WV in the Food Security Sector coordination meetings is important for decision making in the cluster and support successful implementation. Establishing and maintaining strong relationships and timely coordination with government stakeholders, CiC authorities and other actors in the Food Security Sector is instrumental to get approvals and start activities on time and solve the issues encountered during the implementation. • Timely and consistent engagement with vendors to negotiate and change food commodities prices in alignment with identified market trends is crucial to maximize the beneficiaries’ food basket as well as ensure fairness to vendors so they do not lose potential profit due to price fluctuations. Stipulations and protocols for measures to address price fluctuation should be specifically outlined in the agreement with vendors. The example of regular monthly meetings to follow up meetings with vendors and suppliers should be undertaken in future programming to ensure the quality of supplies/food/services provided. • The FFV value entitlements based on family size should be in alignment with general food distribution criteria (4 categories of households as per family size) to meet adequately households needs according to their size. Voucher entitlement for beneficiaries based on two categories of family size (1-7 members and 8+) was not adequate to the different needs as per family size, and was insufficient to provide households with enough food for a full month. Multipurpose Training Centers • Interest in MTC activities is very high. There is interest for inclusion of both men and women in training, and expand opportunities for training in cooking and handicrafts. Given the high interest and high needs consider designing of interventions targeting a higher number of beneficiaries. 15 WV Bangladesh, EFSP Cox’s Bazar, Award #72DFFP19GR00058, Camp Lessons Learned Report. September 2020. 35 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com • Coordination with other WV teams (WASH, CP etc.) is essential for maximizing the benefits for refugees’ beneficiaries. This allowed for more diverse messaging and individuals with expertise to participate in training sessions. • Upon completion of vocation training, it would be very useful to provide the participants with sewing machine or access to sewing facilities to start up their activity and ensure the training is able to be implemented after the cohort period. • Stronger coordination with community leaders (beyond CiC) would benefit the mobilization of beneficiaries and addressing of issues. Community Cooking and Learning Centers • Accommodations are needed to ensure women’s participation in activities is possible. For example, the cooking sessions schedule/time should be set to fit women’s daily schedule and childcare options should be available. Similarly, in order to ensure meaningful participation is possible in the conservative context/gender norms within the camp populations, it should be a priority to hire female facilitators to engage in activities focused on women. • The approvals process can significantly delay activities. WV government liaison departments role in strengthening the relationships and advocacy with government offices is crucial for effective implementation and getting approvals on time for CCLC activities. Also, coordination with camp site management authorities and Majhi has to be strengthened to ensure smooth implementation in the future. Coordination with Government and Camp Leadership • Good relationships with CiC and other actors help for successful implementation particularly for FFV. Monthly coordination meetings and one to one meeting with CiC reps are very helpful for ensuring timely implementation of the project activities and addressing issues in timely manner. CiC visits in the camps to observe interventions are good for the visibility of the project and accountability. Informing the CiC authorities prior to final selection of vendors or service providers in camps is important to get their buy in and support. • Majhis are instrumental to a successful implementation of activities in the camps and they have to be invited and attend regularly the coordination meetings. 4.Conclusions Despite the unanticipated impacts of COVID-19 and the related government lockdown and restrictions which impacted program implementation, EFSP successfully practiced adaptive management to meet evolving needs and opportunities, and effectively achieved the intended program objectives. Overall, EFSP met or exceeded all but one of the topline project indicators based on point estimates. Out of the 8 total topline indicators, 4 met end of project targets, 3 exceeded the end of project targets, and one did not meet the end of project target. 36 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com The FFV modality is found to have been highly effective, and even within the context of variable food prices and restricted movement due to COVID-19, refugee beneficiaries food security improved since baseline. All three indicators (Food Consumption Score, Household Hunger Scale, and Household Dietary Diversity Score) used to measure food security indicate a positive direction of change indicating households are eating a wider diversity of foods and are more food secure than at baseline. Voucher entitlement for beneficiaries were based on two family size categories (1-7 members and 8+). The vouchers did not provide an adequate amount of food to meet the needs of families with eight or more members. Refugee households also have limited storage options, and it is recommended that in the future fresh food be delivered two times per month to ensure fresh food is available in sufficient quantities without spoilage. Almost all respondents reported awareness of the Corona Virus/COVID-19, and nearly 40 percent reported being directly impacted by COVID-19. The primary coping strategy over 75 percent of households reported to cope with the impacts of COVID-19 is to receive food aid or assistance from an NGO (including food/cash-for-work). This is followed by taking children out of school and reducing household food consumption. Interviews with beneficiaries found that households feel the impacts of COVID-19 was not as severe as it would have been had they not had access to FFVA. Overall, the rCSI score increased from baseline to endline, indicating that more severe coping strategies are being used at endline as compared to baseline. This is in alignment with expected impacts on households in the camp experiencing the impacts of COVID-19 and related restrictions. At endline, more households reported needing to borrow food or rely on help for food access, reduced adult consumption, and reduced number of meals eaten per day. The effectiveness of the complementary interventions, MTCs and CCLCs was made possible due to the efficient adaptive management of EFSP staff and donor support. Programming shifted to a remote model as much as possible. EFSP staff successfully adapted MTC programming to produce cloth masks for PPE. This reclassified the MTCs as a lifesaving activity, which allowed them to reopen in a limited capacity in alignment with COVID-guidelines. Lead mothers were utilized as a focal point for program information dissemination, which successfully allowed for CCLC program messaging to still be communicated through communities from lead mothers through their neighbors and social networks. 55 out of 62 planned CCLCs were established. CCLCs were adapted to include smaller class sizes which allowed for women to still utilize the kitchens in a limited capacity. 37 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com 5. Appendix 5.1 Sample Size Calculation 5.2 Sample Weights for Strata and Clusters Ward/ Sub-Block (Cluster) Sample size Response Non-response Non-response weight Cluster Code 1E Block-A SubBlock-A14 20 20 0 1.00000 1 1E Block-B SubBlock-B9 20 20 0 1.00000 2 1E Block-D SubBlock-D6 20 19 1 1.05263 3 1E Block-F SubBlock-F3 20 20 0 1.00000 4 1W Block-A SubBlock-A14 20 18 2 1.11111 5 1W Block-C SubBlock-C10 20 20 0 1.00000 6 1W Block-D SubBlock-D8 20 19 1 1.05263 7 1W Block-F SubBlock-F8 20 20 0 1.00000 8 3 Block-A SubBlock-AA15 20 20 0 1.00000 9 3 Block-B SubBlock-AA9 20 19 1 1.05263 10 3 Block-E SubBlock-DD12 20 19 1 1.05263 11 3 Block-F SubBlock-DD16 20 20 0 1.00000 12 4 Block-C SubBlock-OO24 20 18 2 1.11111 13 4 Block-E SubBlock-PP15 20 20 0 1.00000 14 4 Block-G SubBlock-G6 20 18 2 1.11111 15 5 Block-A SubBlock-G4- Hamid Ullah 20 20 0 1.00000 16 5 Block-C SubBlock-E5 20 20 0 1.00000 17 5 Block-E SubBlock-G2- Rosi Alam 20 20 0 1.00000 18 8W Block-B SubBlock-I12 20 20 0 1.00000 19 8W Block-D SubBlock-I20 20 20 0 1.00000 20 14 Block-A SubBlock-A4 20 20 0 1.00000 21 14 Block-C SubBlock-C2 20 17 3 1.17647 22 14 Block-E SubBlock-E2 20 20 0 1.00000 23 15 Block-B-Sub-Block-B5 20 20 0 1.00000 24 15 Block-D-Sub-Block-D2 20 20 0 1.00000 25 15 Block-E SubBlock-E8 20 20 0 1.00000 26 15 Block-G-Sub-Block-G12 20 20 0 1.00000 27 15 Block-H-Sub-Block-H6 20 20 0 1.00000 28 16 Block-B SubBlock-B1 20 20 0 1.00000 29 16 Block-D SubBlock-D3 20 20 0 1.00000 30 20 Block-M Sub-block M-31 20 20 0 1.00000 31 21 Block-E SubBlock-E2 20 20 0 1.00000 32 22 Block-A SubBlock-A3 20 20 0 1.00000 33 22 Block-C SubBlock-C2 20 19 1 1.05263 34 680 666 14 1.021021021 38 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com 5.3 Team Training Agenda Training Agenda: EFSP ENDLINE October 19 – 22nd 2020 Note: Red text indicates the session will include a PowerPoint presentation MKE Timing Toronto Timing Auckland Timing BST Timing Duration Topic Facilitation Lead Zoom Link Day 1 – Monday October 19th 10 pm 10:30 pm 4 pm 4:30 pm 9:00 am 9:30 am 30 min 30 mins Welcome and Introductions Review of COVID-19 Safety Protocols for during training Bruce World Vision TANGO Zoom D [Zoom D - https://us02web.zoom.us/j/8827345847] Password: tango 11 pm 11:30 pm 5 pm 5:30 pm 10:00 am 10:30 am 30 mins 30 mins Overview of EFSP Project Purpose of the study and sampling methodology World Vision Bruce 12 – 12:15 am 6 – 6:15 pm 11 – 11:15 am 15 mins Tea Break 12:15 am 6:15 pm 11:15 am 1 hr Review qualitative and quantitative hard copy questionnaire flow Nasir 1:15 am 7:15 pm 12:15 pm 1 hr Assignment of tablets, tablet orientation and survey download Nasir 2:15 am 8:15– 9:00pm 1:15 – 2:00 pm 45 min Orientation on ODK, Tablet settings Nasir 3:00 – 4:00 am 4:00 – 5:00 am 9:00 – 10:00 pm 2:00 – 3:00 pm 1 hr Lunch 5:00 – 8:00 am 3:00 – 6:00 pm 3 hr Review of survey in ODK using Tablets Towfique 8 – 8:30 am 6:00 – 6:30 pm 30 min Review of day-1 and instruction for Day-2 Homework: read through hard copy of questionnaire Day 2 – Tuesday October 20th 9p 3p 8:00 am – 9:00am 1 hr Overview of Qualitative Data Collection Methods, Intro to the Topical Outline Bruce and Chloe TANGO Zoom D [Zoom D - https://us02web.zoom.us/j/8827345847] Password: tango 10 pm 4 pm 9:00 am 30 mins Recap of day-1 Bruce 10:30 pm 4:30 pm 9:30 am 1 hr Interview protocol and techniques, research ethics Nasir 39 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com MKE Timing Toronto Timing Auckland Timing BST Timing Duration Topic Facilitation Lead Zoom Link 11:30pm 11:30 – 11:45p 5:30 pm 6:30 – 6:45 pm 10:30am 11:30 – 11:45am 1 hr 15 min Review and Q&A on tablet set up, in-field tablet usage protocols Tea Break Nasir 11:45 – 3:00 a 6:45 – 9:00 pm 11:45 – 2:00pm 2.5 hr ODK questionnaire review (role play practice activity) using tablet Nasir 3:00 – 4:00 am 4:00 – 5:00 am 9:00 – 10:00 pm 2:00 – 3:00 pm 1 hr Lunch 5 – 5:30 am 3 – 3:30pm 30 min Answer questions from morning session Towfique 5:30 – 8 am 3:30 – 6:00p 2.5 hr ODK questionnaire review in mobile phone/tablet 8 – 8:30 am 6 – 6:30 pm 30 min Review of day-2 and instruction for Day-3 Day 3 – Wednesday October 21st 9p 3p 8:00 am 1 hr Qualitative Tool Review Bruce and Chloe TANGO Zoom D [Zoom D - https://us02web.zoom.us/j/8827345847] Password: tango 10 pm 4 pm 9:00 am 30 min Recap of day-2 Bruce 10:30 – 3:00 a 4:30 – 9:00 pm 9:30 – 2:00 pm 4.5 hr ODK questionnaire review (role play practice activity) using tablet Nasir 3:00 – 4:00 am 4:00 – 5:00 am 9:00 – 10:00 pm 2:00 – 3:00 pm 1 hr Lunch 5 – 5:30 am 3 – 3:30pm 30 min Answer questions Towfique 5:30 – 8 am 3:30 – 6:00p 2.5 hr ODK questionnaire review in mobile phone/tablet 8 – 8:30 am 6 – 6:30 pm 30 min Review of day-3 and instruction for Day-4 Day 4 – Thursday October 22nd 9p 10p 3p 8:00 am 1 hr Qualitative Tool Review and Final Updates Bruce and Chloe TANGO Zoom D [Zoom D - https://us02web.zoom.us/j/8827345847] Password: tango 10 pm 11 pm 4 pm 9:00 am 30 min Recap of day 3 Bruce 10:30 pm 4:30 pm 9:30 am 1 hr Supervisors Meeting: Data Management and Quality Control Bruce 11: 30 – 3:00a 5:30 – 9pm 10:30 – 2:00 pm 3.5 hr Field Test Nasir 3:00 – 4:00 am 4:00 – 5:00 am 9:00 – 10:00 pm 2:00 – 3:00 pm 1 hr Lunch 5 – 7 am 10 pm – 12am 3:00 – 5:00 pm 2 hr Debrief field test Towfique 7 – 8 am 12 – 1 am 5 – 6pm 1 hr Final Q&A 8 – 8:30 am 1 – 1:30 am 6 – 6:30pm 30 min Closing 40 TANGO: EFSP Endline Evaluation Camp Activities Aug 2019-Aug 2020 tangointernational.com 6.Annex Annex 1 Quantitative Survey Tool Annex 2 Qualitative Survey Tool Annex 3 Qualitative Data Entry Matrices Annex 4 Inception Report Annex 5 Quantitative Data Annex 6 Detailed EFSP Topline Indicator Analysis Tables Annex 7 Qualitative Data Annex 8 EFSP Monitoring and Evaluation Framework