Baseline Study of the Ifaa Resilience Food Security Activity in Ethiopia March 2023 | Volume II – Annexes A, B, & C IMPEL | Implementer-Led Evaluation & Learning Associate Award ABOUT IMPEL The Implementer-Led Evaluation & Learning (IMPEL) Associate Award works to improve the design and implementation of Bureau for Humanitarian Assistance-(BHA)-funded resilience food security activities (RFSAs) through implementer-led evaluations and knowledge sharing. Funded by the United States Agency for International Development (USAID) BHA, IMPEL will gather information and knowledge in order to measure performance of RFSAs, strengthen accountability, and improve guidance and policy. This information will help the food security community of practice and USAID to design projects and modify existing projects to bolster performance, efficiency, and effectiveness. IMPEL is a seven-year activity (2019—2026) implemented by Save the Children (lead), TANGO International, Tulane University, Causal Design, Innovations for Poverty Action, and International Food Policy Research Institute (IFPRI). RECOMMENDED CITATION IMPEL. (2023). Baseline Study of the Ifaa Resilience Food Security Activity in Ethiopia (Vol. II). Washington, DC: The Implementer-Led Evaluation & Learning Associate Award. PHOTO CREDITS Melikte Tadesse / Catholic Relief Services Ethiopia DISCLAIMER This report is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the Implementer-Led Evaluation & Learning (IMPEL) award and do not necessarily reflect the views of USAID or the United States Government. CONTACT INFORMATION IMPEL Associate Award c/o Save the Children 899 North Capitol Street NE, Suite #900 Washington, DC 20002 www.fsnnetwork.org/IMPEL IMPEL@fsnnetwork.org PREPARED BY: Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Acknowledgments i ACKNOWLEDGMENTS The Causal Design research team would like to recognize several individuals who were instrumental in creating and refining this document. This includes Shane Lennon, Gregory Makibila with Ifaa, and Mohamoud Ahmed, Corena Sharp, and Tom Spangler with IMPEL. Special thanks also to Mara Mordini, Takawira Kapikinyu, Hilary Cook, Benita O’Colmain, Arif Rashid, and Adam Trowbridge with USAID for their ongoing technical support. IMPEL | Implementer-Led Evaluation and Learning ii Table of Contents TABLE OF CONTENTS Acknowledgments...............................................................................................i List of Tables......................................................................................................iii List of Figures.....................................................................................................iii Acronyms...........................................................................................................iv Annex A: Pre-Analysis Plan .................................................................................1 1. Description of Study.......................................................................................................................1 1.1. Impact Evaluation Overview...............................................................................................1 2. Evaluation Approach ......................................................................................................................2 2.1. Research Objective .............................................................................................................2 2.2. Evaluation Design................................................................................................................2 2.3. Power Calculations and Sampling Strategy ........................................................................7 2.4. Cost-Effective Analysis......................................................................................................12 3. Data Collection.............................................................................................................................14 3.1. Survey Design....................................................................................................................14 3.2. Outcome Indicators ..........................................................................................................15 3.3. Enumerator Training.........................................................................................................15 3.4. Data Management............................................................................................................15 3.5. Tentative data collection timeline....................................................................................16 4. Analysis.........................................................................................................................................16 4.1. Impact Analysis.................................................................................................................17 4.2. Supplementary Analysis on Resilience Indices.................................................................19 4.3. Cost-Effectiveness Analysis...............................................................................................19 5. Appendix ......................................................................................................................................21 5.1. Data Management............................................................................................................21 5.2. Data Quality ......................................................................................................................21 5.3. Data Security & Research Ethics.......................................................................................23 5.4. Knowledge Management & Transparency .......................................................................24 5.5. Data Storing/Sharing guidelines .......................................................................................24 5.6. Intervention Packages.......................................................................................................28 5.7. Indicators List....................................................................................................................33 Annex B: List of Interventions...........................................................................35 Annex C: List of Kebeles with Treatment Assignment and Livelihood Status.....63 VOLUME I Baseline Study of the Ifaa Resilience Food Security Activity in Ethiopia Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) List of Tables and Figures iii LIST OF TABLES Table 1. List of interventions across the two evaluation packages..............................................................4 Table 2. Descriptive statistics of selected outcome indicators.....................................................................7 Table 3. Minimum Detectable Effect (MDE) Sizes for Key Food Security Outcomes in Amhara and Oromia Assuming a cluster size of 50 for the control group and 70 for the treatment group (35 households per cluster) ..........................................................................................................................................................8 Table 4. Minimum Detectable Effect (MDE) Sizes for Key Food Security Outcomes in Amhara and Oromia Assuming a cluster size of 25 for the control group and 35 for the treatment group (35 households per cluster) ..........................................................................................................................................................9 Table 5. Sampling Strategy..........................................................................................................................11 Table 6. Baseline Data Collection timeline (preliminary)............................................................................16 Table 7. Endline Data Collection timeline (preliminary).............................................................................16 Table 8. Data Quality Assurance Activities .................................................................................................26 Table 9. Intervention Packages...................................................................................................................28 Table 10. Ifaa Indicators List.......................................................................................................................33 Table 11. Intervention Packages.................................................................................................................35 Table 12. Impact evaluation kebeles with treatment and livelihood status ..............................................63 LIST OF FIGURES Figure 1. MDE for the indicator Household is moderately or severely food insecure based on FIES score (different cluster sizes, 50 control clusters and 70 treatment clusters).....................................................10 IMPEL | Implementer-Led Evaluation and Learning iv Acronyms ACRONYMS ANC Antenatal Care BHA Bureau for Humanitarian Assistance CAHW Community Animal Health Workers CCFLS Community-Based Complementary Feeding and Learning Sessions CGM Care Group Model CLTSH Community Led Total Sanitation and Hygiene CMAM Community-based management of Acute Malnutrition CRS Catholic Relief Services CTCF Community Technical Coordinating Forum DFSA1 Development Food Security Activity 1 DS Direct Support ENA/EHA Essential Nutrition and Hygiene Actions ESMF Environment and Social Management Framework FCS Food Consumption Score FDP Food Distribution Points FF Follower Farmer FIES Food Insecurity Experience Scale FMNR Farmer Managed Natural Regeneration FSTF Food security task force GBV Gender-Based Violence GoE Government of Ethiopia HEP Health Extension Programme HEW Health Extension Workers HH Household Head IE Impact Evaluation IGA Income Generation Activities IMPEL Implementer-Led Evaluation and Learning Associate Award IMNCI Integrated Management of Newborn & Childhood Illnesses IWM+ Integrated Watershed Management+ IWRM Integrated Water Resources Management IYCF Infant and young child feeding LEW Livelihood Extension Workers LF Lead Farmer LH Lead Herders MAD Minimum Acceptable Diet Baseline Study of the Ifaa RFSA in Ethiopia (Vol. I) List of Tables and Figures v MDD Minimum Dietary Diversity MFI Micro-Finance Institution NGO Non-Governmental Organization NRM Natural Resource Management ODF Open Defecation Free ORT Oral Rehydration Therapy PAP Pre-Analysis Plan PDP Primary Distribution Points PDS Permanent Direct Support PIM Program Implementation Manual PLW Pregnant and Lactating Women PSNP Productive Safety Net Programme PSNP5 Productive Safety Net Programme Phase 5 PSP Private Service Providers PW Public Work RCT Randomized Controlled Trial RFSA Resilience Food Security Activity RL Religious Leader RuSACCO Rural Savings and Credit Cooperatives SBC Social and Behavioral Change SILC Savings and Internal Lending Communities SPIR Strengthen PSNP Institutions and Resilience TDS Temporary Direct Support USAID United States Agency for International Development USD United States Dollar WASH Water, Sanitation, and Hygiene WASHCO WASH Community Organization WFSTF / KFSTF Woreda / Kebele Food Security Task Forces YES Youth Employability Skills YPA Youth Peace Ambassadors This page is left intentionally blank. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 1 ANNEX A: PRE-ANALYSIS PLAN 1. DESCRIPTION OF STUDY Causal Design is part of the Implementer-Led Evaluation and Learning (IMPEL) Associate Award consortium, the Bureau for Humanitarian Assistance’s (BHA) primary mechanism for carrying out the evaluations of the resilience food security activities (RFSA) in Ethiopia. Causal Design’s support for this project will include a survey and evaluation design of the Ifaa RFSA implemented by Catholic Relief Services (CRS). Throughout the implementation period, Causal Design will conduct an Impact Evaluation study using an experimental evaluation (randomized control trial (RCT)) approach comprised of: • Evaluability Assessment that will lay out the research design and approach to inform the Pre￾Analysis Plan. • A Pre-Analysis Plan that will outline the sampling strategy, survey design, outcomes for the analysis, estimation strategy, and additional methods (as appropriate). • A Baseline Report that will summarize and analyze baseline (BL) survey data. • An Impact Evaluation Report that will use the BL and endline (EL) data to estimate the impact of the RFSA. • A Cost-Effectiveness Analysis (CEA) at EL, tied to the impact evaluation data. In addition to these activities, Causal Design will also perform a process monitoring evaluation, which is meant to assess the implementation process itself. This evaluation is not part of the experimental evaluation and will not be discussed in this Pre-Analysis plan (PAP). A separate PAP plan for the process monitoring will be delivered at the end of the year. This PAP outlines the experimental approach for evaluating the effectiveness of the Ifaa RFSA program. It outlines the overall evaluation design and approach, data collection and management protocols, and analysis methods for each evaluation component. 1.1. Impact Evaluation Overview Causal Design will conduct an Impact Evaluation (IE) using BL and EL survey data in the target areas of the Ifaa RFSA in Oromia, Ethiopia. A BL survey for the RFSA will be carried out in the second quarter of 2022, and a corresponding EL survey will be conducted in the second quarter of 2025. Additionally, a cost-effectiveness analysis, designed below, will be carried out in conjunction with the EL report. Overview of BL Study: The BL study will rely on quantitative methods to measure BHA standard indicators collected in the RFSA experimental evaluation area in 2022. The survey will provide BL data on the status of communities and households that are part of the experimental evaluation. 1 Causal Design will work closely with BHA and relevant stakeholders to identify other key learning objectives and ensure that the BL survey and study are able to contribute to this learning where possible. 1 The results should not be extrapolated and interpreted as representative of subgroups that are not a part of the research. For example, the results shouldn’t be extrapolated to a whole kebele, because the study is only looking at PSNP households with women/girl of reproductive age in the kebeles part of the study. IMPEL | Implementer-Led Evaluation and Learning 2 Annex A: Pre-Analysis Plan Overview of EL Study: The EL study will collect survey data from the same communities and households in the BL survey to estimate the ability of the RFSA intervention to directly impact household food security and well-being indicators as listed in BHA’s standard indicators. The EL study will be carried out in 2025 during the same months as the BL survey (second quarter) and will allow suitable time for any potential benefits to occur. The same, or slightly modified, survey will be administered to the same households as in the BL activity to ensure comparability across the two time periods. Overview of Cost-Effectiveness Analysis: The CEA will focus on the cost-effectiveness of the entire RFSA implementation over the course of its entire implementation. It will not examine the relative cost￾effectiveness of different packages or the PSNP Basic package of interventions since CRS does not track their expenses by intervention. This CEA will rely on performance monitoring indicators, rather than impact data, for measures of effectiveness. The results of the CEAs will be presented in the EL report. 2. EVALUATION APPROACH 2.1. Research Objective The Ifaa RFSA activities and services are a package of interventions aimed at improving food security of vulnerable households in targeted Productive Safety Net Programme (PSNP) communities, contributing to a sustained reduction in rural poverty. Ifaa will provide one of two packages of interventions—Basic PSNP and Ifaa Enhanced—to 241 kebeles in the Region of Oromia. We will refer to these two packages of interventions as the evaluation packages. The Basic PSNP package is a set of interventions selected and implemented by the Government of Ethiopia to support its most vulnerable populations. The Ifaa Enhanced package implements those "basic" interventions plus additional interventions. The primary objective of the impact evaluation will be to measure the impact of the Ifaa Enhanced package for PSNP5 participants on food security and related outcomes in the targeted communities and determine possible attribution to changes in key indicators. The evaluation seeks to inform the larger knowledge base around the efficacy of the RFSA among vulnerable populations and how benefits to vulnerable households can be further maximized. 2.1.1. Research Question Research Question: What is the impact of the Ifaa Enhanced package of interventions for PSNP5 households compared to the PSNP Basic package on reducing food insecurity, nutrition, and other related outcomes? 2.2. Evaluation Design The evaluation team will implement a cluster RCT, designed to estimate the impact of the Ifaa Enhanced package of interventions. The evaluation will use a randomized controlled trial cluster design (cluster RCT) which randomizes the selection of kebeles that receive the Ifaa Enhanced package and the kebeles that receive the PSNP basic package in the Oromia region of Ethiopia. The group of kebeles receiving the Ifaa Enhanced package will be referred to as the treatment group, while the set of kebeles receiving the PSNP Basic package will be referred to as the control group. The two evaluation packages were proposed by the implementing partner (IP). The proposed research question allows us to evaluate the Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 3 impact of the Ifaa Enhanced package compared to PSNP Basic. This is an important research result, since CRS considers those interventions to be the ones leading to significant improvements for the PSNP clients. BL and EL data will be collected, and statistical analysis will be used to estimate the direct impact of receiving the Ifaa Enhanced package compared to the PSNP Basic package, using BHA food security and nutrition indicators2,3 and other focal indicators such as Resilience and Poverty. The RCT design will maximize the ability of the research to measure direct and attributional impacts and will employ statistical tools and methodologies for estimating impacts. The CEA, combined with the impact evaluation findings, will allow the research team to explore a value-for-money dimension that assesses program effectiveness. The following sections outline the specific interventions, identification, randomization, and sampling strategies for the impact evaluation as well as the CEA methods and strategy. 2.2.1. Ifaa Interventions Per the goals and objectives of reducing food insecurity and promoting well-being and welfare among participant kebeles, the Ifaa RFSA aims to provide a range of support interventions to more than 60,000 households. These interventions are aimed at strengthening and improving government services, agriculture and livelihood opportunities, health and nutrition, WASH, gender and youth empowerment, and natural resource management and environment. Table 1 shows a subset of the interventions and how they are mapped to the two evaluation packages mentioned before—Basic PSNP and Ifaa Enhanced.4 The Ifaa Enhanced package is the most comprehensive package and contains all the interventions in the PSNP Basic package, plus additional interventions under each domain. Each kebele that is part of the evaluation study will receive the interventions associated with only one of the two evaluation packages. The interventions under Livelihoods are special in the sense that only a subset of the kebeles are eligible5 to receive those interventions. To be able to account for this, the IE study will include two groups of kebeles: (i) those eligible to receive livelihood interventions, and (ii) those that are not eligible. Within each group, kebeles will be randomly assigned to control or treatment (resulting in four groups of kebeles). Livelihood kebeles assigned to the control group will receive all the Basic PSNP interventions (including the livelihood ones), while livelihood kebeles in the treatment group will receive the Ifaa Enhanced package. In the case of the group of kebeles not eligible to receive livelihood interventions, control kebeles will receive the Basic PSNP interventions but without livelihood interventions, while the treatment kebeles will receive all the Ifaa Enhanced package but without livelihood interventions. 2 https://www.usaid.gov/food-assistance/partner-with-us/implementation-and-reporting 3 We use a set of 5 indicators to power the study. In section 2.3we present the selected indicators and discuss the reasons for that selection. 4 Table 8 in section 5.6 contains the list of all interventions. 5 CRS used different criteria to select the eligible kebeles, like proximity to market. IMPEL | Implementer-Led Evaluation and Learning 4 Annex A: Pre-Analysis Plan Table 1. List of interventions across the two evaluation packages6 Interventions Basic PSNP Ifaa Enhanced Livelihoods Saving Group X X Financial Literacy Training X X Support and training in business plan development X X Credit guarantee fund (conditional capacity building) X Value chain financing co-investment X Youth fund ($250) X Gender Youth and Social Dynamics Implementation of PIM Gender provisions: Monitoring implementation of PIM gender provisions (e.g., exemption of Pregnant and Lactating Women (PLW) until the child is 2 years, FHHs without able-bodied labor in their house (i.e. have young children only provide her share of the household labor; Women will work 50% fewer hours on public works than men; Women will be assigned to light works. Construction of day care centers) X X Monitoring implementation of gender-based violence (GBV) action plan included in the PSNP 5 (e.g., The program Grievance Redress Mechanism (GRM) supporting to accept appeals related to GBV and refer to locally available GBV response services X X Leadership training for women and youth in leadership position X Community Conversations for adults and youth X PSNP Systems Provision of three food components (wheat, oil, and pulse) to Permanent Direct Support (PDS) and PW clients—to meet the daily food kilo/calorie requirement. X X Woreda / Kebele Food Security Task Forces (WFSTF/KFSTF) Capacity Building (woreda, kebele, community level) X X FSTF Capacity Building, specifically focusing on Leadership & Communication Essential training, in addition to the basic FSTF capacity building X Private sector engagement: transportation of food from PDP to FDP, construction of SEIs (same as PSNP basic) X Health and Nutrition 6 The complete list of interventions can be found in section 5.6 Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 5 Interventions Basic PSNP Ifaa Enhanced Linkage to services (Ensuring transfers for Temporary Direct Support (TDS), supporting TDS PW clients to attend PW SBC sessions, and other Health and Nutrition services like ANC (antenatal care), growth monitoring, immunizations, etc.) X X Capacity building training for government and partner staffs￾Adolescent nutrition, Community based management of Acute Malnutrition (CMAM), RLs materials. X X Government of Ethiopia (GoE) basic health extension program X X Additional health extension programs (includes remote trainings, lead parents, motivation of health development armies) X System strengthening through the capacity building and provision of materials (referral pads, formats, reg. books, SC materials, etc.) X Environment and Natural Resource Management (NRM) Training Woreda GoE on equitable allocation and disbursement of resources for PSNP plan implementation X X Watershed management planning X X Implementation of the environment and social management framework (ESMF) X X IWRM (water supply, risk management) and Water Benefits Calculator) X Farmer Managed Natural Regeneration (FMNR): land restoration technique—introducing FMNR approach, organizing user groups and leveraging with agro-forestry practices. X WASH Water development, monitoring, and governance X X Community Led Total Sanitation and Hygiene (CLTSH) X Market Based Sanitation and Hygiene X 2.2.2. Identification Strategy The evaluation will compare the outcomes in treatment kebeles receiving Ifaa Enhanced interventions to those same outcomes in control kebeles who receive only Basic PSNP interventions. 7 Kebeles will be randomly assigned to either the treatment or control group; because these two groups are randomly selected, on average, they have similar characteristics. Thus, the differences in outcomes 7 Because of the differentiation between livelihood eligible and livelihood non-eligible kebeles, we are effectively making two comparisons. Comparison 1 is livelihood kebeles receiving PSNP Basic (including livelihood interventions) against livelihood kebeles receiving the Ifaa Enhanced package. Comparison 2 is non-livelihood kebeles receiving PSNP Basic (excluding livelihood interventions) against non-livelihood kebeles receiving the Ifaa Enhanced package without livelihood interventions. IMPEL | Implementer-Led Evaluation and Learning 6 Annex A: Pre-Analysis Plan can be attributed to the differences in interventions across the two groups. In the baseline study the research team will perform a series of statistical exercises to verify that the two groups are effectively similar. In addition to this, the statistical methodologies used to evaluate the research questions will allow us to account for the possibility of imbalance in some dimensions. Kebeles are the administrative unit for many of the planned RFSA interventions. Given that not all interventions will reach all kebeles, and, not all kebeles will receive an Enhanced package, it was deemed ethical to randomly select the kebeles that would receive the Enhanced package. Moreover, it was deemed infeasible by the research and program teams to conduct randomization at the household or individual level as spillover effects to neighboring households would be large. In addition, ethical considerations arising from excluding neighboring households from interventions within a kebele were too large. 2.2.3. Randomization Strategy The research team will work with CRS to randomize the rollout of the Ifaa Enhanced and PSNP Basic interventions at the kebele level. Based on power level calculations (see section2.3), kebele-level randomization would be adequately powered to detect impacts on most outcomes of interest. Selection criteria of kebeles: Ifaa will work in 9 Woredas, made up of 241 kebeles. Out of these, 34 kebeles were excluded from the study due to being perpetually insecure and other reasons specified by CRS. In addition to this, 11 kebeles were purposefully selected to receive Integrated Watershed Management+ (IWM+) 8 and won't be included in the IE study. Out of the remaining 197 kebeles, 120 kebeles were randomly selected to be part of the IE study. Half of them are livelihood eligible kebeles and the other half are not eligible livelihood kebeles. The kebeles that are part of the IE study will be randomly allocated to one of the two groups; 50 will receive the Basic PSNP package and the remaining 70 will receive the Ifaa Enhanced package of interventions. 9 The evaluation team will use a stratified randomization approach to guarantee a better balance. This technique involves dividing the sample of kebeles into groups sharing similar characteristics. Based on the information provided by CRS, the evaluation team will use two strata, namely Woredas10 and livelihood eligibility. The next step is to allocate kebeles within a given Woreda and livelihood eligibility status to treatment and control. For example, let’s consider a hypothetical Woreda with 24 kebeles, out of which 12 are livelihood eligible kebeles. Within this group of kebeles, 5 will be allocated to the control group and 7 will be allocated to the treatment group. The same will be done for the other 12 kebeles that are not eligible to receive livelihood interventions. This approach ensures that treatment and control groups are balanced by the strata used. 8 Since these kebeles were selected based on specific characteristics and not randomly selected, it is not possible to construct a valid control group. 9 A randomization where both packages are given to the same number of kebeles was deemed infeasible due to the total number of households targeted by the IP. 10 The Woreda was used as the stratum of randomization for two main reasons: (i) it will allow to provide balance of treatment assignment geographically and (2) the woreda is the main administrative structure for local government which shapes local public expenditure and public service delivery. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 7 2.3. Power Calculations and Sampling Strategy 2.3.1. Power Calculations To calculate the required sample size, we considered a set of focal outcome indicators that satisfied the following criteria: were relevant to the research question, were available, easy to collect, and covered the population groups of interest. Table 2 contains descriptive statistics for 5 focal indicators for Amhara and Oromia, computed using data collected as part of the SPIR program endline evaluation.11 These indicators are directly related to food insecurity for the relevant populations affected by the RFSA package: 2 indicators at the household level, one indicator for women, and 2 for children. The proposed sample size would allow the evaluation team to identify a reasonable minimum detectable effect (MDE) for all of them. We are confident that the statistical power will be enough to identify changes in other outcomes of interest. Table 2. Descriptive statistics of selected outcome indicators Variable Mean SD ICC N. HH12 Raw score from 8 Food Insecurity Experience Scale (FIES) questions 3.39 2.63 0.32 3,775 Household is moderately or severely food insecure based on FIES score 45.3% 50% 0.27 3,775 Met Minimum Dietary Diversity for Women 7.9% 27% 0.06 3,704 Children aged 6–23 months who meet minimum dietary diversity (5 of 8 food groups 1.2% 0.11 0.10 722 Children 6–23 months of age who meet the minimum meal frequency 67.9% 0.47 0.17 720 Note: SD stands for standard deviation. ICC stands for Intra-cluster correlation. N. HH stands for number of households To select the sample size, the evaluation team started with a sample size of around 4200 households, 13 which is aligned with initial budget discussions. This number of households corresponds to 35 households per kebele, or 1750 households in treatment and 2450 households in the control arm. It is important to note that for the outcomes related to children we only considered 6 households per kebele (around 19% of the 35 households) since not all households have children 6–23 months old. This mirrors the number of households with children under 36 months sampled in the SPIR endline evaluation survey (720 out of 3775), as well as the percentage of households that had a child 6–23 months old in the 2016 11 Because information for the 120 kebeles part of the IE study was not available, we relied on information collected as part of the SPIR program in the areas of Amhara and Oromia. Outcome Data from SPIR evaluation endline; See Alderman, Harold; Gilligan, Daniel O.; Hidrobo, Melissa; Leight, Jessica; Ramani, Gayathri V.; Taffesse, Alemayehu Seyoum; and Tambet, Heleene. 2021. Impact evaluation of the strengthen PSNP4 institutions and resilience (SPIR) development food security activity (DFSA): Endline report. 12 N of HH corresponds to the number of households in the SPIR endline report. For outcomes related with children only households with children aged 6–23 months were included. 13 This number includes the treatment and control groups. Since 120 kebeles will be part of the study, this corresponds to surveying 35 households per kebele. IMPEL | Implementer-Led Evaluation and Learning 8 Annex A: Pre-Analysis Plan Ethiopia DHS survey (17.8 percent). In later steps the research team explores the implications of changes in the proposed cluster size, to come to a final decision about the total number of households to be surveyed. Table 3 presents the MDE sizes for each of the 5 indicators. The following assumptions were used in their computation: • Sample size will be based on a cluster randomized design • Intra-cluster correlation (ICC), 14 base level (mean) and standard deviation (SD): values specified in Table 1 • Power level: 80% • Confidence level: 95% Table 3. 15 Minimum Detectable Effect (MDE) Sizes for Key Food Security Outcomes in Amhara and Oromia Assuming a cluster size of 50 for the control group and 70 for the treatment group (35 households per cluster) Variable Mean MDE MDE as % of SD Raw score from 8 FIES questions 3.39 0.268 0.101 Household is moderately or severely food insecure based on FIES score 45.3% 12.11pp 0.243 Met Minimum Dietary Diversity for Women 7.9% 4.1pp 0.154 Children aged 6–23 months who meet minimum dietary diversity (5 of 8 food groups) 1.2% 4.14pp 0.37 Children 6–23 months of age who meet the minimum meal frequency 67.9% 11.25pp 0.24 Table 4 presents the power calculations for a design with 50 clusters in the control group and 70 in the treatment group. This corresponds to comparisons between PSNP Basic (the control group) and the Ifaa Enhanced package. The last column of the table computes the ratio of the MDE to the standard deviation of the indicator. Based on the proposed 35 households (6 for children’s outcomes) per kebele, we can see that the MDE for the indicators at the household level and the ones associated with women are between 0.10 and 0.24. This is reasonable for an RCT that is trying to measure the impact of a large package of interventions and is also aligned with the values for similar studies. 16 The MDE for the 2 14 The intra-cluster correlation is the fraction of the total variance of an outcome that can be explained by the within cluster variance. 15 The MDE was computed using the command power in STATA. For continuous variables the command used was power twomeans `var_mean’, m1(30 33 35 40 45 50 60) k1(50) k2(70) power(0.8) rho(`icc’) direction(upper/lower) one-sided cluster. For binary variables the command used was power twoprop `var_mean',m1(30 33 35 40 45 50 60) k1(50) k2(70) power(0.8) rho(`icc') direction(upper/under) one-sided cluster. The variables `var_mean’ and `icc’ where directly obtained from the SPIR data. 16 See Alderman, Harold; Bachewe, Fantu; Gilligan, Daniel O.; Hidrobo, Melissa; Leight, Jessica; Ledlie, Natasha; Ramani, Gayathri V. and Taffesse, Alemayehu Seyoum, . 2019. Impact evaluation of the strengthen PSNP4 institutions and resilience (SPIR) development food security activity (DFSA): Baseline report. This study reports values of MDE for three variables (Child HAZ, Mother’s nutrition knowledge and household food gap) that are around 0.3 SD. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 9 indicators related to children’s outcomes are larger than the other three; this is partly a reflection of the lower cluster size for this indicator. A reduction in this MDE would require a sizable increase in the sample size of the study and was not deemed feasible from a budgetary point of view. In addition to looking at the whole 120 kebeles, the research team explored the statistical power associated to looking within livelihood eligible or not eligible kebeles. For this exercise we considered 25 clusters in the control group and 35 clusters in the treatment group. Because we are considering half of the kebeles (60), the MDE increases for all the indicators. The values obtained, even though larger, are still aligned with the values for similar studies. 17 The one indicator with a very high MDE is associated with a children’s outcome. As discussed before, that high value is due to only some households having children in the specified age bracket. Table 4. Minimum Detectable Effect (MDE) Sizes for Key Food Security Outcomes in Amhara and Oromia Assuming a cluster size of 25 for the control group and 35 for the treatment group (35 households per cluster) Variable Mean MDE MDE as % of SD Raw score from 8 FIES questions 3.39 0.380 0.14 Household is moderately or severely food insecure based on FIES score 45.3% 16.87pp 0.33 Met Minimum Dietary Diversity for Women 7.9% 6.1pp 0.22 Children aged 6–23 months who meet minimum dietary diversity (5 of 8 food groups) 1.2% 6.93pp 0.62 Children 6–23 months of age who meet the minimum meal frequency 67.9% 15.41pp 0.33 The research team explored the implications of increasing the proposed sample size of 35 households per kebele. As can be seen in Figure 1, an increase in the number of households of 50% (increase of 20 households) has only a marginal impact on the MDE for the indicator “Household is moderately or severely food insecure based on FIES score.” A similar behavior was observed for the other indicators. The low impact of an increase in the cluster size is related with the characteristics of a cluster RCT. Because of the similarity of households within a kebele, increasing the sample size within a cluster has diminishing returns in terms of reductions of the MDE. 17 See previous footnote. IMPEL | Implementer-Led Evaluation and Learning 10 Annex A: Pre-Analysis Plan Figure 1. 18 MDE for the indicator Household is moderately or severely food insecure based on FIES score (different cluster sizes, 50 control clusters and 70 treatment clusters) The final aspect to determine the sample size considers the possibility of attrition (non-response). Based on the literature and an estimate of the attrition from SPIR phase I, the research team considered that an attrition rate of 10% was a conservative enough value. Given this value, the study suggests sampling 4,680 households or 39 households per cluster at baseline. 19 2.3.2. Sampling Strategy Based on the discussion in the previous section, the evaluation team will be sampling 39 households in each one of the 120 kebeles, for a total of 4680 households at baseline. This section will discuss the strategy used to sample households and individuals within the households. At the end of the section, we provide a description of how the sampling will be operationalized. Households that meet the inclusion criteria will be randomly sampled in each kebele. Household inclusion criteria: To be included in the sample, households must be PSNP participant, as these are the households targeted for the CRS Ifaa interventions. We will also limit the sample of households to those with women of reproductive age (15–49 years old), as these are target women for many indicators. In addition, this strategy will increase the likelihood that the household will have a child under the age of 5. This strategy will primarily exclude elderly households and is justified by the following reasons: (i) most PSNP households have at least one woman of reproductive age, 20 and (ii) because of the nature of the Ifaa Enhanced Package, elderly households are not likely to be substantially affected by the enhanced 18 All the power calculations portrayed in this graph were computed using the Stata command twoprop `var_mean’, k1(30 45) k2(45) m1(35 40 45 50 60) power(0.8) rho(`icc’) onesided(lower) cluster. The negative effect size is related to the fact that we expect the interventions to reduce the FIES score. 19 According to our calculations: 120*35/0.90 = 4,666 households total, or 38.8 households per cluster rounded to 39 households per cluster or 4,680 households. 20 Based on authors calculations from a PSNP4 dataset, 82.5 percent of PSNP households had a woman of reproductive age. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 11 package. Most of the interventions, such as those related with nutrition, youth, and livelihoods, target households with children, youth, or working age members. In addition, many nutrition and health outcomes are specific to young children or women of reproductive age. Accordingly, the sample selected will be representative of the households predominantly targeted by the PSNP+RFSA interventions. Within individual household selection: The evaluation team proposes to randomly select or purposively21 select one target individual for each outcome22 as opposed to interviewing every eligible individual. The primary reason for this choice is that interviewing every eligible individual in the household is very time-consuming and costly not only for the household but also for data collection. For power reasons, we do not need more than one individual per household, and in general outcomes would be highly correlated within households. Thus, the additional information provided is limited. Typically, interviewing multiple individuals per household is useful if the objective is to compare outcomes across individuals in the same household: for example, in polygamous households comparing outcomes for first versus second wives. However, this is not part of the IE design, and thus it is not worth the additional costs. Given the previous considerations, Table 4 provides more detail about the sampling strategy. Table 5. Sampling Strategy Module Sampling Choice D: Children’s Nutritional Status and Feeding Practices23 Prevalence of exclusive breastfeeding (children 0–5 months): randomly sample one child in this age range. Children’s feeding practices and diets (children 6– 23 months): randomly sample one child in this age range. Children’s diarrhea (children 0–59 months): randomly sample one child in this age range. E: Women’s Health, Nutritional Status, Dietary Diversity, and Family Planning Randomly sample one women 15–49 years old G: Agriculture Select the person most informed about agriculture production in the household24 J: Gender (Cash) Select adult most knowledgeable about household affairs and spouse25 21 This selection will be used when we want to target one person more knowledgeable about the set of questions. 22 The within household selection processes will be embedded in the survey tool. When a random member of the household needs to be selected the survey tool will do the randomization using the household roster. 23 Causal Design will randomly select children 0-5 months, 0-23 months and 0-59. This selection will be done independently, so that the same child could be selected twice (e.g., a child 8 months old could be selected for age bracket 6-23 months and age bracket 0-59 months). 24 This selection will be done by asking the household head. To address issues around ownership and control, the survey will contain follow up questions on specific individuals involved in different activities. 25 In the case that the household head is not married or in a union, the questions related to a couple won’t be asked. In the case of polygamous households, we will randomly select from available wives. IMPEL | Implementer-Led Evaluation and Learning 12 Annex A: Pre-Analysis Plan Module Sampling Choice K: Gender Access to Credit and Group Participation Select adult most knowledgeable about household affairs and spouse26 To sample the households, we will use a sample frame of PSNP households provided by the IP. The limitation of this dataset is that no information on the age of the household’s members is provided. For each kebele, PSNP households will be randomly ranked27 and the first 39 households will be surveyed. If a household doesn’t have a woman of reproductive age, the survey will be immediately terminated, and the next ranked household will be surveyed. 2.4. Cost-Effective Analysis The CEA will focus on the cost-effectiveness of the entire RFSA implementation (measured by the Ifaa Enhanced package of interventions) over the course of its entire implementation. It will not examine the relative cost-effectiveness of different packages of interventions or the PSNP Basic package of interventions since CRS does not track their expenses by intervention. Due to some specificities in the way CRS collects its costs (see section 2.4.2 on cost accounting), the Causal Design team is working on an updated CEA protocol. This section, as well as section 4.4, should be considered as preliminary and will be updated in the next months. This analysis will be viewed from the following dimensions: • Output or outcome results data from monitoring indicators of the entire intervention. • Detailed cost per outcome calculations for selected and relevant monitoring indicators; and, • Perceptions of effectiveness from the implementers. The conclusions will contribute to building a body of knowledge towards understanding the cost￾effectiveness of resilience, food security, and emergency interventions. 2.4.1. CEA Perspective The CEA will use the donor’s perspective of costs and outputs: often called the program perspective. While this perspective does not capture the true cost of providing interventions to society (such as the cost of the farmer’s time or materials), it is a useful perspective for understanding the cost-effectiveness of programs. This perspective is often used to understand where cost savings can be achieved for the IPs. It is also done to compare alternative development approaches for achieving the same outcome or output. In the program perspective, expenditures by external stakeholders (e.g., households) will not be included in the final cost-effectiveness analysis, but these will be considered and discussed as leveraged contributions from other parties. 26 In the case that the household head is not married or in a union, the questions related to a couple won’t be asked. In the case of polygamous households, we will randomly select from available wives. 27 Within kebele each household will be assigned a random number. Households will be ranked by this number. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 13 2.4.2. Ingredient-Based Costing We will use ingredient-based accounting of project delivery costs. 28 This approach allows for a detailed, disaggregated understanding of the implementation costs by the expenses (or resources) that make up an intervention. This approach will be facilitated by the total non-salary costs accounting done by CRS Ifaa and its IPs. This should provide more information on interventions where economies of scale or efficiencies can be achieved, which may be useful for future program design and management decisions and may also be helpful for understanding the cost for scaling any interventions. Ingredient costs will be disaggregated into expenditures such as training, travel, consultant fees, communications, office supplies, building rent, equipment rentals, utilities, facilities costs, vehicle expenses, and equipment. CRS and its IPs will be using the same financial software, so costs across the entire activity will be accounted for using the same ingredients, or expenditure types. We will collect administration costs separately (e.g., project staff salaries, equipment, supplies, fringe benefits). We will also include indirect expenses from headquarter staff, or those who are not billable entirely to Ifaa. However, these costs will be treated separately (see analysis section below). Lastly, we will separate the refinement period and implementation period costs in the final analysis. 2.4.3. Period of Analysis and Real Figures for all Cases The timeline for both the cost and the measure of outputs will be from the beginning of the BL evaluation data collection (2022) until EL data collection (2025). We will use 2022 as a base year (using real figures by removing inflation from the analysis).29 2.4.4. Cost-Effectiveness Ratio The analysis will evaluate the cost-effectiveness ratio (CER) of difference before and after the RFSA from key monitoring indicators The CER will reflect the cost on a per unit basis, with different measures of effectiveness for the units. These measures of effectiveness are incremental outcome measures that will come directly from monitoring indicators. The ratio takes the form below: 𝐶𝐶𝐶𝐶𝐶𝐶 = 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 + 𝐴𝐴𝐴𝐴 𝐴𝐴 𝐴𝐴 𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶 𝑀𝑀 𝑀𝑀 𝑀𝑀𝑀𝑀 𝑜𝑜 𝐸𝐸 A discussion on fixed and variable costs and which measures of effectiveness might be used are included in the CEA Analysis section below. 28 An earlier version of this PAP proposed activity-based costing. However, a conversation with CRS on March 29, 2022 confirmed that they do not have the ability to track expenses by activities. CRS and its partners will be reporting expenses by “ingredients.” CRS did however confirm that they will be able to track expenses by their sub-contractors, which are each working on different activities. We have requested more information to see if it might be possible to still track expenditures by activity (e.g., “Strengthening government services,” “Expanding extension services”) if there is little to no overlap in activities by the sub-contractors; however, this information was not shared by the time of this draft. If later it becomes clear that there is little to no overlap in activities implemented by the sub-contractors; we may still be able to provide costs by activities as well as ingredients. 29 Base year means the analysis will be done in 2022 USD. This implies that inflation in years after 2022 will be removed from the cost calculations, allowing for a comparison on real costs alone. IMPEL | Implementer-Led Evaluation and Learning 14 Annex A: Pre-Analysis Plan 2.4.5. Qualitative Context for Interpretation CEA Results To complement the CER results, we will also interview key stakeholders to better understand where they perceive any cost efficiencies to exist and what may have been driving those results. Once the CER results are available, interviews with project managers and other key stakeholders will explore qualitative aspects of implementation. This line of inquiry will help the evaluation team understand what aspects of implementation may have been rather expensive and any strategies for making the implementation more affordable. Finally, we will leverage these interviews to understand whether the CEA results might be capturing non-RFSA interventions and if that might lead to interventions appearing more or less cost-efficient than they would be in the absence of the actions of external stakeholders. Key questions for the qualitative CEA inquiry include: • Have the planned outcomes been achieved, and if not, why not? Was this due to implementation challenges or to other factors, independent of the project’s ability to deliver? • If the achievement is significantly beyond what was expected, what are the reasons? Are there external factors that contributed to this over-achievement? Or was there something about the implementation that led to these results? • Was the project able to leverage resources from other parties? What other costs were there (beyond IPs) that were incurred for Ifaa’s achievements? • What other interventions were operational in the same areas as Ifaa that may have helped contribute to Ifaa’s achievements? Were other activities or interventions in the area leveraged well? • From your perspective, what aspects of this intervention were done cost-effectively? Why? Which aspects were not done cost-effectively? Why not? • What were unexpected costs or relatively expensive costs in the Ifaa program? Additionally, we will thoroughly review project documentation (e.g., quarterly and annual reports and the midterm evaluation) for possible cost variations and their causes. 3. DATA COLLECTION All questionnaires will be drafted initially in English using Open Data Kit (ODK) software. After receiving BHA, IMPEL, and IP’s approval, Causal Design will translate surveys into Oromo, Amharic, and Somali utilizing local partner staff. 3.1. Survey Design Impact Evaluation: Causal Design intends to implement the same household questionnaire at both the BL and EL periods, with the exception of anthropometry which will only be collected at endline. The foundation of the survey will draw from selected BHA indicators from the BHA baseline/endline indicators handbook30 as confirmed by BHA, IMPEL, and IPs. Where required, surveys will be adapted to 30 https://www.usaid.gov/food-assistance/documents/ffp-indicators-handbook-part-i-indicators-baseline-and-endline-surveys￾RFSA Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 15 local context, and adjustments will be made between survey periods. The questionnaires include a combination of the following modules: • Module A: Household identification and informed consent • Module B: Household roster • Module C: Food access (e.g., FCS and FIES) • Module D: Children’s nutrition and health • Module E: Women’s nutrition, breastfeeding, and antenatal care • Module F: Household water, sanitation, and hygiene • Module G: Agriculture • Module H: Household Poverty • Module K: Gender Access to Credit and Group Participation • Module J: Gender–Cash • Module R: Resilience Module 3.2. Outcome Indicators The list of outcome indicators for the RSFA are in the appendix. These indicators reflect discussions with USAID/BHA and intervention programming that CRS will implement in target areas. These indicators are also listed in the BHA baseline/endline indicators handbook referenced in section 3.1 above. Anthropometric data: Due to the COVID-19 pandemic and the ability to evaluate impact using only EL data within this evaluation design, anthropometric data won’t be collected as part of the baseline survey. 3.3. Enumerator Training Causal Design will engage a local data collection partner to recruit and train enumerators. Survey manuals and other training materials will be developed prior to the training period. The Causal Design team will monitor practice surveys during the training to verify comprehension and functionality of the instrument and the performance of the enumerators. Currently, Causal Design has scheduled (1) a pre￾test focused on the survey tool, followed by 2) testing during enumerator training and then 3) field piloting31 before starting the survey process. 3.4. Data Management Data protocols: Questionnaires will be drafted using ODK, and all household survey data will be collected with electronic tablets utilizing SurveyCTO, a standard data collection application that allows for secure data storage and options for monitoring data quality. Causal Design staff will monitor incoming survey data to flag potential enumeration errors early in the data collection process. In accordance with best practices and regulation around human subject testing and data privacy, access to personally identifiable data will be limited, and anonymized data will be utilized for analysis. Data management protocols will also be approved by a U.S. certified Internal Review Board (IRB) and by the 31 The pilots will take place in 3 kebeles outside of the area of study. These kebeles will be chosen so as to have similar characteristics to the area of study. IMPEL | Implementer-Led Evaluation and Learning 16 Annex A: Pre-Analysis Plan Ethiopian Society of Sociologists, Social Workers, and Anthropologists (ESSSWA) prior to any data collection, storage, and analysis. Initial validation: Causal Design will work closely with IPs and BHA to review preliminary findings in accordance with the contextual validation activity outlined previously. Quality assurance: Causal Design’s internal Data Management Protocol (see section 5) outlines the activities and strategies that the research team implements to ensure that all data collection efforts meet industry and sector standards and expectations. This includes assurance that the data reflect high levels of USAID’s five data quality standards: 32 validity, reliability, precision, integrity, and timeliness. These efforts are then mapped onto the wider phases of the project to demonstrate when they are being implemented and at what level. A full version of this protocol is available upon request. 3.5. Tentative data collection timeline The following table contains a list of all the activities related to data collection, as well as a tentative timeline. Note that these timelines are preliminary and might be subject to changes. Table 6. Baseline Data Collection timeline (preliminary) Activity Completion Time IRB Approval (Local and US) May 9, 2022 Survey training and pilot May 3, 2022 – May 14, 2022 Quantitative data collection May 15, 2022 – June 24,2022 Preliminary Indicator Tables July 5, 2022 Table 7. Endline Data Collection timeline (preliminary) Activity Completion Time IRB Approval (Local and US) May 9, 2022 Survey training and pilot October 28 – November 15, 2024 Quantitative data collection November 18 – December 13, 2024 4. ANALYSIS To assess the impact of the RFSA intervention over the course of the program, the Causal Design team will (1) present preliminary descriptive analysis utilizing baseline data, (2) conduct regression analysis to estimate the impact of the Ifaa RFSA utilizing both BL and EL rounds of data, and (3) estimate the cost￾effectiveness of the different packages of intervention. 33 32 Conducting Data Quality Assessments | Program Cycle | Project Starter 33 In addition to the experimental evaluation, Causal Design will perform a process monitoring evaluation. This evaluation is meant to assess the implementation process itself. Any systematic strengths and weaknesses of the implementation strategy Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 17 1.1 Descriptive Analysis The preliminary analysis using the BL data will show the extent to which the randomization was successful in achieving balance at BL. The analysis will consist of descriptive statistics (presenting means and standard deviations) for the full evaluation sample and by treatment arm for all the BHA indicators specified in section 5.7. To ensure that the randomization was successful, we will present means for the treatment and control groups, 34 and then test whether any differences in means across the two groups are significant. The subsequent impact analysis (presented in section 4.2) can then correct for any imbalances by adding additional control variables as needed. 4.1. Impact Analysis The evaluation activities will use Ordinary Least Squares or OLS35 for continuous outcomes and linear probability models for binary outcomes. As needed, the research team will incorporate additional specifications that are meant to enhance statistical power, increase the validity of constructed comparison groups, or both. Examples include ANCOVA methodology that can be used to better account for BL levels of indicators and outcomes of interest. Based on the original research question36 the evaluation will use the BL and EL data to estimate the impact of the Ifaa Enhanced interventions on indicators observed at EL. The IE model will estimate the average effect of the Ifaa Enhanced interventions on households within treated kebeles, compared to households in control kebeles who received only PSNP Basic interventions. The basic ANCOVA model: (1) 𝑖𝑖𝑖𝑖 = 𝛽𝛽0 + 𝛽𝛽1 + 𝛽𝛽2𝐿𝐿 + 𝛽𝛽3 ∗ 𝐿𝐿 + 𝛽𝛽4 𝑖𝑖𝑖𝑖0 + 𝜀𝜀𝑖𝑖𝑖𝑖 where 𝑖𝑖𝑖𝑖 refers to the outcome variable for individual or household i in kebele j at EL and 𝑖𝑖𝑖𝑖0 refers to BL values for the same outcome indicator; is an indicator equal to 1 if kebele j received the Ifaa Enhanced interventions; 𝐿𝐿 is an indicator equal to 1 if kebele j is a livelihood eligible kebele. The impact of Ifaa enhanced package compared to PSNP basic package for non-livelihood kebeles is represented by 𝛽𝛽1. 𝛽𝛽3 measures the additional impact of the livelihoods enhanced interventions and 𝛽𝛽2 measures the difference between livelihoods eligible and non-eligible kebeles in the PSNP basic arm. 37 The error term, 𝜀𝜀𝑖𝑖, will be clustered at the kebele level. (2) 𝑖𝑖𝑖𝑖 = 𝛽𝛽0 + 𝛽𝛽1 + 𝛽𝛽2𝐿𝐿 + 𝛽𝛽3 ∗ 𝐿𝐿 + 𝛽𝛽4 𝑖𝑖𝑖𝑖0 + 𝑀𝑀 + 𝜀𝜀𝑖𝑖𝑖𝑖 and process will be examined, as well as any contextual factors that may impinge on effective implementation. This evaluation will provide recommendations based on observations after careful qualitative data collection and analysis. The process monitoring evaluation is not part of the experimental evaluation and is not discussed in this Pre-Analysis plan. A separate Pre-Analysis plan for the process monitoring will be delivered at the end of the year. 34 In addition to this, the information will be available at the kebele and Woreda level and can be provided to the IPs or BHA. 35 In statistics, ordinary least squares (OLS) and linear probability models (LPM) are methods for estimating the unknown parameters in a linear regression model. They are standard econometric methods used to establish and estimate empirical relationships between outcomes and a range of explanatory factors 36 Section 2.1.1 contains the research question. 37 This difference is both due to kebeles in the two groups being inherently different and also because livelihood eligible kebeles are receiving PSNP Basic livelihood interventions. IMPEL | Implementer-Led Evaluation and Learning 18 Annex A: Pre-Analysis Plan To account for the stratification, equation (2) adds a set of block dummies 𝑀𝑀 . Although the random selection of kebeles ensures the regressors in (1) and (2) are exogenous, we include a third specification controlling for various household-level 𝑋𝑋𝑖𝑖 and kebele-level 𝑋𝑋 covariates in equation (3). This is because randomization was done prior to the BL with limited information. Therefore, we cannot rule out the possibility of imbalance in some dimensions. (3) 𝑖𝑖𝑖𝑖 = 𝛽𝛽0 + 𝛽𝛽1 + 𝛽𝛽2𝐿𝐿 + 𝛽𝛽3 ∗ 𝐿𝐿 + 𝛽𝛽4 𝑖𝑖𝑖𝑖0 + 𝛽𝛽5𝑋𝑋𝑖𝑖𝑖𝑖 + 𝛽𝛽5𝑋𝑋 + 𝑀𝑀 + 𝜀𝜀𝑖𝑖𝑖𝑖 Sampling weights: In order for the evaluation team to include sample weights, we need to have a complete list of PSNP households with women of reproductive age for each study kebele. If we can obtain such a list, then the evaluation will include results of both weighted and unweighted estimations. Sampling weights will be calculated as the inverse of the probability of selection of the household in each kebele. This will give us a representative sample of PSNP households with women of reproductive age in the 120 target kebeles, however, these 120 target kebeles are selected based on specific criteria mentioned above and not representative of all Ifaa kebeles. Separate weights will also be calculated for indicators and adjusted to compensate for household and individual non-response. Given that sample weights are not needed to measure the causal impacts of the Enhanced package compared to the Basic package, we will also conduct unweighted estimations. 38 Standard errors and p-values: Standard errors will be clustered at the kebele level. Given the large number of outcome variables, it is important to consider that as the number of outcomes tested increases, the likelihood of finding a statistically significant effect when there is no true effect (Type I error) increases. To account for this, we will report both the standard p-values and the p-values corrected for Family-Wise Error Rate and the sharpened q-values corrected for the False Discovery Rate. To generate q-values, outcomes will be organized into outcome ‘families’ according to sector (i.e., food security, child nutrition and health, women’s health, WASH, agriculture, poverty, gender dynamics and resilience). Attrition and missing data: In the case of significant levels of attrition, BL data on originally selected households will be compared with BL data of households that are present at EL. The research team will be able to test if attrition (or non-response) was imbalanced (by regressing the attrition dummy on treatment status) and/or non-random (by regressing the attrition dummy on various outcome indicators measured at BL). If attrition was found to be non-random and imbalanced, we can construct Lee Bounds—a conservative measure of the upper and lower bounds based on the most extreme sample selection—or conduct Inverse Probability Weighting. The Analysis will not attempt to impute missing data points and responses will be ignored for the purposes of impact analysis. This will apply to questions where respondents refused to answer, stated an inability to answer, or otherwise unable to respond. Cases of implausible data will be shared with the enumeration team to verify the validity of the response or understand the root of the error. 38 For a review of when sample weights are needed for causal estimates see Solon, Gary, Steven J. Haider, and Jeffrey M. Wooldridge. "What are we weighting for?." Journal of Human resources 50.2 (2015): 301-316. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 19 4.2. Supplementary Analysis on Resilience Indices In addition to the impact analysis described above, we will conduct additional descriptive analyses on the BL 8 (Adaptive Capacity Index), BL 9 (Absorptive Capacity Index), and BL 25 (Transformative Capacity Index). This analysis aims to provide additional insights on which elements of adaptive, absorptive, and transformative capacities are driving the overall index scores to provide useful programming insights for CRS. The analysis will be conducted at endline for both baseline and endline adaptive, absorptive, and transformative indices to provide insights for CRS on which capacities were relatively strongest and weakest at both points in time and which have seen the most growth over the evaluation period. We will report the weights on the index subcomponents generated from the principal components analysis (PCA) procedure to assess this. We will also assess how these weights compare to the overall distributions of each subcomponent to determine which subcomponents are most driving the overarching index scores. 4.3. Cost-Effectiveness Analysis CRS measures its costs by ingredient (or type of expenditure, see section 2.4.2 on cost accounting), rather than by intervention or activity. Therefore, it is not possible to differentiate costs between the PSNP Basic (the control group) and the Enhanced package of interventions (the treatment group), which is necessary to tie the costs to the impact evaluation. Therefore, we will instead look at all costs for the RFSA and compare those to the progress of the activity as measured by the monitoring indicators, which should be reflective of the entire RFSA rather than just one package of interventions. In this way the costs match the output indicator. This means the CEA will be not reflect the results of the impact evaluation but of the before-after measures of progress. Since monitoring indicators do not contain a rigorously defined counterfactual, the CEA will not be able to directly measure cost-effectiveness attributable to an intervention. Monitoring indicators for the Ifaa evaluation are proposed below. These measures were identified based on how closely these results can be attributed to CRS’ programming for the entire RFSA (encompassing the full Ifaa Enhanced package of interventions). At the same time, we have also tried to identify measures that can best capture the full extent of the interventions and its results, or at least reflect most of the interventions. These overall outcome measures could include costs per: • Reduced incidence of people living on less than $1.90 per day • Household with reduced poor or borderline food consumption score (FCS) • Increased incidence of children under five years of age with a healthy weight 4.3.1. Cost Data As indicated in the CER explained above, cost data will be captured for fixed costs and for variable costs in the year in which the expense occurs. Definitions and collection plans for each type of cost are outlined below. Fixed costs: Fixed costs are those costs that do not change based on the implementation of interventions. Fixed costs include the salaries of the senior project management personnel (e.g., the Chief of Party), financial, contract, and monitoring and evaluation (M&E) staff. Additional fixed costs IMPEL | Implementer-Led Evaluation and Learning 20 Annex A: Pre-Analysis Plan such as rent, security, and utilities that were incurred in Ethiopia will also be considered (we will not include non-Ethiopia fixed costs, which might slightly underestimate the costs of project implementation, but we believe the burden of collecting the data will be high). This data will need to be collected in cooperation with CRS and its partners. Variable Costs: Variable costs are operational costs. Data for these costs (and the associated ingredient￾based costing) will be pulled from the Ifaa financial database. To the extent possible, we will provide detailed cost data for the materials that were required for implementation (e.g., training, travel, consultant fees). We will also perform the analysis for all implementation costs, and only those after the refinement period. Costs will be collected from CRS as well as its partners. 4.3.2. Cost-Effectiveness Analysis Results Because our approach only can measure the costs for the whole RFSA, the associated CEA will produce one cost-effectiveness ratio per outcome measure relying on performance monitoring measures for Ifaa. Our results may suggest, for example, that it costs $15 per household with an improved FCS score. Without the ability to measure costs for the PSNP Basic package, there is no immediate relevant comparison group to compare this figure to. As a result, the question becomes whether $15 per household with improved FCS score is relatively cost-effective. To answer this question, we would need to compare this cost-effectiveness ratios to another cost￾effectiveness ratio for a similar program with the same outcome measure. There are several other external programs that may help characterize or provide some insights about the relative effectiveness of Ifaa: • We believe that we may be able to compare these results to a cost-effectiveness analysis from other RFSA impact evaluations in Madagascar which will have similar cost accounting and outcome measures. (although based on impact data rather than performance monitoring indicators). Other RFSA impact evaluations and accompanying CEAs may also be directly comparable (e.g., in Malawi, Uganda, and the other impact evaluations from Ethiopia). • We anticipate that we can compare the Ifaa CEA results to the CEA results for the second RFSA that Causal Design is evaluating in Ethiopia, which will use the same outcome measures and a similar cost perspective (though the programming that is measured under the impact evaluation is different) , although this RFSA will also rely on impact data. • Finally, we will review literature to identify other similar programming, such as Feed the Future activities or IEs from IFPRI, that were subject to an impact evaluation and measured cost￾effectiveness using the same outcome measures. We believe this specific criterion will limit the amount of evidence that may be available for comparison; however, it may be possible to impute cost-effective ratios with available information (e.g., we may be able to create our own cost-effectiveness ratios if impact evaluations and performance indicators provide overall program costs and the same outcome measures). There will be limitations to comparing the RFSA CEA results directly to any of these other programs listed above, specifically related to the similarity of the programming, type of data used to measure Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 21 effectiveness, and the context in which each activity operated. All limitations will be discussed in the final report. 4.3.3. Qualitative Data We intend to complement our understanding of these results in interviews with key stakeholders (in the form of key informant interviews). These interviews will take place after the cost-effectiveness analysis is complete, and contingent on the availability of the stakeholders for interviews. Selected interviewees for key informant interviews will be individuals with relevant experience and who are knowledgeable about project implementation and the associated costs to provide rich insight. Currently, we anticipate these key informant interviews to be held with project management and possibly USAID personnel who are very familiar with the implementation of the interventions—but we may also include external stakeholders or direct participants. These interviews will be semi-structured interviews, driven by the methodology questions identified earlier in this document, as well as the results of the CEA analysis. 5. APPENDIX 5.1. Data Management The objective of this section is to provide detailed guidance towards Causal Design’s policy and protocols when storing, coding, and reporting data collected or shared with any staff member. All staff members including permanent salaried staff, permanent/part-time consultants, and previous staff are bound to uphold these agreements as part of their employment agreement with Causal Design as indicated in the employee handbook. If any violations or accidental sharing of information that is not encrypted is mistakenly shared outside of Causal Design. The staff member shall immediately notify the Chief Privacy Officer (Keith Ives, also CEO) and the appropriate notifications will be sent to the IRB, clients, and any study participant whose data has been compromised. This handbook drawn from an array or resources around data management and data quality assurance mechanisms including: • Handbook for Safeguarding Sensitive Personally Identifiable Information. Department of Homeland Security, March 2012 • Callahan, Mary Ellen. Handbook for Safeguarding Sensitive Personally Identifiable Information, Handbook for Safeguarding Sensitive Personally Identifiable Information § (2012). • “Research Protocols.” Innovations for Poverty Action, August 23, 2018. https://www.poverty￾action.org/researchers/research-resources/research-protocols. • Chuang, Erica; Diamond Pollock, Harrison; and Wylstra, Stephanie. “Reproducible Research: Best Practices for Data and Code Management.” Innovations for Poverty Action., November 2015. 5.2. Data Quality • Create inception plan before launching survey operations: The inception plan is an operational plan that covers timelines, staffing needs, logistics, and procurement for your survey, for all IMPEL | Implementer-Led Evaluation and Learning 22 Annex A: Pre-Analysis Plan stages including questionnaire development, training, piloting, tracking, interviews, and quality assurance. Your inception plan must be in line with your budget(s); for example, you cannot survey more respondents in the baseline than your budget estimated — without overspending during your endline. • Create data quality assurance plan and materials before launch: The data quality assurance plan lays out in detail the requirements for backchecks, high frequency checks, accompaniments, spot checks, and any other data quality assurance activities. The scope of the data quality assurance plan should not only include technical products, but also data flow, roles and responsibilities, reporting schedules, actionable items based on output, and incentive programs for the field team. It also includes your staffing needs, which may change over the course of the survey. • Bench test survey (ideally at least two weeks in advance): Bench testing means testing your survey in the office with a minimum of three different testers. You will save time and money by making sure your survey works well BEFORE launching field data collection. Bench testing is an iterative process wherein testers run the survey in different scenarios and provide feedback, while the programmer(s) make changes; note that even small changes to a survey must go through the bench testing process again, as it is easy to make mistakes that affect other parts of the survey. This process works best if the "paper" survey is considered mostly complete and has already been reviewed by central decision-makers on the project. • Pilot survey (ideally at least one week in advance): Every survey must be piloted prior to the beginning of the survey in communities outside your study sample. Your pilot should look as close to actual surveying as possible — you may even decide not to tell your field team it is a pilot. Ideally, every question that is included in the final survey should be piloted prior to launch. For surveys using Digital Data Collection, a pilot should include field testing of both the survey program and devices. Remember to leave time to make corrections to errors you identified during piloting. • Accompany surveyors in first week of survey: Field supervisors must accompany a subset of field officers' interviews to monitor field officer performance and to check for survey issues. All field officers must be personally accompanied at least once during the first week of the survey. Accompaniments can be scaled down as the survey progresses, especially by leveraging digital supplements like audio recordings and meta-data. • Implement and act on high frequency checks: High frequency checks provide insight into ongoing field team and data quality concerns before they become too entrenched or too late to manage. By running HFCs, you can regularly analyze (comparative) field officer performance, compliance with ethics requirements, response frequencies and outliers, duplicates, and other project-specific data quality issues. HFCs are meant to provide the evidence needed to successfully guide and manage a field team on a daily basis, and thus must be accompanied by strict guidance on roles and responsibilities, reporting schedules, and triggered actions (e.g., what outliers would trigger re-interviewing a household). • Implement and act on backchecks: A backcheck (also known as a field audit or re-interview) refers to when a highly qualified field officer (also known as a back-checker) visits a respondent a second time to re-administer a selection of questions from the original questionnaire. Those backcheck responses are then compared to the original responses. An IPA generated code bcstats program can be used to identify discrepancies between answers, and thus to identify problems with the questionnaire, field team, or both. Your quality assurance plan should have included a backcheck randomization plan, as well as an action plan for what to do when you encounter discrepancies. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 23 • Double enter & reconcile paper surveys: Although paper surveying is now uncommon, there are strict protocols for data entry from paper surveys. Each survey must be entered by two separate data entry operators who cannot compare responses. When there are discrepancies between their entries, they must be reconciled by a third data entry operator who looks at the original survey closely. In-house data entry can be replaced by online firms, which also provide double entry and allow for you to review discrepancies against the original survey responses. 5.3. Data Security & Research Ethics If the IRB is used on you project: • The Principal Investigator (PI) is responsible for maintaining Institutional Review Board (IRB) approval throughout project lifecycle (e.g. submissions, renewals, amendments, human subjects certificates): Any study conducting human subjects research must have the approval of at least one IRB; note that each project is different, so you should consult with your PIs and IRB Coordinator about how best to get IRB coverage for your project. A typical lifecycle includes approval of the initial research protocol, annual renewals, and amendments when critical items change, such as the questionnaire, staffing, research protocol, or risk level. All project staff, partners and investigators who can see encrypted personally identifying information (PII) must have up-to-date human subjects’ certificates. Any deviation from the protocol, or any unexpected risk to respondents, must be reported as unexpected events to the IRB. Use Salesforce to keep track of all IRB approvals and upcoming renewal dates. • Retire your project with all IRBs once the project is complete: Once your study is complete, you should retire or otherwise officially close out your IRB with all the reviewing IRBs. For the Causal Design IRB, you should retire your study when (A) all study interventions and activities are complete, and (B) you are no longer actively, regularly working with identified data. Other IRB(s) may have slightly different standards or procedures, so you should check with your reviewing IRB administrator(s) where relevant as well. Whether the IRB is used on your project or not: • Create data security plan and set up encryption (using Whisp.ly to transfer between partners https://whisp.ly/en?) before launch: Respondents' confidential data should be encrypted at all stages, starting at the moment of data collection. This includes while it is on the data collection device, during wireless transmission, while on an external server (e.g., Kobotoolbox, Commcare, SurveyCTO, etc.), when it is on a cloud storage system (e.g., Google Drive or Dropbox), and while on laptops and removable media (hard drives, flash drives). Any time the data is stored on a server that is not controlled by Causal Design; it must be separately encrypted so that the company that controls the server cannot access the data. You must plan beforehand how you will ensure encryption at each of these steps, and how it will be maintained after your project has been officially closed if you are retaining any PII. If you are using any IRB any un-encrypted data is uploaded to the cloud or emailed, you must file an unexpected event report to your IRB(s) and comply with any ruling they make. If you are not using an IRB you should report this to the Chief Privacy Office of Causal Design, Keith Ives. • Maintain data security plan (especially encryption) throughout project lifecycle: At every stage of the project lifecycle, data should be properly protected. Among other things, this means PII should remain encrypted during storage and transmission, and passwords should be restricted to the critical members of your research staff. IMPEL | Implementer-Led Evaluation and Learning 24 Annex A: Pre-Analysis Plan • Use new UID in deidentified dataset: When you share or publish un-encrypted data, it must be deidentified, i.e. there must be no identifying information in the dataset, such as name or address, or a combination of variables that can be used to identify a respondent. You should also replace your original unique identifier (UID) with a new unique identifier. You should do this at the end stage of your project, when you have finished matching across waves or different data collection activities. 5.4. Knowledge Management & Transparency • Back up data in at least two locations: There must be at least two copies of the data available at all times. During data collection, this will likely mean on a KoboToolBox/SurveyCTO/CommCare server, as well as on a laptop and synced to Google Drive; do not delete server data until it has fully synced to Google Drive as a protection from laptop theft. Post data collection, this could mean backing up your data on an external hard drive on the extremely rare chance that a major cloud service like Google Drive fails. • Save ALL project files to and ONLY to Google Drive: Causal Design project files must be stored in the My Drive\3_CD_Projects superstructure on Google Drive. This includes in particular: raw data files, final versions of questionnaires, back check questionnaires, survey manual, project log and survey notes, high frequency check files, analysis do-files, IRB documentation, and replication code. 5.5. Data Storing/Sharing guidelines The following bullets are intended for projects which are completed and are going to be stored long term on the Google Drive or any other survey. 5.5.1. Detailed Steps for Preparing Data and Code: Remove PII: Check thoroughly for PII, and make sure to remove before sharing with the data repository team. • All direct identifiers such as unique IDs (social security numbers, bank account numbers, and so on) should be removed before storing or with the. Indirectly identifying data such as combinations of variables which could uniquely identify participants should also be considered carefully before storing or sharing data. Include clear variable labels and code value labels: • Make sure that variables are clearly labeled. • If it is a variable collected directly from the questionnaires, indicate this with a question number. If it is constructed, either include the construction in the name or label, or if complex/lengthy, include additional information in notes. • Ensure that value code labels are provided, as they are needed for interpreting the data. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 25 Include code file(s) with headers/comments: • Headers: Include header with name of person who last wrote/edited the code, date, and software used (package and version). • Comments: Use comments in the code to indicate which tables are produced. Prepare Readme files: • Please indicate: 1) which files are included in what is shared; and 2) how data and code files relate (i.e., what code runs on which data, to produce which outputs). We have a template for readme files that we are happy to share and is located on every project folder. Include survey instruments: • Ensure that you are sharing the final version used to collect the data. 5.5.2. Data Curation Steps That Data Repository Staff Will Complete As the data repository team works on the dataset submitted, we will conduct the following three steps to ensure the quality of the materials that we share in our repository. Confirming there is no PII shared in data or code files: • It is the responsibility of the original researcher (s) to ensure that PII is removed, and IRB protocols do not permit sharing PII with the data repository team. However, the DR Unit will double-check that PII is removed before sharing, because of the high level of importance of maintaining confidentiality of research participant’s information. Examining data and code for usability: • The data repository team will examine variable names and labels, value codes, and the statistical code. As a part of sharing high-quality data, we will attempt to fill in variable labels and/or notes in the dataset where we are able to glean further information from published tables or communication with researchers. Where there are many unclear variables, we may ask the researcher(s) to improve the dataset before publishing. • We will run the statistical code to ensure that it produces the published tables. Checking and sharing related materials: • Supplementary readme file: As we conduct our data curation steps, we will track and share information that will help site users understand the steps that we took, and what we found. We will confirm with the original researcher before sharing this file along with the data. • Study-level metadata: We have created a custom template with fields that we will fill in from all studies. 5.5.3. Project Language for Quality Assurance and Control The following plan outlines the activities and strategies that the research team intends to put in place to help ensure that the data collection for the IMPEL meets industry and sector standards and expectations. This includes assurance that the data reflect high levels of the following dimensions: IMPEL | Implementer-Led Evaluation and Learning 26 Annex A: Pre-Analysis Plan validity, reliability, precision, integrity, and timeliness (USAID 2016). These efforts are then mapped onto the wider phases of the project to demonstrate when they are being implemented and at what level. Table 8. Data Quality Assurance Activities Project Phases Data Quality Assurance Activity Quality Dimensions Phase 1: Discovery and Design Literature Review and Sector Assessment Validity Indicator Mapping Validity Questionnaire Designed to Promote Proper Response Coding Integrity Phase II: Collection Integration of Data Collection Activities with Existing IMPEL staff capacity Reliability/Integrity Develop Data Collection Protocols and enumerator training Reliability Phase III: Analysis Preliminary Data Spot Checks Integrity Enumeration team review and Feedback Validity Phase IV: Reporting Scheduled Analysis and Reporting Timeliness Validity The research team will work closely with IMPEL and BHA project staff to ensure that the indicators and research design are valid measures. This is primarily addressed through efforts leading up to the proposed design of research activities. In this case, the research team combines the following to ensure that the proposed indicators and methods are valid for the scope of the research: • Literature Review and Sector Assessment of current thinking and practice focused both on wider academic and implementation-based publications and on IMPEL specific reports and projects related to measurement; and • Sector Experts feedback and consultation is included into all phases of the baseline. Analysis created as a result of research efforts will undergo stakeholder review to further ensure that findings are interpreted correctly and account for contextual realities. Reliability The research team will also ensure that protocols are put in place to ensure consistency in data collection efforts. This includes the creation and implementation of training (if necessary), sampling, and data collection protocols, which undergo internal peer-review. Precision At the outset, the project will build on efforts to ensure data validity and utilize the literature review, desk research, and project documentation to comment on and revise our analysis plan that connects theories of change pathways to research objectives. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 27 Integrity Data integrity within the IMPEL program is delivered through specific systems and processes that manage data entry and safeguards to ensure proper data input. The data input will rely on the following ecosystem: The questionnaire will be designed to provide clear instruction on proper response coding; Daily updates to our server will ensure proper data input by centralizing data input across data collectors; and The research team will conduct spot checks on data taken during the population survey. Timeliness To ensure data timeliness, the research team has created a project work plan to ensure that scheduled analysis quickly follows data collection in order to guarantee that relevant findings can inform project implementation decisions and strategy. 5.5.4. Quality Assurance During the implementation of this research, our Team Lead, Reimar Macaranas, supported by Causal Design’s Project Management Office, will use state-of-the-art enterprise resource planning software to manage the project timeline, budget, and resources, to ensure high-quality, on-time delivery of all work products. Causal Design uses Intuit’s suite of programs, which integrates timesheets, accounting, staff availability, budgeting, and project management functions to provide integrated access to all information needed to effectively manage projects. Mr. Macaranas will adhere to Causal Design’s policy that any changes to implementation plans, or timelines are immediately updated in this system, to ensure we can always provide a real-time estimate of the expected resources necessary to complete a task or project, including both staff time and budget. Causal Design also understands the paramount importance of quality assurance/quality control on all work products and technical deliverables, and of effective and frequent communication between the Team Lead and IMPEL. Causal Design’s “no surprises” policy requires all project managers to keep clients regularly informed about progress, challenges, solutions, and concerns. IMPEL will therefore always be fully informed of all relevant activities and immediately consulted when guidance is needed. This policy ensures that the Causal Design team and KWSH will be partners in critical decision making on, and problem resolution in, all matters. 5.5.5. Quality Control Quality control for all products will be managed by Mr. Reimar Macaranas. His academic training, years of leadership in research and evaluation, and role as Chief Operations Officer will be utilized to ensure all the Team’s products meet or exceed the expectations of IMPEL. Mr. Macaranas will also provide executive-level oversight and senior technical review of all project tasks and deliverables. He will ensure IMPEL has access to the Causal Design team’s key technical personnel that can answer questions at any time. He will verify that Causal Design’s rigorous QC procedures are implemented and ensure that all IMPEL | Implementer-Led Evaluation and Learning 28 Annex A: Pre-Analysis Plan deliverables submitted to IMPEL meet the highest quality standards and require minimal rounds of revision. These quality control systems will ensure that the Causal Design team provides the highest possible quality services to IMPEL with minimal service disruption. 5.5.6. Data Processing and Procedures Quantitative data will be collected using tablets and stored in a secure cloud-based server; analysis will be done using STATA. Causal Design will manage team for doing the data clean up, data entry, data analysis and reporting. 5.5.7. Ethical Considerations We will ensure that our team, including all enumerators and contractors working on the project, adhere to the ethical guidelines outlined in the American Evaluation Association’s Guiding Principles for Evaluators. The Causal Design team has experience in preparing IRB protocols for evaluations. For many evaluations, we have successfully worked with Solutions IRB to obtain IRB clearances on domestic and international studies. After recruiting household survey participants, we will obtain informed consent for each person to be interviewed. We will explain the purpose of the study, the topics of the interview/focus group, the person’s rights as a participant, including that their responses will remain confidential, and that participation is voluntary. We will provide contact information for the study investigators and appropriate IRB(s) (if used). The data collectors will review the information to be collected. We will use plain language and translate into Khmer. Participants will provide oral consent. The Causal Design team will work with IMPEL to obtain any necessary national or local IRB clearances as appropriate. 5.6. Intervention Packages Table 9. Intervention Packages Basic PSNP Ifaa Enhanced Livelihoods Saving Group X X SILC PSP (Private Service Providers) Model X Financial Literacy Training X X Financial education X Support and training in business plan development X X Life skill training X On-farm Livelihoods pathway X X Off-farm Livelihoods Pathway X X Wage Employment Pathway X X Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 29 Basic PSNP Ifaa Enhanced Climate smart agriculture practices promotion though Lead Herders/Lead Farmers — Follower Farmers (LH/LF-FF) X Technical training on selected pathways X X Producers marketing groups X Seven steps of marketing training X Business and marketing skills training X X Access to Finance - Formal financial linkage (Credit track) X X Livelihood transfer track ($300) X X Credit guarantee fund (conditional capacity building) X Value chain financing co-investment X Youth fund ($250) X Technical support and follow up (coach and mentor) X X Public work livelihoods linkage (e.g., Area closure) X X Private Sector Engagement (Agro-dealers, SILC PSP, community animal health workers (CAHWs) & Private veterinary pharmacist) X Gender Youth and Social Dynamics Implementation of PIM Gender provisions: Monitoring implementation of PIM gender provisions (e.g., exemption of PLW until the child is 2 years, HHs without able-bodied labor in their house (i.e., have young children only provide her share of the household labor; Women will work 50% fewer hours on public works than men; Women will be assigned to light works. Construction of day care centers) X X Monitoring implementation of GBV action plan included in the PSNP 5 (e.g., The program Grievance Redress Mechanism (GRM) supporting to accept appeals related to GBV and refer to locally available GBV response services X X Leadership training for women and youth in leadership position X Community Conversations for adults and youth X The Faithful House/Islamic Families Life X Functional literacy (it targets women and youth in leadership including in kebele and community FSTFs) X Youth voluntarism X Youth Employability Skills (YES) curriculum, I am an Entrepreneur (IAE) X School gender club's curriculum X IMPEL | Implementer-Led Evaluation and Learning 30 Annex A: Pre-Analysis Plan Basic PSNP Ifaa Enhanced · Trainings on GBV Interventions at various levels using developed curriculum X · Establishment of GBV committees at kebele level X · Support to Woreda GBV task force (including linkage with Kebele level committees, material supports) X Male engagement X Gender champions X Social cohesion including youth peace ambassadors (YPA) X Dignified Families Approach X PSNP Systems Provision of three food components (wheat, oil, and pulse) to PDS and PW clients—to meet the daily food kilo/calorie requirement. X X FSTF Capacity Building (woreda, kebele, community level) X X Private sector engagement: transportation of food from PDP to FDP, construction of SEIs X X Strengthen KFSTF as well as establishing and supporting an inclusive and participatory Community Technical Coordinating Forum (CTCF) to facilitate community visioning and inclusive kebele development plan X FSTF Capacity Building, specifically focusing on Leadership & Communication Essential training, in addition to the basic FSTF capacity building X Private sector engagement: transportation of food from PDP to FDP, construction of SEIs (same as PSNP basic) X Health and Nutrition Linkage to services (Ensuring transfers for TDS, supporting TDS PW clients to attend PW SBC sessions, and other Health and Nutrition services like ANC (antenatal care), growth monitoring, immunizations, etc.) X X Capacity building training for government and partner staffs￾Adolescent nutrition, CMAM, RLs materials. X X GoE basic health extension program X X GoE supportive supervision coaching X X GoE led community SSB sessions (expected to be held monthly targeting temporary direct support clients). X X Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 31 Basic PSNP Ifaa Enhanced Additional health extension programs (includes remote trainings, lead parents, motivation of health development armies) X System strengthening through the capacity building and provision of materials (referral pads, formats, reg. books, SC materials, etc.) X CRS Enhanced SBC (those interactive SBC tools like Speaking books, child Nutrition cards, Care Group Model (CGM) modules related counseling cards, ATK Communication Materials and other reinforcing messages embedded in CMAM/IYCF materials, etc.) X Audio toolkit communication material on nutrition, harmful traditional practices (HTPs) X CGM approach X Adolescent nutrition - School clubs X Religious leaders’ mobilization and training on prioritizing Pregnant and lactating women, CU5 and adolescents during fasting, etc. X Labor and time saving technologies for mothers & care givers X Home garden promotion X Nutrition budgeting (using seasonal food calendar) X Environment and NRM Training Woreda GoE on equitable allocation and disbursement of resources for PSNP plan implementation X X Watershed management planning X X Land capability classification for soil and water conservation purpose X X Standard operation plan (SOP) X X Train community members in planning, implementation and sustaining community assets X X Training on the public work (PW) operational maintenance manual (OM) X X Linking communities to WFSTFs for planning, implementation, and monitoring of community assets X X PW implementation (Biological Soil and water conservation interventions - Agroforestry) X X Support watershed management committees to transition to cooperatives X Implementation of the ESMF X X IMPEL | Implementer-Led Evaluation and Learning 32 Annex A: Pre-Analysis Plan Basic PSNP Ifaa Enhanced IWM+: 39 merging IWM (to develop, restore, and protect degraded water, soil, and land resources) and IWRM (water supply, risk management) and Water Benefits Calculator) X FMNR: land restoration technique—introducing FMNR approach, organizing user groups and leveraging with agro-forestry practices. X Train GoE staffs on the participatory watershed planning with practical demonstration based on using revised watershed guidelines X Strengthen community level capacity to work with the W/KFSTFs to mobilize PW labor during PW sub-project planning, implementation, joint monitoring, and evaluation. X Enhance the level of participation of WFSTF and the woreda watershed technical team to participate on community level PW planning process and joint supervision. X Technical training on ESDM (environment sound design and management) for project staff and local partners. X Strengthen watershed users’ cooperative establishment through capacity building on local resource governance and sustainable management X Quality data management with ICT4D/GIS, developing and dissemination of visualizations/reports of all infrastructure. X Sensitization and orientation on Revised watershed guidelines and proclamations to establish watershed users’ cooperative through capacity building on local resource governance and sustainable management X Digitizing/geo-referencing of all PW activities (point, line, and polygon). X WASH Water development, monitoring, and governance X X CLTSH X Market Based Sanitation and Hygiene X Private sector engagement X Ensuring water quality and safety X School WASH X WASH system strengthening X 39 IWM+ won’t be evaluated as part of the impact evaluation study. A specific set of kebeles was purposefully selected to receive those interventions and thus it is not possible to perform any type of randomization to have a control group. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex A: Pre-Analysis Plan 33 5.7. Indicators List Table 10. Ifaa Indicators List BL # RFSA Indicators Relevant Modules BL 6 Prevalence of moderate and severe food insecurity in the population, based on the FIES C BL 10 Percent of households with poor, borderline, and adequate FCS C BL 12 Prevalence of children 6–23 months receiving a minimum acceptable diet (MAD) D BL 13 Prevalence of exclusive breastfeeding of children under six months of age D BL 14 Percent of children under age five who had diarrhea in the prior two weeks D BL 15 Percent of children under five years old with diarrhea treated with Oral Rehydration Therapy D BL 39 Prevalence of children 6–23 months consuming a diet of minimum diversity (MDD-C) D BL 11 Percent of women of reproductive age consuming a diet of minimum diversity (RiA) E BL 26 Percent of births receiving at least four antenatal care (ANC) visits during pregnancy E BL 36 Percent of women in a union who have knowledge of modern family planning methods that can be used to delay or avoid pregnancy E BL 37 Percent of women in a union who made decisions about modern family planning methods in the past 12 months E BL 16 Percent of households using basic drinking water services F BL 17 Percent of households with soap and water at a handwashing station on premises F BL 18 Percent of households in target areas practicing correct use of recommended household water treatment technologies F BL 19 Percent of households in target areas practicing open defecation F BL 27 Percent of households with access to a basic sanitation service F BL 21 Percent of producers who have applied improved management practices or technologies G BL 29 Percent of farmers who used financial services (savings, agricultural credit, and/or agricultural insurance) in the past 12 months G BL 30 Percent of farmers who practiced the value chain interventions promoted by the activity in the past 12 months G BL 1 Prevalence of Poverty: Percent of people living on less than $1.90/day H IMPEL | Implementer-Led Evaluation and Learning 34 Annex A: Pre-Analysis Plan BL # RFSA Indicators Relevant Modules BL 2 Depth of Poverty of the Poor: Mean Percent shortfall of the poor relative to the $1.90/day H BL 40 Daily per capita expenditures (as a proxy for income) in USG-assisted areas H BL 32 Percent of women and men in a union who earned cash in the past 12 months J BL 33 Percent of women in union and earning cash who report participation in decisions about the use of self-earned cash J BL 34 Percent of women in union and earning cash who report participation in decisions about the use of spouse/partner's self-earned cash J BL 35 Percent of men in union and earning cash who report spouse/partner participation in decisions about the use of self-earned cash J BL 41 Percent of women/men in a union who are members of a community group K BL 42 Percent of women/men in a union with access to credit K BL 43 Percent of women/men in a union who make decisions about credit K BL 8 Adaptive capacity index R BL 9 Absorptive capacity index R BL 23 Ability to recover from shocks and stresses index R BL 24 Percent of households that believe local government will respond effectively to future shocks and stresses R BL 25 Transformative capacity index R BL 38 Index of social capital at the household level R BL 31 Percent of households participating in group-based savings, micro-finance, or lending programs R/K Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 35 ANNEX B: LIST OF INTERVENTIONS Table 11. Intervention Packages Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Agriculture and Economic Livelihoods Savings and Internal Lending Communities (SILC) Group X X Target clients: 30% of PW livelihood clients. The clients are both youth (18–29) and adults (30+). Intervention Details: 30% of PW clients will be organized into SILC groups with membership ranging from 20–25. The groups will be self-selected but will consist mainly of PSNP clients. The groups are formed by Field Agents (FAs) who continue providing support to the group as needed. FAs visit the groups once every month for mentorship and guidance. The groups can reform at the end of a cycle (1 year). The groups will meet every week or by weekly and it is an entry point for the livelihood and agricultural sector interventions. Expected outcome: SILC will improve culture of saving and lending and provide members opportunity to borrow money they can use for livelihoods, Income Generation Activities (IGA) engagement and other interventions. A small portion of the SILC money goes towards social support including medical and school fees. SILC Private Service Providers (PSP) Model X Target clients: 30% of PW livelihood clients. The clients are both youth (18–29) and adults (30+). Intervention Details: 30% of PW clients will be organized into SILC groups with membership ranging from 20–25. The groups will be self-selected but mainly PSNP clients. The groups are formed by FAs who also continue providing support to the group as needed. FAs visit the groups once every month for mentorship and guidance. The groups can reform at the end of a cycle (1 year). The groups will meet every week or by weekly and it is an entry point for the livelihood and agricultural sector interventions. In addition to these, SILC PSPs will convert the FAs to PSPs who will continue to support groups on a fee for service basis. CRS will stop paying FAs once they become PSPs. The PSPs will also be support to for PSP networks and they will continue to nurture apprentices from strong SILC members. To diversify PSPs economy, Ifaa will also network IMPEL | Implementer-Led Evaluation and Learning 36 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention PSPs under private sector agro-dealers as a sells agent based on commission (5%). Expected outcome: SILC will improve culture of saving and lending and provide members opportunity to borrow money they can use for livelihoods and other interventions. A small portion of the SILC money goes towards social support including medical and school fees. PSP kebeles are expected to be more sustainable since the PSP are skilled and continue to exist beyond Ifaa support. Financial Literacy Training X X Target Clients: All SILC members. Intervention Details: This a GoE PSNP PIM required training that will be provided to all SILC members on CRS GoE financial literacy curriculum. All households that are selected to participate in livelihoods activities should participate in financial literacy training. Training for PSNP clients will be provided by a variety of service providers depending on the woreda: Financial Service Provider agents, cooperative promoters and accountants, and other specialized service providers such as non-governmental organizations. Training topics: Financial literacy trainings will use a toolkit that will be revised based on the assessment financial planning and budgeting. • Savings—the importance of savings, and how to save. • Understanding credit and manage business. • Calculating at least gross profits. • Risk management and insurance (tailored to locally available insurance types, e.g., credit life insurance). Expected Outcomes: provision of financial literacy skills is expected to increase program clients’ employability and/or engagement in IGAs. Financial education X Target Clients: All SILC members. Intervention Details: This will entail training using CRS smart skill curriculum, financial education module that comprise of four booklets. Main additional topics to the financial literacy above include: • Goals, Income, Expenses, and Budgeting. • Borrowing. Expected outcome: The training will equip members with three key financial management skills. Saving for a purpose: Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 37 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Saving to achieve a set purpose by making and following a savings plan and setting aside surplus income to establish a fund to cover costs should an emergency arise. Borrowing wisely: Borrowing responsibly to meet cash needs by accessing loans based on repayment capacity, using loans for the intended purpose, and repaying loans on time. Effective financial management: Managing finances to meet cash needs and save by identifying cashflows throughout the year, establishing financial goals, prioritizing household, and business expenses, and preparing and following a budget. Support and training in business plan development X X Target Clients: PW who are SILC members. Intervention Details: Following the successful completion of financial literacy training and all technical and business/marketing trainings on the livelihoods checklist, the Development Agents (DAs), Livelihood Extension Workers (LEW), supervisor / coordinator will certify that a client has successfully completed the livelihoods checklist. DAs and LEWs will then assist clients in developing business plans. This business plan will follow the format provided in this Manual and will include the following sections: • Names of participating clients within the household and profile. • Selected livelihood. • Certification of completed livelihoods checklist. • Labor utilization plan. • Input and technology requirement. • Credit or livelihoods transfer requirement. • Production plan. • Marketing plan. • Planned financial flow. • Expected revenue by quarter. • Expected expenditures by quarter Simple sensitivity analysis. • Loan repayment plan (for credit referral clients only). • Declaration, signed by each client. Business plans will be developed at the individual level rather than the household level but will be linked via the Household PSNP Client ID Number and updated in the Management Information System (MIS). IMPEL | Implementer-Led Evaluation and Learning 38 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Expected Outcomes: • To guide clients towards productive and profitable livelihood investments, by helping the client think through how they will earn an income, how they will manage their cash flow, and how they will repay his or her loan. Therefore, it is critical that client’s/household representatives (spouse, youth) be directly involved in the development of their business plan and that they understand all the information that it contains. • To help clients / households better understand the basics of business planning and management. • To help the client obtain financing for a specific livelihood investment. Life skill training X Target Clients: PW who are SILC members mainly youth. Intervention Details: CRS will provide training on life skills using a standard curriculum. Topics include the development of body awareness, critical consciousness, positive self￾worth and parenting, inter-personal communication skills, goal setting and conflict prevention. Expected Outcomes: This will improve agency and assets, includes family, peer, and community engagement to engender support, healthy relationships, and a sense of belonging and contribution to increasing household well￾being and income. Youth will gain and strengthen life skills during key life stages (15–19, 20–24 and 25–29) such as the development of body awareness, critical consciousness, positive self-worth and parenting, inter-personal communication skills, goal setting and conflict prevention. On-farm Livelihoods pathway X X Target Clients: PW households, 75% of SILC members. Intervention Details: Potential crop and livestock pathway selection, provide technical training to clients who selected the pathways and increase production and productivity. The selection criteria include youth and gender inclusive, nutrition dense, climate smart, food security, and high market value. Ifaa proposed 75% of SILC will select the on￾farm pathway. CRS will then use the lead herder and lead farmer extension model to transfer technology to the clients and support them on the respective pathways. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 39 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Expected Outcome: Increased production and productivity of selected commodities. In turn this will help increase household income and diversify livelihoods. Off-farm Livelihoods Pathway X X Target clients: 20% of SILC group members. Intervention Details: Ifaa will support 20% of SILC group target clients to choose among different off farm opportunities in their area to diversify their livelihood income. Detail list of off farm opportunities will be included after the assessment. Off-farm pathways will emphasize new opportunities for youth, including community animal health workers (CAHW), toilet makers, agro-dealer agents, FAs/PSPs, agricultural processing, and water operators. Given the lower interest in off-farm opportunities to date, CRS will assess barriers, examine profitability, and learn more about how to encourage interest. Expected Outcome: Increased access to income, diversified livelihoods. Wage Employment Pathway X X Target Clients: 5% of SILC group. Intervention Details: Support 5% of SILC group target clients to choose among different wage employment opportunities in their area to diversify their livelihood income. Detail list of the wage employment opportunities will be included after the assessment. Provide training on Youth Employability Skills (YES) curriculum and coaching and mentoring. Ifaa will also work with public and private enterprises or employment agencies to gainfully employed. Ifaa will also provide support to GoE One-Stop Centers. Outcome: Increased employment, livelihood diversification, increased income, increase household assets. Climate smart agriculture practices promotion through LH/LF-FF X Target Audience: SILC members who participate in on-farm pathway. Intervention Detail: Fundamental to CRS’ strategy is the understanding that IWM+ is critical for improving rainfed agriculture productivity, and CRS will implement its proven Climate-Smart/Water-Smart Agriculture platform within the context of green water, which emphasizes soil as a water resource. Because the dynamics of soil and water are not limited to plots or farms, the approach requires that agriculture development shifts from plot to farm and landscape scale. Ifaa will identify and organize LF/LH groups; IMPEL | Implementer-Led Evaluation and Learning 40 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention under each LF/LH groups organize FF, implement demonstration on LF/LH plots and Farmer and Pastoral Training Centers (FTC/PTC); organize monthly meetings among LF and FF and quarterly meetings among LF/LH at FTC/PTC. Outcome: Improving rainfed agriculture productivity, in￾crease vegetative production-building natural capital to improve people’s lives—with a focus on water productivity at the farm and landscape scales by reducing degrading activities. Technical training on selected pathways X X Target: All SILC members. Intervention Details: The livelihoods component includes technical training and complementary livelihoods interventions in three pathways: crop and livestock, off￾farm, and wage employment. DAs and Woreda Subject Matter Specialists will provide a series of technical trainings to clients on their selected livelihood. The programme will systematize some of the technical training protocols developed by the Extension Service and the Livestock Development Sector into a robust training course for participants. Lists of required trainings will be tailored to the product / business plans and the livelihood investment planned by the household and will be included in the Livelihoods Guidelines. Ifaa will provide technical training on crop and livestock for government staff, technical partner staff, frontline extension workers (DA, LEW) and clients who selected the on-farm pathways. Technical trainings will be substantial and will be provided at FTCs or PTC and/or at the homes of model farmers where possible to facilitate practical knowledge sharing and learning-by-doing. Linkages with research institutes will be promoted where feasible, in coordination with the Climate Smart Initiative. It is expected that technical trainings be provided for a total of at least 10–20 hours over the course of 4–12 weeks, depending on the type of livelihood pathways chosen by the client. Outcome: to enable clients to effectively participate in their selected pathways. Producers marketing groups X Target clients: 20% of on-farm pathway SILC members. Intervention Details: Target and organize in producer marketing groups, organize training on production and Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 41 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention marketing of selected commodities; facilitate linkage with financial service providers, buyers, and input suppliers. Outcome: Improve marketing skills for producers who participate in value chain activities, increase sale of value chain commodities. Seven steps of marketing training X Target Audience: SILC member who select on-farm and off￾farm pathways. Intervention Details: Ifaa will use CRS’s seven steps of marketing guide which focuses on the practical aspects of linking vulnerable farmers with markets. The guide is the second part of the marketing skill set. The first part, marketing basics should be consulted prior to reviewing this guide. The marketing approach of this guide focuses on the needs of poor farmers. The aim is to ensure that farmers produce sufficient food crops for their household needs and improve income through sales of surplus produce at local and regional markets. The principles can also be used for helping to link farmers to higher value markets including national and export markets. The types of farmers targeted in this guide typically produce on farms of two to five acres (1–2 hectares) of land. Typically, at the start of an upgrading process, farmers will not own mechanized tools, use limited inputs, are not well organized, have no savings schemes or links to formal financial lenders, and for the most part have opportunistic trading relationships with buyers. Ifaa will provide advanced technical training on marketing to the established PMGs on on-farm and off-farm. The seven steps of marketing curriculum comprise of the following: 1. Getting organized, 2. Identifying products and organizing groups, 3. Collecting information for the business plan, 4. Building a business plan, 5. Marketing as a group, 6. Reviewing agro-enterprise performance, and 7. Scaling up. Outcome: Increase sale of value chain commodities and participation on markets. Business and marketing skills training X X Target Audience: All SILC members. Intervention Details: In addition to technical trainings, clients will receive a series of business skills and marketing trainings tailored to their selected livelihood. These trainings will be provided by Woreda Subject Matter Specialists, Cooperative Promotion Officers or Marketing Agency Specialists and should be developed in such way that it IMPEL | Implementer-Led Evaluation and Learning 42 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention would be sensitive to adults, women, and youth. Topics may cover: • Calculating input costs (building on initial training provided during the financial literacy sessions). • Marketing and market facilitation topics, e.g.: o Where to find markets. o Optimal marketing timing. o The advantages and disadvantages of collective marketing (based on product). • Simple risk / sensitivity analysis. Outcome: increase sale of value chain commodities and participation on markets. This will also help clients to develop their business plans. Access to Finance— Formal financial linkage (Credit track) X X Target: 70% of AgEL clients who selected on farm and off farm pathway. Intervention Description: 70% of AgEL clients who selected on farm and off farm pathway will be linked to financial service providers (banks, MFI and RuSACCOs). Clients will take their completed checklist and business plan to FSPs, including MFIs (government and non-governmental organization (NGO) supported / parastatals and private) and RuSACCOs, to access loans. Where necessary, credit committees will provide financial institutions with lists of clients whose business plans are reviewed and passed viability check. DAs will also provide additional support to clients in liaising them with financial institutions with their business plans, as needed particularly for clients who have little experience with credit but did not qualify for a livelihoods transfer. If one client finishes their checklist before another household member, that client can be referred for financing. When the FSP has a group collateral requirement the client will have to wait for the group to finish their checklist. Where credit life insurance available, clients will also be referred to this service. Credit availability from both Finance Service Providers and RuSACCOs will be gauged each year during the planning stage. FSPs and RuSACCOs will benefit from conditional capacity building under the programme, subject to the conditions outlined in the section above. (However, if there is a credit constraint following the completion and referral of Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 43 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention business plans, the programme may recommend that women and youth be prioritized, subject to the review and approval processes of financial institutions.) Conditional capacity building of RuSACCOs. The programme will build the capacity of these RuSACCOs to encourage savings and provide services to their members. PSNP capacity building support to RuSACCOs will include: • Technical assistance, e.g., training of RUSACCO leadership and technical assistance on financial product development and linkages to MFIs as appropriate. • Matching funds (e.g., up to 25%) for building construction and safe boxes. • Matching funds for bookkeepers for 1–2 years. Conditional capacity building of FSPs. The program will provide conditional capacity building of FSPs—including private MFIs—and encourage clients to open individual savings accounts at FSPs where available. PSNP capacity building support will vary depending on the size and needs of the FSPs, but may include: • The development of financial products that are acceptable to Muslim clients (i.e., Sharia compliant). • Staff training. • Programme support costs, e.g., for the provision of financial literacy training to PSNP clients and creation of linkages with RuSACCOs. • For FSPs that open a sub-branch in PSNP kebeles: o Transport. o Matching funds for office furniture. o Matching funds for hardware materials. Outcome: this will help to ensure adequate credit availability at the time of referral. Livelihood transfer track ($300) X X Target Participants: 30% of PW households. Intervention details: Within the PW beneficiary households, the bottom 30% of the poorest households are eligible for selection in the LH transfer track. The amount of livelihoods transfer is USD 300 equivalent amount in Birr using the agreed exchange rate at the beginning of the budget year (July). IMPEL | Implementer-Led Evaluation and Learning 44 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Outcome: Access finance, increased participation in selected pathways. Credit guarantee fund (conditional capacity building) X Target Participants: Financial service providers (MFI/RuSACCOs). Intervention details: Selected financial service providers (MFI/RuSACCOs) will be provided with guarantee fund and conditional capacity building support to facilitate loan provision to 70% AgEL clients. Outcome: this will help to ensure adequate credit availability at the time of referral, increase access to finance. Value chain financing co-investment X Target Participants: Private sector firms. Intervention details: Ifaa will provide co-investment with private sectors who are interested to invest in crop, livestock and off farm businesses which will help farmers to access inputs and sell their output products through the private sectors. This is based on the value chain/market system assessment finding and recommendations. Outcome: this will help to ensure adequate credit availability at the time of referral, increase access to finance, Enhanced input, and output market system. Youth fund ($250) X Target Participants: 20% of youth participating in on-farm and off-farm pathways. Intervention details: 20 % of targeted youth will be eligible for a livelihood grant in the amount of $250. Ifaa will prioritize these clients to start them on their path for asset accumulation. Ifaa will select and train most vulnerable and volunteer youth and provide fund ($250) for startup capital. Outcome: Asset accumulation, youth participation in selected pathways. Technical support and follow up (coach and mentor) X X Target Participants: All SILC members. Intervention details: Follow-up support includes facilitation of access to inputs and linkages to markets as needed and coaching and mentoring of clients. This support should continue on an intensive basis through the end of the first year after the client has started participating in livelihoods interventions, or through the end of the second year for livelihoods transfer clients. For the employment pathway, this will be the employment linkages phase. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 45 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Outcome: To ensure clients effectively implement activities within their selected pathways. PW livelihoods linkage (e.g., Area closure) X X Target Participants: All SILC members. Intervention details: linkage of PW interventions contribute to target client’s livelihood improvement through creating income generation opportunity. Ifaa will ensure that PW respond to PSNP client needs through deliberate linkages of community assets with RFSA activities—for example, ensuring integration between PW and livelihoods, including social service delivery. Outcome: PW responds to PSNP client livelihoods needs. Private Sector Engagement (Agro￾dealers, SILC PSP, CAHWs & Private veterinary pharmacist) X Target Participants: Private Sector Individuals or firms working in Ifaa operational kebeles. Intervention Detail: Ifaa will select potential agricultural inputs supplier and engage in output market linkages. Ifaa will select potential PSPs for agro-dealer and link PSPs with existing agro-dealers to facilitate last mile input supply. Ifaa will select and organize practical training to CAHWs to provide animal health services at respective kebeles where there is limited veterinary services. Create linkage with private veterinary pharmacist to provide livestock vaccination and treatment services to improve livestock health. Outcome: Increased access to inputs and markets for PSNP clients. Gender youth and social dynamics Implementation of PIM Gender provisions: Monitoring implementation of PIM gender provisions (e.g., exemption of PLW until the child is 2 years, HHs without able-bodied labor in their house (i.e., have young children only provide her share of the household labor; X X Target Clients: Women who pregnant, lactating mothers; FHH. Intervention details: Ifaa will implement and monitoring implementation of PIM gender provisions (e.g., exemption of PLW until the child is 2 years, FHHs without able-bodied labor in their house (i.e., have young children only provide her share of the household labor; Women will work 50% fewer hours on PW than men; Women will be assigned to light works. Construction of day care centers). Expected Outcomes: Increased capacity PSNP structures and increased women participation at different levels. Moreover, better nutrition and health for children under 5. IMPEL | Implementer-Led Evaluation and Learning 46 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Women will work 50% fewer hours PWs than men; Women will be assigned to light works. Construction of day care centers) Monitoring implementation of GBV action plan included in the PSNP 5 (e.g., The program Grievance Redress Mechanism supporting to accept appeals related to GBV and refer to locally available GBV response services) X X Target Clients: Women, young women, girls, and boys. Intervention Details: Ifaa will establish and strengthen the GBV referral system (Kebele GBV committees, Woreda GBV service providers, one stop center) This includes Capacity building of GBV service providers, communities, survivors etc. Expected Outcome: Reduced gender-based violence, increased participation in leadership and decision making of women and men in community initiatives. Leadership training for women and youth in leadership position X Target: Women, youth and PWD who have leadership positions in different structures. Intervention Details: Leadership capacity development for Women, Youth & PWD who are in Leadership Position in various Ifaa established and community structures. In Ifaa, based on woreda needs and not ignoring WFSTF/KFSTF, Center for Creative Leadership will expand Leadership Essentials to the newly revitalized Community Food Security Task Forces to increase their ability to advocate for all community members, based on an inclusive visioning process. Expected Outcome: Increased participation of women youth and PWD in leadership positions and community initiatives. Community Conversation for Adults and Youth X Target: Community representatives (women, men, youth, community leaders, frontline gov’t workers including DAs, Health Extension Workers (HEW), and teachers). Intervention details: Communities organized and engaged in dialogue and discussion on issues that affect their lives. Community Conversations are inclusive community-level discussions led by community-based facilitators who support and reinforce household-level behavior change; they serve as a secondary adoption tool, reaching diverse PSNP and non-PSNP clients. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 47 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Expected Outcome: Address gender and social norms, increase social cohesion. The Faithful House/Islamic Families Life X Target: Couples. Intervention details: Ifaa will enhance the knowledge and skills of IPs, government staff and community representatives to facilitate SBC among project participants and ensure that both male and female participants understand gender equity and women/girls’ empowerment to be a benefit to all. The Faithful House/Islamic Family Life engages couples to develop and practice critical reflection, dialogue and joint decision-making skills that influence selected themes for optimal health, nutrition, and production, including equitable decision-making processes, sharing of assets, resources, and workloads. Outcome: Facilitate couple communication and discussion to improve interaction, joint decision making and collaboration amongst family members. Functional literacy X Target: It targets women and youth in leadership including in kebele and community FSTFs. Intervention details: Select and train women, youth & PWD leaders in functional literacy to enhance their leadership skills and develop confidence to lead their respective groups. Expected Outcome: increased participation of women, youth and PWD in leadership positions and community initiatives. Youth volunteerism X Target: Youth (15–29) including PWD. Intervention details: Youth (including PWD) will also be organized to develop their skills, knowledge, and practices collectively through Youth Livelihood Groups, youth volunteerism opportunities such as Peace Ambassadors, and Youth Community Conversation Groups. To respond to the diverse factors that lead to vulnerability, Ifaa will engage with both in-school and out-of-school youth and address the intersectionality of age, gender, and disability. Ifaa will support the specific needs of the following groups: • Relatively low vulnerability (nudge toward graduation). • Vulnerable youth (largest Ifaa focus toward supporting their graduation). IMPEL | Implementer-Led Evaluation and Learning 48 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention • High vulnerability (emphasis on progressing to a less vulnerable state). Adolescent girls and young women (ages 15–19 and 20–29) will receive focused efforts due to the persistent gender gap to improve their self-esteem and ability to tackle poverty and food insecurity. Targeted activities will layer an adolescent girl and young woman approach onto general livelihoods that focuses on building voice, choice, and control. Ifaa will select, organize, train, and deploy youth in volunteer activities to support their communities and gain buy in from the community to maintain positive social dynamic. Youth voluntarism enhances their skill and access employment opportunities. Expected Outcome: income opportunities for youth, skill development for youth, increased participation in leadership and community initiatives. YES curriculum, I am an Entrepreneur X Target: youth (aged 18–29) Intervention details: Ifaa will support 20,000 youth to form youth livelihood groups where SILC and (YES) Training will strengthen soft skills, entrepreneurial mindsets, financial and digital literacy, and employability skills and will orient them as they select or make progress in their livelihoods pathways. Expected Outcome: Youth employment skills development, increased youth employment. School gender club's curriculum X Target: School-going youth aged 15–18 Intervention details: CRS will also leverage the DFSA1 school gender clubs, led by local teachers, to reach boys and girls aged 11–17 with CCL’s girls and boys club toolkit. The toolkit facilitates peer learning around gender equity and harmful traditional practices and provides a safe platform for youth to develop their leadership and life skills. Ifaa will establish, train, and provide curriculum to enable boys and girls facilitate discussion on various Ifaa program topics and develop their communication and leadership skill. Expected Outcome: Youth to develop their communication and leadership skill, increased youth participation in community leadership. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 49 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Trainings on GBV Interventions at various levels using developed curriculum X Target Clients: Men, women, community leaders, Intervention details: CRS will provide 3 days GBV training for men, women, and community leaders using developed curriculum. IPs and government staff/woreda will facilitate the training. Expected Outcome: The training expected to raise the knowledge about GBV issues, that will contribute to a reduction of GBV cases in the targeted woredas and improved referral and reporting mechanisms. Participants will play their roles and responsibilities to prevent GBV, responding to GBV reports and supporting GBV committees. Establishment of GBV committees at kebele level X Target: Men, Women, community leaders, and Government staff. Intervention details: GBV committee will be established at kebele level to facilitate prevention and response mechanism. The trained Ifaa formal and informal community leaders, the school community will strengthen kebele GBV committee to work on existing platforms and referral systems. Expected Outcome: The GBV committees will serve as a bridge between the community and woreda level GBV prevention and response task forces through information sharing and reporting as per the training they received and terms of reference. Support to Woreda GBV task force (including linkage with Kebele level committees, material supports) X Target: Woreda GBV task force. Intervention details: Capacity building support will be provided for woreda GBV task force based on identified gaps. In addition, Ifaa will explore options with the Office of Women, Children and Youth to establish safehouses under their management. Expected Outcome: Strengthen capacity of woreda GBV taskforce and establishment of establishment of GBV survivor’s safe house: 3 safehouses in selected 3 woredas will be constructed and furnished with basic equipment and materials. Male engagement X Target: Male (youth and adult) PSNP clients. IMPEL | Implementer-Led Evaluation and Learning 50 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Intervention details: Select and train male volunteer who are willing to engage and promote transformed social norms. Outcome: Increased role of male in tackling gender norms. Gender champions X Target: couples and individuals. Interventions details: Select and train volunteer couples and individuals who practice & promote positive gender/ social norms. Tibeb Girls will target adolescents aged 11–19 with an animated TV, radio series and comic book, featuring three Ethiopian superhero girls guided by enduring values and leading the audience on a journey to fight against the injustice that girls face daily. To ensure engagement with real-life issues, community-based Gender Champions will be featured as superheroes of their community to encourage others to act against gender violence and inequality. Gender Champions (volunteer role models) tackle gender norms, including GBV, within the community to reinforce behavior change at the household level and open space for the elderly, women, and youth to fully participate in community life and livelihoods. In Ifaa, these approaches will expand to reach more youth. The Better Well-Being media series and frontline workers across all sectors will echo gender equity, inclusion and social cohesion messages and model desired behaviors. Outcome: Increased role of couples and individuals in tackling gender norms. Social cohesion including youth peace ambassadors (YPA) X Target: Male and female youths. Intervention details: Youth-led Community Conversations will be scaled in Ifaa to increase youth perspectives in the community and will be linked to YPAs—a cadre of motivated youth community members who are trained in peacebuilding activities and supported by the program to hold discussions and arrange activities around peace and social cohesion. Ifaa will initiate a program engaging 3,128 trained male and female YPAs in peacebuilding, conflict resolution and social cohesion. Outcome: Increased social cohesion. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 51 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Dignified Families Approach X Target: Couples with children, single headed families. Intervention details: Dignified family/worthy family is a human training curriculum comprised of fourteen sessions with the purpose of ensuring that families exercise their educational and socializing role more effectively, contributing to strengthen peace and social cohesion. Outcome: improved peace and social cohesion. PSNP Systems Provision of three food components (wheat, oil, and pulse) to PDS and PW clients—to meet the daily food kilo/calorie requirement. X X Target: All PSPN households and 5% contingency clients. Intervention details: Three food components (wheat, pulse and veg, oil) is provided to PW, TDS, and PDS clients as per the PSNP transfer schedule to meet the food need of these clients. Accordingly, PDS clients receive food for a period of 6 months starting from January. PW clients receive food for 3 months starting from April. Outcome: Consumption smoothed. FSTF Capacity Building (woreda, kebele, community level) X X Target: FSTF members at woreda, kebele, community level. Intervention Details: Various operational, functional, and technical trainings are provided to FSTF members at regional, zone, woreda, kebele and community levels. The training includes: Leadership & Communication Essential, multi-year planning, PSNP5 PIM, Rural Payroll and Attendance Sheet Sustem, etc. Outcome: Improved PSNP PIM implementation. Private sector engagement: transportation of food from PDP to FDP X X Target Participants: Private sector actors including firms and individuals. Intervention details: CRS will be transporting PW and PDS food from Dire Dawa to FDPs using private transporters through bid process. Outcome: Improved timeliness of transfer and quality of Social and economic infrastructure. Strengthen KFSTF as well as establishing and supporting an inclusive and participatory CTCF to facilitate community visioning and inclusive X Target participants: FSTF (Kebele, and community and CTCF). Intervention details: Mobilize, establish, train, and strengthen CTCF to facilitate community visioning and inclusive kebele development plan. Hold community visioning sessions. IMPEL | Implementer-Led Evaluation and Learning 52 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention kebele development plan Outcome: Improved GoE service delivery, increased accountability, improved planning, and implementation of community initiatives. FSTFs capacity building specifically focusing on L & C Essential training in addition to the basic FSTF capacity building X Target participants: FSTF (Kebele, and community and CTCF). Intervention details: Strengthen the leadership and communication capacity of FSTF structures beginning from Region to Community level. CCL built the capacity of K/WFTSF members in leadership, communication, and management skills to support development of a case management system to help PSNP clients navigate services to meet needs and limit problems arising from fragmentation of services, staff turnover and poor coordination. Outcome: Improved GoE service delivery, improved planning and implementation of community initiatives, improved management of staff turnover. Private sector— Construction of SEIs X Target Participants: Private sector actors including firms and individuals. Intervention details: Construction of SEIs infrastructures is carried out using capital and administrative budget allocated by RFSA. These infrastructures are planned in the woreda annual PSNP plans prepared by FSTFs. The infrastructures include human & animal health posts, additional school classrooms, small scale irrigation, water development projects, DA & HEW residences, FTC, etc. Outcome: Improved economic and social services are achieved when good quality construction is done. Health and Nutrition Linkage to services (Ensuring transfers for TDS supporting TDS PW clients to attend PW SBC sessions, and other Health and Nutrition services like ANC, growth monitoring, immunizations, etc.) X X Target clients: Pregnant mothers, mothers/care with children under 1 year, mothers/care with severe acute malnutrition (SAM) children. Intervention Details: The TDS clients will be assured of accessing the necessary transfer, and need to attend SBC sessions, and other health and nutrition services. Expected outcome: The TDS get necessary transfer; they attend SBC sessions and be referred to all other Health and Nutrition services at the HFs and outreaches. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 53 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Capacity building training for government and partner staffs￾Adolescent nutrition, CMAM, RLs materials. X X Target clients: Partners, Zonal sectors, Woreda sectors, Health facilities, schools, and community volunteers. Intervention Details: Series of different level trainings on Adolescent nutrition, CMAM, Religious Leaders mobilizations, Care Group Model, Food preservations, Nutrition budgeting, Integrated Management of Newborn & Childhood Illnesses (IMNCI), Health Extension Programmes (HEPs), etc. Expected outcome: Participants acquire required knowledge and skills around the mentioned training topics/areas. GoE basic health extension program X X Target clients: The whole community/HHs living in the Kebele. Intervention Details: Implementation of 18 government (Ministry of Health) HEP packages. Expected outcome: Communities improve knowledge and skills around feeding practice, hygiene and sanitation, increased service demands, improved Immunization coverage, improved ANC/PNC coverage, etc. GoE supportive supervision coaching X X Target clients: Woreda Health office staffs, Health facility (Health centers and Health Posts) workers, schools, community volunteers, Nutrition technical committees. Intervention Details: Conducting Joint supportive supervisions, and coaching. Technical support around key Health and Nutrition intervention areas, counseling, on the job trainings, support documentations, motivations, review meetings, etc. Expected outcome: Improved knowledge, skills, commitment; and improved quality of overall Health and nutrition services. GoE led community SBC sessions (expected to be held monthly targeting temporary direct support clients). X Target clients: Pregnant mothers, mothers/care with children under 1 year, mothers/care with SAM children. Intervention Details: PW SBC session runs, the TDS clients will be assured of accessing the necessary transfer, and need to attend SBC sessions, and other Health and nutrition Services. Expected outcome: The TDS get necessary transfer; they attend SBC sessions and be referred to all other Health and Nutrition services at the HFs and outreaches. IMPEL | Implementer-Led Evaluation and Learning 54 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Additional health extension programs (includes remote trainings, lead parents, motivation of health development armies) X Target clients: HEWs, CG promotors, lead parents, HDA and other volunteers. Intervention Details: Motivation, training of CG promotors, lead parents, HDAs and other volunteers on: Adolescent nutrition, CMAM, Care Group Model, Food preservations, Nutrition budgeting, IMNCI, HEPs, etc. Expected outcome: Target targeted clients (HEWs, CG promotors, lead parents, HDAs and other volunteers) will improve motivation, improve their knowledge and skills around listed interventions. System strengthening through the capacity building and provision of materials (referral pads, formats, reg. books, SC materials, etc.) X Target clients: Sectors, institutions communities and respective staffs. Intervention Details: Provision of capacity building trainings listed above, and supply of other materials based on Identified gaps. Expected outcome: Sectors, institutions, and their staffs in RFSA Woreda improve system functionality and enhanced by the quality service rendered through capacitated staffs. CRS Enhanced SBC (those interactive SBC tools like Speaking books, child Nutrition cards, CGM modules related counseling cards, ATK Communication Materials and other reinforcing messages embedded in CMAM/ IYCF materials, etc.) X Target clients: Children under 5, PLWs, Caregivers, Adolescents, first time mothers, and other community members as a secondary contact. Intervention Details: Enhanced SBC materials—Community￾Based Complementary feeding and learning sessions (CCFLS), Child Nutrition Cards, speaking books, Other Essential Nutrition and Hygiene Actions (ENA/EHA) related SBCs, and similar user-friendly SBC tool that will help HEWs to deliver key messaging to illiterate populations. Expected outcome: Knowledge and skill increase, behavioral change. Audio toolkit communication material on nutrition, harmful traditional practices Target clients: HEWs, DA Community Animators, Teachers, and Volunteers. Intervention Details: CCL audio toolkits uses radio drama and a narrator with questions to facilitate group learning around a picture-based guidebook. The focus areas of the program may integrate issues such as gender roles, decision making social issues e.g., child marriage, healthcare, GBV, community engagement focus on women and youth, leadership, communication, nutrition and raise awareness of Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 55 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention community roles and responsibilities, sustainable use of water and infant feeding practices etc. Expected outcome: Enhance a community culture of collaborative learning around health and nutrition through sharing owns experience. CGM approach X Target clients: Children under 5, PLWs, Adolescent girls, Care givers, First time mothers. Intervention Details: Rollout of the CGM approach through the MOH structure and further to HHs: Ifaa will improve the quality and coverage of health education and counseling through the proven Care Group model to build mothers’ nutrition knowledge and skills around optimal nutrition and IYCF. The Care Group Model builds teams of volunteer Lead Parents who conduct trainings in each household to improve coverage. Each volunteer regularly visits 10–15 neighbors, sharing learning and facilitating behavior change at the household level. Expected outcome: Key Health, Nutrition and Hygiene behaviors (through improved knowledge and skills) promoted and eventually changed. Adolescent nutrition — School clubs X Target clients: Health professionals, education offices, teachers, school communities, Health extension workers and club leaders. Intervention Details: Series of trainings for education offices, teachers, school communities, Health extension workers and club leaders using the Adolescent nutrition manuals/materials. Implementation of adolescent nutrition in targeted schools (schools' nutrition clubs). Expected outcome: Improved knowledge and practice of the target group and their community on adolescent nutrition, with focus on all forms of malnutrition and life skills required for adolescents to make health related choices. Religious leaders’ mobilization and training on prioritizing PLW, CU5 and adolescents during fasting, etc. X Target clients: Religious leaders, traditional leaders, religious institution members. Intervention Details: Training and mobilization of religious leaders (RLs), traditional leaders, religious institution members using the RL mobilization guides; then, the RLs pass Health/nutrition related messages using religious IMPEL | Implementer-Led Evaluation and Learning 56 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention platforms to provide sample messages that religious leaders can share with your congregations at any gathering. Expected outcome: Built healthy, loving families and strong communities that practice good nutrition to its own families through availing diversified food. Labor and time saving technologies for mothers & care givers X Target clients: Mothers, caregivers, and men. Intervention Details: Promotion of Labor and time saving technologies. Expected outcome: Mothers and care givers get time to care, feed and clean their children; will also get time to attend health and nutrition services. Home garden promotion X Target clients: Households and institutions in RFSA Woredas. Intervention Details: Mobilization, establishment and promotion of home gardens (Permagarden, Keyhole garden and other traditional gardens) with Households and institutions of RFSA intervention Kebeles. Expected outcome: Households and institutions have home gardens established, use the produce for consumption/ HHs dietary diversifications. Nutrition budgeting (using seasonal food calendar) X Target clients: Households, different RFSA Platforms (SILC Groups, CC groups, CCFLS groups, Dignified family/ The Faithful House couples), schools, FTCs, etc. Intervention Details: Inclusion and integration of Nutrition budgeting to different platform sessions, and other CRS enhanced SBC materials using the seasonal food calendar. Expected outcome: Nutrition budgeting for HH dietary diversity improved in RFSA targeted communities. Environment and Natural Resource Management Integrated watershed management planning and implementation X X Target clients: Watershed committees, watershed communities. Intervention Details: This will entail planning for watersheds in line with the 2020 Community-based Participatory Watershed and Rangeland Development. The guided covers the following key topics: • Step 1: Getting Started at Woreda level. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 57 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention • Step 2: Getting started at community level. • Step 3: Socio-economic and Biophysical Survey. • Step 4: Gender & Social Development (GSD), Nutrition, and Integrated Risk Management. • Step 5: Identification, Prioritization and Safeguards of Interventions that Bring Change. • Step 6: Getting the interventions approved by the general assembly. • Step 7: Organizing watershed/rangeland intervention plan. • Step 8: Implementation Strategies. Expected outcome: Planning, construction and management of community assets improved. Train community members in planning, implementation and sustaining community assets X X Target clients: Watershed committees, PW foremen/women and watershed community members. Intervention Details: This will be training of communities using the 2020 Community-based Participatory Watershed and Rangeland Development guideline. The training will be complemented by mentorship session conducted by Ifaa and GoE staff. The training focused on planning, technical lay out for foremen/women, and management of the NRM assets. Expected outcome: Planning, construction and management of community assets improved. PW implementation (Biophysical Soil and water conservation interventions) X X Target clients: PW clients. Intervention Details: Ifaa will guide PW implementation in line with the PIM. To ensure work norms and standards for PW implementation, Ifaa will support foremen, DAs, community facilitators and woreda experts to provide timely and periodic technical support to the PW activities. Ifaa will ensure all gender provisions associated with PW are implemented. Ifaa will also work towards ensuring timely payment of PW clients. Specific activities withing this intervention include: • Commodity transfer (cash and food). • Construction of additional FDPs. • Stakeholder engagement (GoE, transporters and other NGOs) and discussions with GoE. • Training of GoE stakeholders on commodity management. IMPEL | Implementer-Led Evaluation and Learning 58 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention • Supervision of PW. • Step-by-step implementation of PW in line with the PW PIM annex. Expected outcome: Planning, Construction and Management of Community Assets Improved. Implementation of the Environment and Social Management Framework (ESMF) X X Target clients: Community Watershed Teams. Intervention Details: Ifaa will apply the Implementation of the ESMF In order to avoid or mitigate any undesirable impacts during implementation of PW sub projects and maintain a high level of environmental. This is in line with the community-based participatory watershed and rangeland management guideline. Expected outcome: To ensure that environmental management practices are integrated into watershed development planning and implementation activities. Train GoE staffs on the participatory watershed planning based on the revised watershed guidelines X X Target clients: NRM, Gender, Livelihood, WASH, infrastructure, Health, and Nutrition GOE sector officers. Intervention Details: Train GoE staffs on the participatory watershed planning with practical demonstration based on the revised community-based Participatory Watershed and Rangeland management guideline. Expected outcome: Woreda GOE sustainably engage and technically support the community in planning, implementation, and sustainability of community assets. Enhance the level of participation of WFSTF and the woreda watershed technical team to participate on community level PW planning process and joint supervision. X X Target clients: WFSTF Intervention Details: training WFSTF on community visioning, planning and implementation of integrated watershed management and management of community assets Outcome: Expected outcome: Woreda GOE sustainably engage in planning, implementation, and sustainability of community assets. Support watershed management committees to transit to watershed users' cooperatives X Target clients: Community Watershed Teams. Intervention Details: for sustainability, Ifaa will support watershed management committees/teams to transition to cooperatives. Specific activities under this will include training, legalizing (establishing cooperatives), developing Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 59 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention bylaws, building their capacity through office equipment and material. Expected outcome: Planning, Construction and Management of Community Assets Improved. IWM+: merging IWM (to develop, restore, and protect degraded water, soil, and land resources) and IWRM (water supply, risk management) and Water Benefits Calculator) X Target clients: Community Watershed Teams, watershed community members. Intervention Details: IWM+: is an approach integrating difference disciplines (all sectors) and application of Water Benefits Calculator (decision making tool) and source water protection/water safety plan for planning and implementation of integrated watershed management. IWM+ fills in the planning gaps that exist within the GoE’s guidelines/frameworks and pulls together the pieces and combines disconnected activities. Expected outcome: planning, construction and management of community assets improved. WASH Water development, monitoring, and governance X X Target clients: All Clients. Intervention Details: One of the outputs in the PIM is linkages to available social services facilitated for core PSNP clients. Even though creation or development of the social services is not the role and realistic mandate of the PSNP Program, Ifaa will advocate the woreda FSTF to prioritize water infrastructure development during their annual plan preparation coupled with technical support and back up during the design preparation, infrastructure development, training of WASH Community Organizations (WASHCOs) linking WASHCOs with private sectors for better operation and maintenance and provision of basic tools to the WASHCOs. In addition, Ifaa directly develop Water infrastructures in response to water stress resulted by draught shocks, very high demand and overcrowded water sources that may lead communities to migration, travel very long distance, conflicts arouse because of scarcity. To sustain the developed water infrastructures, Ifaa give emphasis to build the capacity of user community through their WASHCOs to ensure good governance and sustainable service delivery of those facilities. It links up the WASHCOs with spare part suppliers in nearby market, train and capacitate private operators interested in Operations & IMPEL | Implementer-Led Evaluation and Learning 60 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Management business, facilitate enhance linkage between WASHCOs and GoE office for frequent follow up and response on maintenance needs, support them set and collect appropriate tariff, install service monitoring remote sensing technology for immediate maintenance need reporting and action. Expected outcome: To provide safe and quality water for participants by developing the water supply schemes. Monitor functionality of the schemes to provide safe, clean, and sustainable water supply to the participants all the time. Community Led Total Sanitation and Hygiene (CLTSH) X Target clients: All HHs. Intervention Details: CLTSH has three steps = pre-triggering, triggering and post triggering follow up and verification to reach to the result Open Defecation Free (ODF) community or Kebele. At the entry point Ifaa target kebeles are found at different level of these steps and the intervention varies depending on the where the Kebeles status is. After profiling of the Kebeles done, Ifaa will do all the steps in Kebeles there is no triggering happened before, make post-triggering follow up, verification then ODF certification in kebeles where triggering happened but not follow up; Verification and ODF certification will be done where triggering and follow up has been done before. Expected outcome: To improve hygiene and sanitation practices and prevent communicable diseases in the community by using an improved sanitation facilities properly and Kebeles become Open Defecation Free (ODF). Market Based Sanitation and Hygiene X Target clients: All HHs. Intervention Details: Based on Ministry of Health National Market Based sanitation (MBS) Guideline, Ifaa will identify and select Masons in target Kebeles, provide them technical training on production and construction of improved toilet technologies, provide them necessary construction and manufacturing tools, provide them start-up fund through revolving fund and link-up with financial service providers, provide them business development service such as mentoring, certifying, business plan preparation, business to business linkage with other private sectors and suppliers of inputs, demand creation and promotion to create encouraging demand for their products and services through CLTSH and other SBC activities. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex B: List of Interventions 61 Interventions Basic PSNP Ifaa Enhanced Summary of the intervention Expected outcome: To avail different types of sanitation and hygiene products & services through private sectors to improve sustainable hygiene and sanitation practices and for the improved healthy life of the project targeted communities Private sector engagement X Target clients: Private sectors—producers, skilled individuals in Kebeles, input suppliers, retailers. Intervention Details: This activity includes engaging and supporting private sector in WASH related businesses. These includes sanitation, water schemes operation and maintenance, water treatment chemicals and filter suppliers. The support includes training, financial linkage, linkage between the businesses and users, linkage with government sector offices for business development support and technology improvement. Expected outcome: Ensuring sustainable supply of WASH products and services. Engagement of the private sector in providing & supplying different types of sanitation products & services for MBS intervention. Construction and maintenance of the water schemes, sanitation facilities, etc. Ensuring water quality and safety X Target clients: All clients. Intervention Details: Ifaa will use USAID BHA approved Water Quality Assurance guideline to monitor water quality of all protected water sources in the target kebeles by testing the minimum water quality parameters. Also, a preventive water safety planning implemented before drinking water sources polluted. The water quality tested by government laboratory (Chemical and Physical parameters) and by portable testing kit by project staffs (biological). Expected outcome: the safety of all BHA funded drinking water sources are ensured before consumption. Outcome: To provide clean and safe water to participants by conducting a regular water quality test and by making the required treatment if required and through preventive water safety planning. School WASH X Target clients: Directly students and indirectly their families Intervention Details: teachers will be given a training of trainers training on CHAST and other School SBC techniques, the teachers form WASH clubs and cascade the training to IMPEL | Implementer-Led Evaluation and Learning 62 Annex B: List of Interventions Interventions Basic PSNP Ifaa Enhanced Summary of the intervention the club members and the club members further cascade the lessons to the rest of students through different schedules. Ifaa also expects a second-generation influence on overall hygiene and sanitation practices of the Kebele through Child—parent influence. Expected outcome: To improve hygiene and sanitation behavior or practices of the students by providing training on CHAST and conducting regular monitoring as it is a best channel to reach the community. WASH system strengthening X Target: Institutions—Woreda WASH Team (The WASH National program Structure), Private sectors, WASHCOs, Financial service providers. Intervention: The WASH systems strengthening approach helps RFSA to see where a failure in one or more of the building blocks is causing a failure in service delivery. By assessing the status of important WASH system building block and identify which of them has the greatest potential to improve the woredas current situation, and the linkages between them, RFSA can identify weak points and target their interventions for greater effect. The activities include training of relevant GOE WASH sector experts, updating the woreda strategic plan and preparing the woreda WASH road map. Outcome: Sustainable WASH Service delivery in target woreda and community. Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex C: List of Kebeles with Treatment and Livelihood Status 63 ANNEX C: LIST OF KEBELES WITH TREATMENT ASSIGNMENT AND LIVELIHOOD STATUS Table 12. Impact evaluation kebeles with treatment and livelihood status Woreda Kebele Treatment Livelihood Babile Abdibuchi Control X Babile Abdulqadir Control X Babile Bishan babile Control X Babile Erer guda Treated X Babile Gambela Treated X Babile Gemechu Control Babile Ibada gemechu Treated X Babile Ifa Control Babile Jalale Control Babile Lekolo Treated Babile Nejata gemechis Treated X Babile Shek husen Treated Babile Tofiq Treated Babile Tuluhoro Treated Chinaksan Amola Treated X Chinaksan Baduelemo Treated X Chinaksan Biftuu waree Treated Chinaksan Chelchale Control Chinaksan Dawe kora Treated X Chinaksan Dembesele Treated X Chinaksan Gela Treated Chinaksan Golewachu Control X Chinaksan Kaleroga Treated Chinaksan Kobobika Control Chinaksan Kocher Treated X Chinaksan Merer Treated X Chinaksan Migira Control IMPEL | Implementer-Led Evaluation and Learning 64 Annex C: List of Kebeles with Treatment and Livelihood Status Woreda Kebele Treatment Livelihood Chinaksan Mudi dawe Treated X Chinaksan Orda sost Treated Chinaksan Tiro gudoo Control Chinaksan Tirosendare Treated Chinaksan Ulanula Control X Chinaksan Wachuand Control X Chinaksan Wachuhulet Control Chinaksan Yugyug Control X Deder Burka bereka Control Deder Burka_geba Treated X Deder Cheka gemechu Treated X Deder Chela negeya Treated X Deder Gegewisa Treated Deder Golu Treated X Deder Hake bas Control X Deder Haremfemekuni Treated Deder Huffe Treated Deder Ifebas Control X Deder Kura deder Treated Deder Lemen welteha Treated Deder Mede jalela Control X Deder Mumicha Control Deder Nedi gelansedi Control Deder Oda kebena Treated X Deder Welteha gudina Treated X Deder Weltehageba Control X Fedis Bareda Control Fedis Bedatu Control Fedis Belina arba Control X Fedis Bid borra Treated Fedis Efitu dada Treated Baseline Study of the Ifaa RFSA in Ethiopia (Vol. II) Annex C: List of Kebeles with Treatment and Livelihood Status 65 Woreda Kebele Treatment Livelihood Fedis Ido baaso Treated X Fedis Kerensa lencho Treated X Fedis Kufa bobasa Control X Fedis Negaya bobasa Treated Fedis Risiki Treated Fedis Umer kule Treated X Gursum Abubeker sadik santala Control X Gursum Awdal Treated X Gursum Berite Treated Gursum Buna Control X Gursum Buyo negeya Treated Gursum Day feres Treated X Gursum Ebsa Control Gursum Elalemi Control X Gursum Gara wadaja Treated Gursum Gefire guda Treated Gursum Goro siyo Treated Gursum Harashi Treated X Gursum Hariro Control Gursum Kasa oromiya Treated X Gursum Kebso Treated Gursum Misira Control Gursum Negeya Treated Gursum Oda oromiya Treated X Gursum Saqabadii Control Jarso Afgug Treated X Jarso Ahamadhiroo Control X Jarso Amen Treated Jarso Aneno mite Control Jarso Bedesa Control X Jarso Burka mete Control IMPEL | Implementer-Led Evaluation and Learning 66 Annex C: List of Kebeles with Treatment and Livelihood Status Woreda Kebele Treatment Livelihood Jarso Chala Treated X Jarso Debub debelo Control Jarso Epa jalela Treated Jarso Gara abdula Treated X Jarso Gidiya licha Treated Jarso Melka jebdu Control X Jarso Oda muda Treated Melka belo Bifitu negeya Treated Melka belo Burika negeya Control X Melka belo Chefe jeneta Treated X Melka belo Chefe weliteha Control X Melka belo Daba kenisa Treated Melka belo Degaya belo Control X Melka belo Dire qufa Treated X Melka belo Fule negeya Control Melka belo Haka mulisi Control Melka belo Mulisa hakwa Treated Melka belo Tokuma bilisumu Control Melka belo Tokuman kane Treated Melka belo Welikituma bilusuma Treated X Midega tola Auriji Control X Midega tola Bilisuma Treated X Midega tola Biyo waraba Treated X Midega tola Ibiro musa Control Midega tola Kerensa Treated Midega tola Kufa Control X Midega tola Lencha Treated X Midega tola Mudibali Control X Midega tola Mukura Treated Midega tola Roba Treated X Midega tola Terkafeta Control