By Inclusive Development and Research Consultancy (IDRC) Mobile: +232 (076) 299645/ (077) 956841 Email: teddymorlai@gmail.com/ info@criid.org Commissioned by CARE International in Sierra Leone This document has been produced with the financial assistance of the USAID. However, the contents of the document are the sole responsibility of CARE in Sierra Leone; and can therefore under no circumstances be regarded as reflecting the position of USAID. It is also noteworthy that the findings herein serve as a guide for the RESSNER project intervention in nine (9) target chiefdoms in Bombali and Tonkolli Districts; and therefore are not for comparison with national data. However the report would be referenced where and when necessary. October 2017 EVALUATION REPORT Rapid Ebola Social Safety Net and Economic Recovery (RESSNER) Program i End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Table of Contents Table of Contents.........................................................................................................................................i List of Tables................................................................................................................................................i List of Figures..............................................................................................................................................ii Acknowledgement......................................................................................................................................iii List of Abbreviations and Acronyms .......................................................................................................iv Executive Summary....................................................................................................................................1 1 BACKGROUND OF EVALUATION ...................................................................................................1 1.1 Evaluation Purpose and Objectives.....................................................................................................1 1.2 Background/Context of the Project.....................................................................................................2 2 METHODS AND PROCEDURES.........................................................................................................4 2.1 Design of the Evaluation.....................................................................................................................4 2.2 Sources of Information and Evaluation Tools.....................................................................................4 2.3 Sampling Design .................................................................................................................................5 2.4 Data Analyzes and Reporting..............................................................................................................7 2.5 Implementation Strategy and Plan ......................................................................................................9 2.6 Limitations........................................................................................................................................10 3 EVALUATION FINDINGS AND DISCUSSIONS ............................................................................10 3.1 Program Impact (Achievement and Changes made by the Project)..................................................10 3.2 Procedures and Processes in Project Implementation .......................................................................24 3.3 Collaboration and Coordination........................................................................................................33 4 CONCLUSION AND RECOMMENDATIONS.................................................................................36 4.1 Recommendations.............................................................................................................................36 ANNEXES ....................................................................................................................................................i Annex I: Terms of Reference ....................................................................................................................ii Annex II: Checklist for USAID FFP Partners’ Recommended Questions................................................x Annex III: Personal Interviews Questionnaire..........................................................................................xi Annex IV: Focus Group Discussions Guide...........................................................................................xix Annex V: Key Informant Interviews Guide ...........................................................................................xxi Annex VI: List of Stakeholders Engaged .............................................................................................xxiii Annex VII: Bibliography......................................................................................................................xxiv List of Tables Table 1: Cluster design sampling selection procedures................................................................................6 Table 2: Actual number of communities and beneficiaries interviewed in the course of evaluation ...........7 Table 3 : Analyzes matrix for key indicators................................................................................................8 Table 4: Activities schedule for the study.....................................................................................................9 Table 5: Means test for significant different on expenditure ......................................................................15 Table 6: Changes in coping strategies adopted by vulnerable households before RESSNER project inception.........17 Table 7: Training received by both cash and seed voucher beneficiaries...................................................20 Table 8: Table showing results of multi-indicator correlation coefficient analyzes...................................21 Table 9 : Entent of achievement of purpose of the cash transfer intervention............................................22 ii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone List of Figures Figure 1: Selected districts for the evaluation............................................. Error! Bookmark not defined. Figure 2: Severity of hunger reported by household beneficiaries in Bombal and Tonklili districts .........12 Figure 3: Household beneficiaries who reportedly to have eaten at least 2 meals/ day..............................12 Figure 4: Dietary diversity score demonstrated by household beneficiaries across the intervention districts.............13 Figure 5: Percent change in dietary diversity among beneficiary households in the intervention districts 13 Figure 6: Changes in expenditure on food and non-food items..................................................................14 Figure 7: Responses to price changes during implementation period.........................................................15 Figure 8: Reduced coping strategy index for beneficiary households in RESSNER intervention districts 16 Figure 9: Percent distribution of coping strategies adopted by household beneficiaries............................17 Figure 10: Key areas of investment from cash received by household beneficiaries.................................18 Figure 11: Examples of beneficiary investment noted during field interviews ..........................................19 Figure 12: Access to training versus engagement in income generating activities.....................................20 Figure 13: Beneficiaries displaying payment vouchers during field interviews.........................................25 Figure 14: Satisfaction level expressed by beneficiary on the cash-based support ....................................28 Figure 15: Percent of most vulnerable persons interviewed in the evaluation............................................29 Figure 16: Gender distribution of registered beneficiaries by project component and survey data............30 Figure 17: Distribution of key economic activities by gender....................................................................30 Figure 18: Change in decision-making on use of cash resources and expenditure.....................................31 Figure 19: Economic decision making in both male- and female- headed households..............................32 iii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Acknowledgement The End-of-Project Evaluation was commissioned by CARE International in Sierra Leone to assess the extent to which the RESSNER project contributed to restoring food security, improve economic recovery and reducing the negative coping mechanisms of EVD affected participating households in Tonkolili and Bombali districts. The project was funded by USAID as part of their broader Cash-based Ebola Emergency Food Security Program. Many thanks to the Senior Project Management Team at the CARE Country Office for their invaluable support in ensuring that appropriate indicators and tools are developed in compliance with the harmonized tools/questions recommended by the Food for Peace (FFP) partners (including CARE, World Vision, Save the Children, CRS and SNAP/ACDI/VOCA). The Evaluation Team also appreciates the guidance from RESSNER field staff in both Bombali and Tonkolili districts to locate beneficiaries selected for interviews. The responses from USAID and other actors involved in the RESSNER project implementation including NaCSA, ACC, the Presidential Delivery Team (PDT), Local Councils and the Community Identification Committees were remarkable eye opener to the essence of community-driven approaches in local development interventions. Teddy Amara Morlai (M.Sc) Lead Consultant iv End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone List of Abbreviations and Acronyms AAR After Action Review ACC Anti-Corruption Commission CARE Cooperative for Assistance and Relief Everywhere CCT Conditional Cash Transfer CEFORD Community Empowerment for Rural Development CIC Community Identification Committee DDI Dietary Diversity Index EVD Ebola Virus Disease FAO Food and Agricultural Organization FCS Food Consumption Score FEWSNET Famine Early Warning System Network FFP Food for Peace FSP Financial Service Providers GRM Grievance Redress Mechanism GRN Goods Received Note HCES Household Consumption and Expenditure Survey HDDS Household Dietary Diversity Score HHS Household Hunger Scale KII Key Informant Interview MADAM Mankind Activities for Development Accreditation Movement MAFFS Ministry of Agriculture, Forestry and Food Security MOU Memorandum of Understanding NGO Non-Governmental Organization OPHI Oxford Poverty and Human Development Initiative PDM Post Distribution Monitoring PDT Presidential Delivery Team PI Personal Interview PMT Proxy Means Test PO Purchase Order PPP Purchasing Power Parity PR Purchase Request rCSI Reduced Coping Strategy Index RESSNER Rapid Ebola Social Safety Net and Economic Recovery RODA Rofuta Development Association ROSCA Rotating Savings and Credit Association SLIHS Sierra Leone Integrated Household Survey SLIHS Sierra Leone Integrated Household Survey SNAP Sustainable Nutrition and Agriculture Promotion SPSS Statistical Package for the Social Sciences SSN Social Safety Net STC Save the Children UCT Unconditional Cash Transfer UNICEF United Nations Children’s Fund USAID United States Agency for International Development VSLA Village Savings and Loan Association WFP World Food Programme 1 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Executive Summary This report presents findings from the end line and end-of-project performance evaluation of the Rapid Ebola Social Safety Net and Economic Recovery (RESSNER) Program. The RESSNER program was designed to ‘provide immediate access to cash for purchase of basic food items and support to local economic recovery through unconditional cash transfers in nine Ebola Virus Disease (EVD) affected chiefdoms across the districts of Bombali (5 chiefdoms) and Tonkolili (4 chiefdoms)’. Overall the program targeted and reached a total of 8,100 extremely poor households with unconditional cash transfers (implemented in both phases) and 900 poor households with a one-off seed voucher (implemented in the 2 nd phase only). The end line evaluation therefore covered these two phases of the program to document the extent to which the program contributed to restoring food security, improving economic recovery and reducing the negative coping mechanisms of EVD affected households in Bombali and Tonkolili districts in northern Sierra Leone. The evaluation targeted 587 household beneficiaries from 112 communities across nine chiefdoms using Probability Proportional-to-Size (PPS) systematic cluster sampling design. The sample size includes 526 cash transfer and 61 seed voucher household beneficiaries; and the primary unit of analysis was the household. Multi-indicator analyses using the Pearson’s Rho (product moment correlation coefficients), means analyses and percentages were done using both spreadsheet and the Statistical Package for the Social Sciences (SPSS) software. Visual presentations of findings were done using cross-tabulations, charts and pictures. Key Findings Findings from the evaluation undoubtedly revealed that the unconditional cash transfers (UCT) program is an excellent modality for humanitarian emergency response targeting poor and vulnerable households particularly in remote communities. This was evident from the following key findings: Program Impact (Achievement and Changes Made by the Project) The evaluation findings revealed that the RESSNER cash transfer program achieved its aim of providing support for immediate access to cash for basic food needs; and also resulted in other unexpected outcomes including economic independence, high dietary diversity and low hunger situation for extremely poor and vulnerable households. These achievements were evidenced from outcomes such as reduced negative coping strategies, high dietary diversity scores, low household hunger scale, expanded expenditures and improved economic activities (as sustainability strategies) among extremely poor households across communities in the two intervention districts. ▪ Household Hunger Scale Findings from analyses of Household Hunger Scale (HHS) revealed highly impressive low hunger situations across the two intervention districts. About 67 percent of all household beneficiaries interviewed reportedly experienced little or no hunger and 33 percent reported moderate hunger in the last 30 days preceding interviews. Further observed was that almost none of the household beneficiaries actually experienced severe hunger. Although no baseline value was available for comparison, this result indicates that the RESSNER program contributed to a larger extent, to improved access to food for about 67 percent of beneficiary households across the two intervention districts. In addition, a marked increase of 19 percent from the baseline figure was observed in terms of number of meals eaten per day; and this was statistically tested as a highly significant change at 95 percent confidence level. ▪ Household Dietary Diversity Score About 94 percent of all household beneficiaries interviewed demonstrated high dietary diversity, and this was consistent across the two intervention districts- Bombali (95.9%) and Tonkolili (93.0%). Comparatively, the proportion of household beneficiaries demonstrating high dietary diversity markedly increased by 53 percent- from a baseline value of 41 percent to endline value of 94 percent. These findings 2 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone better reflect on dietary quality, and can be attributed to program success on improved outcomes in areas such as birth weight, child anthropometrics, improved hemoglobin concentrations and caloric and protein adequacy among beneficiary households in the intervention districts. ▪ Expenditures on Food and Non-food Items The evaluation revealed that household beneficiaries are already engaged in income expansion across communities in the intervention districts. Notably the mean monthly expenditure increased by 59 percent from baseline figure to end line result, but mean monthly expenditure on food sharply fell by 9 percent, while expenditure on healthcare and schooling increased. Test of difference (taking cognizance of inflation rates) proved that the decrease in mean monthly expenditures on food and increase in expenditure on healthcare were statistically significant at 95 percent confidence level; while there was no significant difference in expenditure on education. The observed expanded expenditure demonstrates the significance of unconditional cash transfer to vulnerable households during humanitarian response; and proved that with increase in household income, expenditure on non-food items does not influence food consumption. The findings also concluded that expenditure was not influenced by exogenous factors such as price increase. About 76 percent of beneficiary households reported not experiencing significant price increases throughout the course of implementation. ▪ Reduced Coping Strategy Index The computed reduced coping strategy index (rCSI) was noted to be low (rCSI=11.2)1 for all beneficiary households. Comparatively, this was an encouraging result, showing a drop of 61.2 percent points below the WFP 2015 value for Northern Province (rCSI=18.3)2 . This low rSCI is unsurprising, and reinforces validity with the low Household Hunger Scale (HHS) demonstrated by beneficiary households across beneficiary communities in the intervention districts. In particular results from multi-indicator comparisons analyzed using the Pearson’s Rho (correlation coefficient) showed that Household Hunger Scale (HHS) has strong correlation with both Coping Strategy Index score (r=0.649, p<0.01) and Reduced Coping Strategy Index (r=0.508, p<0.01). This implies that increased hunger may have provoked some households to engage in some form of predictable negative coping reactions; and proved that the project’s contributions to increased household access to basic food led to marked reduction of negative coping mechanisms adopted during the Ebola outbreak. This was further validated by outcomes of item response analyzes, which revealed that the practice of consuming or selling of reserved grains for farming, borrowing money or food items, and begging of food items and/or money fell sharply at high level of significance. While reduction in these coping strategies was tested to be statistically significant, RESSNER Program specific targets set for these indicators were however not met- partly due to over-ambitious targets. For instance the project target of reducing ‘percent of beneficiary households reporting consumption or selling of reserved grains for farming’ from a baseline value of 80 percent to a target of 30 percent was not met, but there was marked drop by 50 percent. ▪ Economic and Livelihoods Activities Evolved from Project Intervention The findings revealed that most cash transfers and seed voucher household beneficiaries had reportedly engaged in some form of economic and livelihoods activities as a way of building resilience to shocks. In particular, cash transfer beneficiaries have reportedly used the cash received to invest in five major 1 rCSI=sum of frequency of occurrence(f) x university rating weight (w)= ∑fw; See calculation of rCSI in the USAID/WFP Field Methods Manual (p.17) available at: http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp211058.pdf 2 See WFP Mvam BULLETIN # 10: January 2015 for Bombali, Tonkolili & Koinadugu districts. Available at: http://documents.wfp.org/stellent/groups/public/documents/ena/wfp272359.pdf 3 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone economic activities - with agricultural inputs (78%), farm labor (59%) and trading/business (54%) receiving the greatest attention. Other economic activities noted were savings (37%) and animal rearing (19%). Interestingly about 79 percent of cash transfer beneficiaries interviewed claimed their investment will sustain them even when they have no access to cash-based assistance. Further observed was that training served as a powerful motivator for the sustainability strategies adopted by both cash and seed voucher beneficiaries. Accordingly, those beneficiaries (both cash and seed voucher beneficiaries) who received training are about two times more likely to be engaged in income generating activities than those who never received training; and this ratio was found to be consistent across main income generating activities reported. The programmatic achievement observed from the analyses was noted to be strongly connected with achievement of key outcomes and outputs targets set in the program design. About 87 percent of a combination of key outcomes and outputs targets (13 of 15 outcomes/output targets) was met. Meanwhile two of the 15 output/outcome targets captured in the evaluation were observably not achieved, partly due to overambitious targets and some other exogenous factors such as inflation. For instance, the target of the outcome indicator ‘percent of household beneficiaries reporting increased average income from base income (Le10,000)’ was not achieved, but the percent increase was tested to be statistically significant. Notably, the low rate of achievement of the average income threshold target was partly due to the high inflation rates from base period (February 2016) to current period (September 2017) 3 . However, household beneficiaries demonstrated potentials for improved ability to recover from the impacts of EVD. In particular 89 percent of household beneficiaries reported to have reinvested cash received in agricultural activities and 59 percent are engaged in savings activities- which will consequently increase their income and prepare them to respond to future shocks. Specific achievements of the project targets are presented in the summary table below. Summary table showing extent of achievement of project targets Design summary/Results Indicators Baseline values Target Endline/ PPE values z-value4 ρ-value Result Project Purpose: To reduce the negative coping strategies of extremely poor HH in EVD affected communities % of household beneficiaries reporting negative coping strategies (sale of household items, borrowing cash, absenting children from school). 80% 30% 40% 7.7 0.0001 ρ <0.05, highly significant change Intermediate Outcome1: Increased access to cash for purchase of basic food items for selected household beneficiaries % of household beneficiaries with increased number of meals per day as compared to baseline 26% 70% 84% 12.6 0.0001 ρ <0.05, highly significant change % of household beneficiaries who report satisfaction with regards to relevance, 0% 75% 91% 20.2 0.0000 ρ <0.05, highly significant change 3 Exchange rates were US$1.00=SLL4,027 (as at 29 Feb 2016) and US$1.00=SLL7,522.96 (as at 30 Sep 2017). See Oanda forex website: https://www.oanda.com/currency/travel-exchange-rates 4 See EpiTool for z-test available at: http://epitools.ausvet.com.au/content.php?page=z-test-2 4 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone timeliness and accountability of cash transfer % of household beneficiaries reporting increased consumption of food groups (consumption of more than 6 food groups) 41% 70% 94% 14.1 0.000 ρ <0.05, highly significant change % of household beneficiaries reporting increased average daily income from current base income (Le10,000) 33% 80% 66% 6.5 <0.0001 ρ <0.05, highly significant change % of total average daily cash spent by household beneficiaries on food items 47% 60% 59% 2.3 0.0203 ρ <0.05, significant change % of total average daily cash spent by household beneficiaries on non-food items (incl. health & education) 35% 30% 40% 1.0 0.3267 ρ >0.05, no significant change % of community members reporting increased knowledge of nutrition and food consumption patterns 13% 80% 88% 16.6 0.0000 ρ <0.05, highly significant change Output 1: 3,600 Registered HH beneficiaries are provided with unconditional cash grant # of HH in EVD-affected communities receiving cash disaggregated by age and sex (& disability) 3,600 HH (21,600) 100% 100% Number and value of cash distributions made by the end of the project USD972, 000 @US$30 /Mo USD9 72,000 @US$ 30/Mo USD972,000 @US$30/Mo Intermediate Outcome 2: Improved ability of HH to recover from impacts of EVD % of household beneficiaries reporting they have re￾engaged in agricultural activities 9% 80% 89% 17.7 0.0000 ρ <0.05, highly significant change # of household beneficiaries who are able to save seeds for the following planting season 0% 75% 95% 17.7 0.000 ρ <0.05, highly significant change % of beneficiary h/h engaged in savings activities (VSLAs/ROSCAs) 15% 40% 59% 8.4 0.0000 ρ <0.05, highly significant change Output 2.1: Basic agricultural inputs are provided for 900 poor farming HH # of household beneficiaries who received agriculture inputs for the next planting season 0 900 900 Output 2.2: 4,500 Household beneficiaries are provided with Nutrition and EVD prevention training and sensitization # of household beneficiaries who participated in the nutrition and EVD prevention training and sensitization 0 4,500 8,450 Budget Utilization vis-à-vis Achievements Overall, about US$4.55 million was allocated for the entire program implementation, and 96 percent (US$4.37 million) of the total grant was used to reach 100 percent of household beneficiary targets- that is 5 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 9,000 household beneficiaries (8,100 cash transfers and 900 seed voucher beneficiaries). At outcome level, this expenditure resulted in achieving 9 of 10 (90%) intermediate outcome targets, and had helped to improve access to highly diverse food groups for 94 percent of all household targets. Given an average of 6 members per household target (as noted in the project documents), this result has meant approximately US$82 of the grant was spent on each individual of the targeted households over the course of the program implementation to achieve food access by 94 percent of 54,000 members of vulnerable households targeted across the implementation areas. Further, modalities where observed to have been instituted during program implementation for efficient use of the budget, while optimizing achievement of outcome targets. For instance, the online payment of cash was discontinued in the second phase of the RESSNER Program implementation; and this resulted in a total savings of about US$34,000 transfer fees to financial service providers. The evaluation also observed that a participatory community-driven approach was used in the RESSNER program design and implementation in order to build a more transparent and accountable system that covers the prelisting, beneficiary identification and selection, and cash disbursement processes. At the beneficiary listing stage, Community Identification Committees (CIC) were reportedly formed and allowed to lead the pre-listing process using basic pre-defined criteria. This was followed by a validation exercise, where potential beneficiaries were further screened through cloud-based Open Data Kit (ODK) to select the final list of beneficiary households. This method was observed to have promoted local ownership, and ensured that the right beneficiaries were selected, and given the due amount of benefits. In an effort to reduce cost and at the same improve on effective cash disbursement; community-clustering was one of the mechanisms used. Meanwhile, initial modalities such as the online payment through Airtel vendors had some flaws. Further, beneficiaries presented mixed views about waiting time, walking distance and grievance redress. In particular, long waiting time (sometimes for a period of 24 hours) and distance (of about four miles on average) were reported as critical issues. In addition few beneficiaries (about 2%) reportedly missed out payments as a result of absenteeism due to unavoidable circumstance; and had reportedly waited for about eight weeks before backlogs were paid. However, this issue was notably resolved in the second phase of implementation by considering next-of-kin to registered beneficiaries using consent forms. The payment process was commended as transparent, and therefore beneficiaries reportedly had no pressure or had not experienced any harassment from CIC members, family members or local authorities. Also a two-way communication mechanism between beneficiaries and program team was observed to have been developed in the course of implementation; and this had allowed project staff and CICs to give constant feedback on emerging situations. This mechanism developed a strong feedback loop that ensured beneficiaries, the public and management were fully aware of upcoming activities. Collaboration and Coordination The RESSNER program was observed to have built strong synergy with existing systems and structures on Social Safety Net (SSN). This was enhanced by the strong involvement of the National Commission for Social Action (NaCSA)- the lead agency on social protection programs in Sierra Leone. Given that NaCSA maintains a database of all cash transfer beneficiary households (including those covered by NaCSA and other cash transfer partners), the direct involvement of the agency therefore addressed issues of parallel systems and structures, and had undoubtedly led to avoidance of duplication of efforts and waste of resources. For instance, during the process of chiefdom level selection and resource allocation at the start of the RESSNER Program, NaCSA used its database to map out sections and communities already targeted for Cash Transfers Programs done by other agencies and organizations (NaCSA, SNAP/ACDI/VOCA) operating in the two targeted districts. This mapping allowed for excluding areas already targeted by other 6 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone agencies/organizations in the selection process to avoid duplication of efforts, and waste of resources, whilst expanding coverage of vulnerable communities in the two intervention districts. Further observed was that collaborative partnership with Food for Peace (FFP) partners and other organizations implementing cash transfer has been carried out at both district and national levels through forums, meetings and working groups. This partnership has notably ensured smooth alignment; and therefore overlapping situations (in which interventions duplicate efforts) were duly avoided. The program was also observed to be appropriately streamlined with other interventions in the targeted communities, which in part influenced results and created an avenue for leveraging funds, complementarity and sustainability (at phase out). Cognizant of the evaluation results, recommendations have been made and require critical considerations in any future humanitarian and/or health emergency response. Recommendations ▪ Multi-sectoral meetings through district forums, after action reviews, working groups should always be core components in humanitarian and/or health emergency response. ▪ Considering local actors and related community structures as part of the cash transfers intervention proved to be a model of participatory monitoring approach, and must be replicated in any future emergency response. ▪ Two-way communication approach between the project team and beneficiaries in an emergency response program is imperative. This approach, which created strong feedback loop between the project team and other project participants (including beneficiaries), was observed to have solved the problems associated with long waiting time experienced by project participants in RESSNER Phase I. ▪ Developing and formalizing Consent Form for absenteeism by beneficiaries proved to be very effective in the RESSNER Program intervention, and must be observed as an important part of future cash-based programming. ▪ Mapping intervention areas by mobile network coverage and assessing the capacity of financial services providers (FSPs) are critical in cash-based interventions, and can avert unnecessary waste of time and resources. As observed in Phase I of the RESSNER Program intervention carrying out online payment proved difficult due to poor network coverage and lack of cash-out points- making the modality unnecessarily time-consuming and inefficient. Discontinuing the modality in the second phase resulted in savings of approximately US$39,000, while still achieving 90 percent of intermediate outcome targets within the implementation period. ▪ The seed voucher modality and organizing seed fairs where local seed varieties were sold and beneficiaries could choose among different crop varieties of their choice is an innovative idea that should be adopted in any seed multiplication program. ▪ Building synergy with existing community-based programs should be an important part of sustainability strategy in humanitarian and/or health emergency response programming. This can promote leveraging and complementarity during implementation, and may lead to continuity after phase out. ▪ The RESSNER Program intervention proved that targeting more female-headed households in cash￾based programming increases the chances of more women to make independent decisions over use of cash resources. It is therefore recommended that due considerations be made to target more vulnerable female-headed households in future cash-based emergency responses. This increases the chances of women to be more empowered in decision making- giving that it gives them the freedom to make decisions. ▪ Similar future cash transfer program must ensure that community clustering should be consciously done to reduce longer working distances to payment points. 2 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 1 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 1 BACKGROUND OF EVALUATION 1.1 Evaluation Purpose and Objectives 1.1.1 Evaluation Purpose, Audience and Objectives The purpose of the performance evaluation is to provide answers to a specific set of research questions developed to investigate the extent to which the Rapid Ebola Social Safety Net and Economic Recovery (RESSNER) project contributed to restoring food security, improve economic recovery and reducing the negative coping mechanisms of EVD affected participating households in Tonkolili and Bombali districts. The evaluation was also to done to determine the effectiveness and appropriateness of cash transfer as an approach towards responding to a health related emergency as an alternative to providing food items. The outcome of the evaluation will contribute to the body of knowledge concerning the use of direct cash transfersrather than food/commodity distribution as a recovery strategy following these types of extreme health emergencies that have widespread impacts on local and national economic and social structures. It will also document learning on CARE’s implementation approaches, and where appropriate, inform future programming of similar interventions. Knowledge of project performance and impact will be better deepened from analyses of effectiveness, impact, and sustainability of project performance and coordination. The evaluation will be a comprehensive overall assessment of the extent to which the project has achieved its intended objectives; and how efficient and sustainable the project was in converting resources into activities, outputs and outcomes and the changes it brought about for direct beneficiaries. The evaluation captured key responses to questions linked to the following intermediate results: ▪ Increased access to cash for the purchase of basic food items for selected households beneficiaries ▪ Improved ability of household beneficiaries to recover from the impact of the EVD ▪ Reduced negative coping strategies of 4,500 extremely poor households in EVD affected communities The main audiences expected to receive the evaluation findings include USAID, CARE Sierra Leone, CARE USA, NaCSA and other national partners (including FFP partners). The specific objectives of the project include the following: 1. Assess comparative achievements (to the baseline) and changes made by the project 2. Determine efficiency and effectiveness of project implementation. 3. Draw out lessons to be learnt- what worked well and what didn’t work so well, why and how they were mitigated 4. Assess the interaction and impact of the project on the various community and government bodies and stakeholders 5. Explore community’s acceptance, perceptions and attitudes towards the project. 2 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 1.1.2 Evaluation questions The evaluation was designed to generate and provide information to the expected audience. Based on the harmonized tools/questions recommended by the FFP partners, a comprehensive set of compact questions was developed. The key evaluation questions that guided the development of the tools are as follows: ▪ To what extent did the households that participated in the RESSNER project (as part of the USAID EFSP) achieve food security and improved resilience, as measured by the household hunger scale, household dietary diversity score, reduced coping strategy index and income and livelihoods? ▪ To what extent were the systems, processes and procedures used by the project for delivering cash transfers and seed vouchers to participating households effective, efficient and sustainable during the period of program implementation? ▪ To what extent did the collaboration and coordination with public and private sector stakeholders result in enhanced capacity to deliver cash transfers as a form of social protection to vulnerable households and contribute to effective delivery of cash assistance during the emergency and recovery phases? ▪ To what extent did the cash transfers have unintended (positive or negative) outcomes such as gender, inter- and intra- households and community and intra-community relations? 1.2 Background/Context of the Project CARE started a Rapid Ebola Social Safety Net and Economic Recovery (RESSNER) program with unconditional cash transfers to extremely poor households, including those who have been directly or indirectly affected by Ebola Virus Disease (EVD) in Bombali and Tonkolili districts in the northern part of Sierra Leone. The RESSNER program was particularly designed to ‘provide immediate access to cash for purchase of basic food items and to support local economic recovery through unconditional cash transfers in nine EVD affected chiefdoms across the districts of Bombali (5 chiefdoms) and Tonkolili (4 chiefdoms). The program was implemented in two phases. The first phase (RESSNER I) started in August 2015 and implementation ended in January 15, 2017, while the second phase (RESSNER II) was implemented over a period covering January 16, 2016 to December 2017. During Phase I, 4,500 extremely poor households (comprising of 27,000 individuals) were reached with unconditional cash transfer of US$30/household/month for 10 months and US$15 for the eleventh month of implementation across the nine chiefdoms targeted in Bombali and Tonkolili districts. According to various project monitoring and evaluation reports, the RESSNER cash transfers intervention assisted in preventing negative coping strategies of 4,500 extremely poor households and allowed for the consumption of a diversity of food options during the Phase I period. However, some critical issues were also observed. For instance, the high inflation rate during this period adversely affected the budget of poor households, and significantly reduced their purchasing power due to lack of assets and low casual labor opportunities to increase their incomes- which in combination resulted in expenditure gaps of poor households. As evident in the September 2016 post distribution monitoring (PDM) report, beneficiary households reported an average monthly expenditure of Le183,000 - which is 41% more than the total cash support (Le130,000)- on nutritious food alone. As a result 20 percent of the total number of project beneficiaries targeted in Phase I was noted to be in dire need of further support; and communities that did not receive 3 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone any economic recovery support continued to face more challenges to recover and were also highly recommended for support. Hence Phase II was a cost extension period recommended for additional support to non-project communities and project communities (that needed further support) in Bombali and Tonkolili districts. Phase II of RESSNER program was also used to further help communities overcome the challenge of food insecurity and resume normal farm activities. Considering that the EVD outbreak exhausted poor households’ seeds and that lack of income hindered the ability of households to purchase seeds to resume farm activities, this extended period supported these poor households with seeds. Overall, the RESSNER Program supported 4,500 and 3,600 extremely poor households with unconditional cash transfer of US$30/household/month, in Phase I and Phase II respectively. Meanwhile 533 of total households supported in Phase II were those households targeted in Phase I but further showed signs of vulnerability. Also one-off conditional seed voucher support of US$30 and training on improved agronomic practices and crop diversification were provided to 900 farming households. Further included in the program were community trainings and sensitizations on Ebola-related health and nutrition, sensitization and hygiene for the entire 4,500 households targeted in Phase II. The RESSNER Program strongly aligned with the National Ebola Recovery Strategy for Sierra Leone, developed as a result of the Ebola Epidemic that affected the nation’s socio-economic environment, and therefore further impacted the development activities outlined in the Agenda for Prosperity (2013-2018). This recovery strategy was designed to help the country transition seamlessly from Ebola response to economic recovery. In particular the RESSNER Program fell under the social protection sector of the four (4) Early Recovery Priority sectors including, (i) restoring basic health, (ii) returning children to school safely, (iii) protecting vulnerable population and (iv) assisting private sector recovery5 . This means, the RESSNER Program contributed to government efforts in providing ‘...immediate relief to vulnerable households through income support...6 .’ during the implementation of the National Ebola Recovery Strategy in the recovery phase. The program design also links with the United States government’s (USG) Ebola response strategy implemented through the United States Agency for International Development/Food for Peace (USAID/FFP) emergency projects in Sierra Leone. Notably, the USG Ebola response strategy had four pillars including i) controlling the epidemic, ii) mitigating the secondary consequences of the EVD outbreak, iii) building a coordinated and broader global response and iv) strengthening global health security infrastructure7 . In Sierra Leone the FFP largely supported the first and second pillars of the strategy. While this report focuses on the USAID/FFP cash transfer programming (CTP), it was noted that USAID/FFP-funded cash transfer projects in Sierra Leone were implemented by five (5) FFP partners89 to ‘address the food security impacts of the response to 5 See President’s Recovery Priorities, available at: http://www.presidentsrecoverypriorities.gov.sl/the-early-recovery 6 Ibid 7 See ‘An Unprecedented Response to the Ebola Crisis’, available at: https://blog.usaid.gov/2014/09/an-unprecedented￾response-to-the-ebola-crisis 8 USAID/FFP Partners include CARE International, CRS, Save the Children, World Vision and SNAP/ACDI/VOCA 9 Interviews with Lovetta Sannoh, Project Management Specialist, USAID in Sierra Leone 4 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Ebola in ways that support and promote recovery’. Specifically, USAID/FFP funded CARE International with US$4.55 million which was used to support 8,100 extremely poor households during 2015-17 through unconditional cash transfer (UCT) and 900 poor households with a one￾off seed voucher in Bombali and Tonkolili districts. The progam had a mix of delivery mechanism including use of vouchers, mobile phones and direct cash delivery. 2 METHODS AND PROCEDURES 2.1 Design of the Evaluation The design of the evaluation was highly exploratory through extensive use of qualitative tools including KII and FGD guides. However, considering that the evaluation type were both impact￾and performance-based, descriptive research design techniques were used to carry out impact analyses and target-actual comparisons. While key focus was placed on evaluation questions asked in the TOR, some flexibility was made in terms of compact questions formulated for discussions with target stakeholders. Such considerations were made due to the different level of respondents who were engaged through in-depth interview technique. 2.2 Sources of Information and Evaluation Tools Information gathering was done in two phases during the course of the evaluation exercise- the ‘Desk Review Phase’ and ‘Field Interview Phase’. 2.2.1 The Desk Review Phase Desk review was done to generate project information from CARE, partners, as well as relevant literature that would be used to triangulate some findings to the general framework of the evaluation. Documents reviewed were project-related documents (including revised proposals, After Action Review documents, project budgets, M and E documents, baseline assessment report, logframe and Post Distribution Monitoring reports) and USAID and other related documents for consistency in approaches on sampling, analyses and reporting. Related documents reviewed from trusted websites include the following (also noted as references): ▪ The Coping Strategies Index: A tool for rapid measurement of household food security and the impact of food aid programs in humanitarian emergencies ▪ (FANTA III, USAID, 1999): Sampling Guide (Addendum) ▪ (FANTA, 1999): Food Security Indicators and Framework for Use in the Monitoring and Evaluation of Food Aid Programs ▪ (USAID, 2016): USAID’s Office of Food for Peace Policy and Guidance for Monitoring, Evaluation, and Reporting for Development Food Security Activities ▪ (GoSL, 2009): National Draft Framework for Social Protection in Sierra Leone ▪ (GoSL): President’s Recovery Priority ▪ (FANTA III, USAID, FAO, Tufts University, 2011): Household Hunger Scale: Indicator Definition and Measurement Guide ▪ (USAID, 2013): Manual for Implementation of USAID Poverty Assessment Tools ▪ USAID Office of Food for Peace, 2016): 2016-2025 Food Assistance and Food Security Strategy ▪ (Mediterranean Journal of Social Science, 2014): Food Insecure Household Coping Strategies: The Case of a Low Income Neighborhood in South Africa ▪ (USAID, FANTA III, FEWS NET, 2015): Comparing Household Food Consumption Indicators to Inform Acute Food Insecurity Phase Classification ▪ (IFPRI Discussion Paper, 2012): The Women Empowerment in Agriculture Index 5 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone ▪ (USAID,2015): Baseline Study for the Title II Development Food Assistance Programs in Zimbabwe 2.2.2 Field Interview Phase The field interview phase was used to gather primary information during the evaluation exercise. At the field level, various types of evaluation tools were used to generate both qualitative and quantitative information from respondents’ interviews. The tools used to solicit information (considering the varying levels of participation during project implementation) include: ▪ Focus Group Discussion (FGD) guide administered to both cash and non-cash beneficiary households, ▪ Key Informant Interview (KII) guide for in-depth interviews with key informants such as project staff and partners and other stakeholders (See annex IV for stakeholders interviewed), ▪ Personal interview (PI) questionnaires for beneficiary impact assessment at household level, and ▪ Case studies guide for most significant change interviews with beneficiaries. 2.3 Sampling Design 2.3.1 Sampling Selection Procedures Figure 1: Selected districts for the evaluation The sampling design adopted for the evaluation was the probability-proportional-to-size (PPS) sample selection using two-stage cluster sampling technique. However, this evaluation targeted households that directly benefited from the RESSNER Program and therefore households were used as units of analysis, while chiefdoms and beneficiary communities were considered as clusters. It is however noteworthy that the scope of work covered the two RESSNER Program intervention districts namely, Bombali and Tonkolili districts (See Figure 1). The first stage was the selection of communities from a list of all beneficiary communities targeted by the RESSNER project. At the second stage, beneficiary households were selected from each of the clusters selected during the first stage. The sample frame (or universe) used was a total of 9,000 vulnerable households (8,100 cash transfer beneficiaries and 900 seed voucher beneficiaries) targeted by the RESSNER Program. The desired sample was calculated for each of the two phases using equation (i) below: n=z2pqN/(z2 pq+Ne2 )------------------------------------------------- (i) Where n=the desired minimum sample size z= the value of the standard normal deviation corresponding to the level of confidence 6 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone p= the estimated population experiencing positive outcome measures (proportion of vulnerable households experiencing improved food access measured in terms of HDDS or HHS) q= 1-p= estimated population experiencing negative outcome measures (proportion of vulnerable households who lack improved access to food measured in terms of HDDS or HHS) N= total population targeted for the targeted by the RESSNER Program e= desired minimum level of precision/accuracy Assumptions: ✓z=1.96 (for 95% confidence level assumed for the study) ✓Beneficiaries who lack improved access to food (measured in terms of HDDS as estimated from baseline study)= 59%10 ✓Level of precision is 5% at 95 percent confidence level ✓10% added for refusal or incomplete responses ✓Design effect (required for 2-stage cluster sampling) ✓Design effect of 1.5 is applied based on various studies as suggested in WFP food security report Substituting in (i) above, the desired minimum sample size n= 357 Considering that a cluster design was used for the proposed evaluation, the cluster design sample size=DEFF x Simple Random Sample (SRS) size as presented in the assumption above DEFF=1.5. Notably, the program covered four categories of households11 reaching a total of 9,000 household beneficiaries. The desired sample size for cluster design was computed as 589 for representativeness across categories, and this included 10 percent addition for refusal, exclusion due to incomplete responses and/or inaccessibility of target communities. However, the actual number of household beneficiaries interviewed was 587 due to low turnout of beneficiaries in some communities during interviews, and counseling of some communities which were not easily reached due to difficult terrain. Meanwhile, this did not affect the overall purpose of the evaluation, given that the total number of household beneficiaries interviewed exceeded the computed cluster design sample size. Table 1 presents the targeted number of households beneficiaries sampled per category of beneficiaries. Table 1: Cluster design sampling selection procedures Phase Sample frame (N) Proportion of total population Desired sample size (n) Computed cluster design sample size (deff*SRS) 10% addition for non￾response Target # of HHs Phase I 4,500 50% 178 268 27 295 Phase II* 533 6% 21 32 3 35 Phase II 3,067 34% 121 182 18 201 Seed Voucher 900 10% 54 54 5 59 Total 9,000 100% 357 536 53 589 *Phase I beneficiaries with extended cash benefits 10 Estimate from baseline study indicate that only 41% of household beneficiaries at 6 food groups or more (high HDDS) 11 This included 4,100 HHs targeted for cash transfers in Phase I, 533 HH beneficiaries from Phase I with extended cash benefits in Phase II, 3,067 new HHs targeted for cash transfers in Phase II and 900 HHs targeted for Seed Vouchers 7 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 2.3.2 Calculation of Expected Number of Clusters Given that that a list of household beneficiaries already exists for all clusters (villages), a Systematic Probability Proportion to Size (PPS) cluster sampling selection technique was used to select both clusters and beneficiary households for the personal interviews. This selection procedure was done from a database of beneficiary communities and households for all categories of household beneficiaries targeted in RESSNER Phases 1 and II. To minimize intra-cluster correlation (or homogeneity in responses) and therefore cover more communities, 5 households were initially targeted per cluster. Hence using a sampling interval of 5 households, the PPS cluster sampling technique resulted in the selection of 112 clusters/communities. Table 2 presents the actual number of communities and households covered during field interviews following procedures used in the PPS cluster sampling selection procedure. The selection showed representativeness across the nine chiefdoms targeted in both Bombali and Tonkolili districts. However, there were slight deviations such as decrease in the number of targeted beneficiaries during field interviews. As inter alia noted this slight decrease was due to some changes in sampled communities which could not be reached as a result of difficult terrain. Other accessible communities selected (as replacement) during field interviews had less, while others have more than the required minimum size targeted per community- but overall, a non-response rate of 2 households (0.3% of total sample size) was observed. Table 2: Actual number of communities and beneficiaries interviewed in the course of evaluation Chiefdom # of communities covered # of beneficiaries interviewed # of cash transfer beneficiaries # of seed voucher beneficiaries Bombali District Safroko Limba 7 50 50 0 Paki Masabong 6 45 45 0 Makari Gbanti 6 25 25 0 Leibeysaygahun 11 61 41 20 Biriwa 17 65 65 0 Sub-total 47 246 226 20 Tonkolili District Konike Sanda 9 45 45 0 Bonkolenke 18 113 84 29 Kholifa Mabang 14 66 66 0 Konike Barina 24 117 105 12 Sub-total 65 341 300 41 Grand Total 112 587 526 61 2.4 Data Analyzes and Reporting Various analytical techniques were used to better understand the extent to which the RESSNER project activities resulted in improved food security and its impact on household beneficiaries in the target districts. It is noteworthy that content analysis was particularly expected to be used for responses to qualitative questions in the designed tools vis-à-vis KIIs, FGDs and open-ended questions in the PIs. For descriptive analyses special caution was taken on the analyses and validation process. In particular careful considerations were made to analyze findings on a) Household Dietary Diversity Score (HDDS), b) Household Hunger Scale (HHS), c) reduced coping strategies index, and d) Expenditure on food and non￾food items. Such caution was required given that the FFP partners (with approval from USAID) agreed on harmonized tools and questions in the end line evaluations for the Emergency Food Security Program (EFSP) in targeted districts. Table 3 presents the analyses matrix for these indicators (including financial asset possession analyses as an addendum): 8 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Table 3 : Analyzes matrix for key indicators Measure Indicators Formula needed/descriptions Household Dietary Diversity Score (HDDS) analyses 12 food groups recommended for analyses include: 1) cereals, 2) roots and tubers, 3) vegetables, 4) fruits, 5) meat and poultry, 6) eggs, 7) fish and sea food, 8) pulses/legumes/nuts, 9) milk and milk products, 10) oil/fats, 11) sugar/ honey, 12) miscellaneous Total HDDS (0-12)= ∑(n1+n2+…n12) Where ‘n’ is a designated food group assigned values 1 (consumed) and 0 (not consumed) over a recall period of seven days. Average HDDS= ∑(HDDS)/total number of households The scores of the total HDDS were grouped into three and interpreted as: i) high dietary diversity (for ≥6 food groups consumed), ii) medium dietary diversity (for 4&5 food groups) and iii) lowest dietary diversity (for ≤3 food groups consumed). House Hunger Scale (HHS) analyses12 9 Household Hunger Scale (HHS) items are recommended for a recall period of 4 weeks (30 days) 1. Worry that the household would not have enough food 2. Not able to eat the kinds of food preferred 3. Eat a limited variety of foods 4. Eat some foods that you really did not want to eat 5. Eat smaller meal than you felt you needed 6. Eat fewer meals in a day 7. No food to eat of any kind in your household 8. Go to sleep at night hungry 9. Go a whole day and night without food Analyzes will be done using direct response scale of [1] yes, [0] No, and frequency scale of [1] Rarely (1-2 times), [2] sometimes (3-10 times) and [3] Often (more than 10 times), using appropriate recoding techniques, two different cut-off values (>1 and >3) will be applied to define the household hunger scale in three score categories: 0-1=Little to no hunger in the household 2-3=Moderate hunger in the household 4-6=Severe hunger in the household Reduced Coping Strategies Index (rCSI) analysis 5 standard coping strategies and their associated severity rating are recommended to analyze comparative (reduced) coping strategy index (rCSI): ▪ Eating less-preferred foods (1.0 severity weight) ▪ Borrowing food/money from friends and relatives (2.0 severity weight ▪ Limiting portions of mealtime (1.0 severity weight) ▪ Limiting adult intake (3.0 severity weight), and ▪ Reducing the number of meals per day (1.0 severity weight) The general Coping Strategy Index (CSI) recommended by FFP partners for a harmonized tool include the following indicators: 1. Limit portion sizes at meal times 2. Reduce the number of meals per day 3. Skip entire days without eating 4. Borrow food or rely on help from a friend or relative 5. Rely on less expensive or less preferred foods 6. Purchase food on credit/ take a loan to buy food 7. Gather unusual types / amounts of wild food or hunt 8. Harvest immature crops 9. Send household members elsewhere 10. Send household members to beg 11. Restrict consumption by adults so children can eat more 12. Rely on casual labor for food The reduced coping strategy index is derived iteratively as follows: 1. Weighted score=frequency of occurrence x universal severity weight= fw 2. rCSI=∑fw for each individual household Weights recommended for each score as frequency ranging from [1] Never [2] seldom (1 day a week) [3] Once in a while (1-2 days per week) [4] Pretty often (3- 6 days per week), [5] Almost every day. The item response scales recommended are as follows: Item [1] [2] [3] [4] [5] 1. 2.3 5.6 6.9 9.2 11.5 2. 2.9 5.8 8.7 11.6 15.5 3. 3.7 7.5 11.1 15.6 18.5 4. 2.3 5.6 6.9 9.2 11.5 5. 3.0 6.0 9.0 12.0 15.0 6. 2.3 5.6 6.9 9.2 11.5 7. 3.1 6.2 9.3 12.5 15.5 8. 3.1 6.2 9.3 12.5 15.5 9. 3.0 6.0 9.0 12.0 15.0 10. 3.5 7.0 10.5 15.0 17.5 11. 3.1 6.2 9.3 12.5 15.5 12. 2.3 5.6 6.9 9.2 11.5 Income and expenditure analysis The following indicators are required for income and expenditure analyses: ▪ % share of total expenditure on food ▪ % share of total expenditure on healthcare ▪ % share of total expenditure on education Mean analyses and comparisons were done for all indicators 12 Household Hunger Scale (HHS) was a new indicator recommended by the USAID/FFP working group, but was not captured in the baseline 9 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 13Financial Asset possession analysis Access to financial assets will be computed using the following indicators: ▪ Ownership of agricultural land ▪ Ownership of commercial motorcycle ▪ Ownership of work oxen ▪ Ownership of cattle ▪ Ownership of sheep, goat and/or pig ▪ Ownership of poultry ▪ Ownership of power tiller/tractors ▪ Ownership of rice mill or other food processing machine ▪ Ownership of building for rental For each household ‘h’, the following asset index was constructed: Eh = w1·Ih1 + w2·Ih2 + … + wk·Ihk Where Ih1 indicates the ownership of asset ‘1’ in household ‘h’, Ih2 indicates the ownership of asset ‘2’ in household ‘h’, and so on. These indices took the form of binary values of 1 when household ‘h’ owns the corresponding asset and 0 otherwise. Finally, w1, w2…wk are the weights associated to the different assets. Households with scores below the 33rd percentile are regarded as asset-poor. Other analytical procedures included means analyses, frequencies and cross-tabulations, charts, etc. through SPSS and Excel spreadsheet. Specifically, multi-indicator comparisons were made using Pearson’s correlation (r) for all key indicators including CSI scores, rCSI, HDDS, HHS, expenditure and Livelihoods activities. The key pointers here are the coping strategy indices—i.e. whether other indicators are highly correlated with the types of coping/survival strategies adopted. Reporting on findings was guided by a routine theme of evaluation guidelines that was set for the proposed study. However, the key targets of the evaluation were the household beneficiaries. Therefore discussions of findings largely centered on households as ‘unit of analyses’. 2.5 Implementation Strategy and Plan The interviews were conducted vertically- from project offices, implementing partners’ offices, district levels and community levels. The entire process of the study was completed within 30 calendar days. Table 4 presents the activity schedule of the study. Table 4: Activities schedule for the study Due Date Activity 25 September 2017 Review all project documents and reports 27 September 2017 Submission of inception report which includes survey work plan, data collection tools/questionnaire, sampling determination, logistics, etc. 27 September 2017 Review of inception report by CARE and/or USAID-FFP and Feedback to Consultant on the inception report 27 September 2017 Evaluation team incorporates comments and submit final inception report and evaluation tools 28-30 September 2017 Discussion with CARE Project team, Development of Tools and digitization, Field preparation 2-3 October 2017 Training/orientation of enumerators 5-9 October 2017 Field work/data collection and analysis 11-15 October 2017 Data cleaning and Analysis 18 October 2017 Presentation of preliminary findings with CARE RESSNER staff and management; USAID-FFP and other relevant partners 23 October 2017 Submission of draft report for internal review 24 October 2017 CARE provides feedback on draft report for incorporation 13 Assets indicators were captured in the baseline, but the endline captured these indicators only for the purpose of gender analyses on asset possession 10 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 27 October 2017 Submission of final report to CARE 31 October 2017 USAID provides feedback on the report 15 November 2017 Submission of Final Evaluation report 2.6 Limitations It is noteworthy that the RESSNER Program design underscores organizing joint meetings for Food for Peace (FFP) partners as a means to jointly evaluate the Emergency Food Security Program (EFSP), share best practices and delineate failures and remedial actions. This strategy has been successfully followed, and therefore led to harmonization of tools/questions for end line evaluation of all cash and non-cash assistance under the USAID-EFSP in Sierra Leone. While some of the recommended indicators are new, and were not included in the baseline, it is noteworthy that some findings using these new indicators could not be compared to the baseline. However the new indicators were regarded as an addition to the RESSNER￾specific indictors; hence the performance indicators developed specifically for the RESSNER program evaluation were appropriately used to make target-actual comparisons. Gathering information on caloric intake by households required extra time for training and standardization of caloric content of food being consumed. Enumerators needed ample time to train in collecting accurate and standardized data on caloric intake and anthropometry. However, time was a major constraint in the evaluation process so in lieu of caloric data, the household dietary diversity score (HDDS) was used to indicate the nutritional value of food being consumed by the households. Using the probability proportional-to-size (PPS) systematic cluster sampling proved to be the most unbiased selection method adopted in the methodology. However, achieving equal number of beneficiary households proved difficult, given that a number of selected communities had less than the expected number of beneficiary households per cluster (5 per cluster). This resulted in a reduced number of total beneficiary households expected to be interviewed. However, the sample size is large enough and was statistically significant for statistical inferences. 3 EVALUATION FINDINGS AND DISCUSSIONS Discussions of evaluation findings followed guidelines and topics shared by the FFP partners for standardization of questions to enhance comparison across intervention districts by all partners. Comparison with baseline values are only made for indicators captured in both assessments. Otherwise, only general impacts are discussed based on the endline evaluation findings. 3.1 Program Impact (Achievement and Changes made by the Project) This section discusses the RESSNER Program contribution towards restoring food security, improving economic recovery and reducing the negative coping strategies. It presents quantitative findings of the project impact and discusses (where necessary) comparison of endline results with the baseline findings and/or national food security assessment findings or related data. Discussions particularly focus on Household Hunger Scale (HHS), Household Dietary Diversity Scores (HDDS), expenditure on food and non-food items, economic and livelihoods activities of household beneficiaries and reduced Coping Strategy Index (rCSI). Further, multi-indicator comparison was made using the Pearson’s correlation (r) to 11 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone compare how various indicators are related to each other and the strength of relationship particularly between other indicators and the CSI/rCSI. Also discussed under this section is achievement made with regard to specific targets set to achieve the overall objective of the RESSNER Program. 3.1.1 Household Hunger Scale and Dietary Diversity Score Household Hunger Scale (HHS) and Household Dietary Diversity Score (HDDS) are regarded as proxy indicators for food insecurity measures. Notably, the RESSNER Program was primarily designed ‘to provide immediate access to cash for purchase of basic food items...’ The HHS and HDDS have therefore been useful proxy measures that delineate the RESSNER Program contributions towards restoring food security in the two intervention districts. While the USAID FFP Strategy underpins three variables (including availability, access and utilization) to define food and nutrition security14, HSS and HHDS served two distinct purposes in the context of the endline evaluation. The HHS is better placed to assess project contribution towards improving food access (with focus on quantity dimension), while HDDS examines the patterns of utilization (including improved access to nutritious food). The Household Hunger Scale (HHS) forms part of the Household Food Insecurity Access Scale (HFIAS) that combines 9 indicators to measure food insecurity in a comparable way. However, it has been widely argued that comparability across different countries and settings using HFIAS always proves to be difficult. HHS has therefore been recommended as a simple indicator to measure household hunger in food insecure settings; and it is considered to be universally acceptable indicator for cross-cultural use. Even though the 9 indicators of HFIAS were captured in the evaluation tool, analyses of HHS had only considered the three recommended indicators15 for computation of the score using a recall period of the last 30 days preceding interviews. The three indicators used to analyze HHS include: i) household members had no food to eat of any kind, ii) household members go to sleep at night hungry, and iii) household members go a whole day without eating anything. The scales 0-1 day (little or no hunger in household), 2-3 days (moderate hunger in household) and 4-6 days (severe hunger in household) were used to analyze household hunger scale for the two intervention districts. The endline evaluation findings revealed impressive low hunger situations across the two intervention districts. About 67 percent of all household beneficiaries (including both cash and non-cash beneficiaries) reportedly experienced little or no hunger in the last 30 days preceding interviews. Further observed was that almost none of the household beneficiaries had actually experienced severe hunger. Although no baseline value was available for comparison, this result indicates a significantly improved situation where about 67% of household beneficiaries in the intervention districts have improved access to food. District-wise, food access was observed to be high among household beneficiaries in Bombali district (80.5%) than in Tonkolili district (57.5%). This difference is unsurprising as national data on food insecurity proved Bombali district better off than Tonkolili district.16 14 See p. 5 of the USAID ‘2016-2025 Food Assistance and Food Security Strategy’. 15 See P. 6 of USAID/FAO/EU 2011 note: ‘Household Hunger Scale: Definition and Measurement Guide’ 16 See WFP, GoSL and unicef 2011 report on ‘The State of Food Security and Nutrition in Sierra Leone’. Available at: http://documents.wfp.org/stellent/groups/public/documents/ena/wfp288316.pdf 12 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Figure 1: Severity of hunger reported by household beneficiaries in Bombal and Tonklili districts The findings further revealed high improvement in number of meals consumed by household beneficiaries in the program intervention areas. As depicted in Figure 3, the proportion of household beneficiaries who reportedly ate two or more meals in the last 24 hours preceding the interviews was observed to have markedly increased by 19 percent from baseline figure; and this was tested to be a statistically significant improvement at 95 percent confidence level. The two-tailed z-test was used through the EpiTool17 for the statistical test of significance on observed changes. Figure 2: Household beneficiaries who reportedly have eaten at least 2 meals/ day Household Dietary Diversity Score (HDDS) is also a proxy measure of household food access, but more so, improved consumption for the purpose of the endline evaluation. While measures on malnutrition were not captured in the evaluation, findings on HDDS better reflect on dietary quality, and can be associated with improved outcomes in areas such as birth weight, child anthropometrics, improved hemoglobin 17 EpiTool for z-test available at: http://epitools.ausvet.com.au/content.php?page=z-test-2 67.1% 80.5% 57.5% 32.7% 19.1% 42.5% 0.2% 0.4% 0.0% 0% 20% 40% 60% 80% 100% All households (587) Bombali (n=246) Tonkolili (n=341) % of HH beneficiaries Districts Little or no hunger in household (0-1 day) Moderate hunger in household (2-3 days) Severe hunger in household (4-6 days) 0% 20% 40% 60% 80% 100% Baseline EOP % Change 65% 84% 19% % of HH beneficiaries Project timeline z-value= 4.6 ρ-value<0.0001 α-value= 0.05 Result: significant change 13 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone concentrations, and caloric and protein adequacy. Over 94 percent of all RESSNER household beneficiaries interviewed demonstrated high dietary diversity, and this showed consistency across the two intervention districts (see Figure 4). Figure 3: Dietary diversity score demonstrated by household beneficiaries across the intervention districts The proportion of household beneficiaries demonstrating high dietary diversity markedly increased by 53 percent- from a baseline value of 41 percent to endline value of 94 percent. Z-test showed that this is a statistically significant result (tested at 95% level of confidence). (See Figure 5) Figure 4: Percent change in dietary diversity among beneficiary households in the intervention districts z-value= 14.1 ρ-value=0.000 α-value= 0.05 Result: significant change 0.2% 0.4% 0.0% 5.6% 3.7% 7.0% 94.2% 95.9% 93.0% 0% 20% 40% 60% 80% 100% Total (587) Bombali (n=246) Tonkolili (341) % of HH beneficiaries District Diet diversity score Lowest dietary diversity ( ≤ 3 food groups) Medium dietary diversity (4 & 5 food groups) High dietary diversity (≥ 6 food groups) 0% 50% 100% Baseline Endline % change 41% 94% 53% % of HH beneficiaries Project timeline 14 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 3.1.2 Expenditures on Food and Non-food Items Income consumption was computed to compare patterns of expenditure on food and non-food items using local currency and also purchasing power (in US$) taking cognizance of inflation and difference in means. Computation of income consumption using local currency18 (Sierra Leone Leone) revealed that beneficiary households are already engaged in income expansion across communities in the intervention districts. As presented in Figure 6, mean monthly expenditure (computed in local currency) increased by 59 percent from base year to current year, but mean monthly expenditure on food sharply fell by 9 percent while expenditure on health care and schooling increased. (Also see Table 5) Figure 5: Changes in expenditure on food and non-food items Analyzes of test of differences in mean monthly expenditure were done in foreign currency (US$)19 taking note of months both assessments were done20 . The test of difference proved that the decrease in mean 18 Computation was directly done to determine the dynamics in expenditure, and therefore inflation was not considered. But test of difference in means gives actual purchasing power using US$ as shown in Table 5 19 Exchange rates were US$1.00=SLL4,027 (as at 29 Feb 2016) and US$1.00=SLL7,522.96 (as at 30 Sep 2017). See Ouanda forex website: https://www.oanda.com/currency/travel-exchange-rates 20 RESSNER Baseline Assessment was done in February 2016 and Endline was done in September 2017 0% 10% 20% 30% 40% 50% 60% 70% Baseline Endline Baseline Endline Baseline Endline Change in total expenditure Food expenditure Health expenditure Education expenditure 59% 68% 59% 15% 21% 17% 19% % of HH Beneficiaries HH expenditure across project timeline The 5 Reduced Coping Strategies with associated Severity Weighting ▪ Eating less-preferred foods (1.0 severity weight) ▪ Borrowing food/money from friends and relatives (2.0 severity weight ▪ Limiting portions of mealtime (1.0 severity weight) ▪ Limiting adult intake (3.0 severity weight), and ▪ Reducing the number of meals per day (1.0 severity weight) 15 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone monthly expenditures on food and increase in expenditure on healthcare were statistically significant at 95 percent confidence level; but there was no significant difference in expenditure on education (see Table 5). Table 5: Means test for significant different on expenditure Baseline (N=111) Endline/EOP (N=526) 95% confidence intervals Expenditure category Mean St. Dev. Mean St. Dev. z-value Upper Lower Result Interpretation Total monthly expenditure $ 45.19 $ 24.09 $ 38.44 $ 33.27 2.4927 12.0573 1.4427 P=0.0127 Significantly different Monthly expenditure on food $ 30.91 $ 19.60 $ 22.75 $ 19.64 3.9844 12.174 4.146 p<0.0001 Significantly different Monthly expenditure on healthcare $ 6.74 $ 4.36 $ 8.24 $ 12.39 - 2.2042 -0.1662 - 2.8338 P=0.0275 Significantly different Monthly expenditure on education $ 7.55 $ 8.16 $7.45 $ 8.93 0.1154 1.799 -1.500 P=0.9082 Not Significantly different This expanded income expenditure demonstrates the significance of unconditional cash transfer to vulnerable households during humanitarian emergencies. Prior to the cash transfer assistance a number of vulnerable households reportedly had to trade off food consumption for health care and/or schooling due to lack of sufficient income. According to the RESSNER baseline report conducted at the inception of the program, ‘spending on education and health influenced food consumption for some 38 percent of vulnerable households interviewed....’21 There were reported instances where household members had to stay for a couple of days without food at home due to expenditure on healthcare and education. The endline evaluation however revealed that expenditure on health care and schooling had not compromised with food intake in many instances; and this has undoubted improved on the wellbeing of household beneficiaries members. Interestingly participants engaged in FGD sessions from about 76 percent of 17 communities argued that price change was never experienced during the period of RESSNER project (see Figure 7). Communities (24%) which reportedly claimed price increase in the course of implementation however argued that such 21 See RESSNER Baseline Report, P. 28 Figure 6: Responses to price changes during implementation period 24% 75% 25% 76% 25% 75% 0% 20% 40% 60% 80% 100% All (N=17) Bombali (n=9) Tonkolili (n=8) % of HH Beneficiaries HHs in Districts who reported price change % communities who reportedly experienced price changes during cash transfer Experienced price increase Experienced no price increase 16 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone change was not influenced by the RESSNER project. In particular, price change has been attributed to changes in ‘forex’ rate, decrease in local foods and high transport costs for goods. 3.1.3 Reduced Coping Strategy Index It is noteworthy that the endline evaluation adapted the field data on coping strategies to use the reduced Coping Strategy Index (rCSI) with standard severity rating recommended and widely used across countries and settings22 . While the FFP partners have harmonized questions on coping strategies with scales ratings, CSI scores were noted to be high; and using the scores for rSCI may lead to confusing results that may affect comparability with other district level data. However, the harmonized questions were used to analyze item responses for each coping strategy reportedly adopted every week by the households in the last 30 days (4 weeks) preceding the interviews. The computed reduced coping strategy index (rSCI) was noted to be low (rSCI=11.2) for all beneficiary households- and also depicted similarity in across districts (see Figure 8). This result falls 61.2 percent below the WFP 2015 district level rCSI for Northern Province (rCSI=18.3)23 and therefore proves to be a highly significant improvement- given that the most vulnerable households were targeted by the RESSNER program. Figure 7: Reduced coping strategy index for beneficiary households in RESSNER intervention districts In addition both beneficiary households of cash transfers and seed voucher benefits demonstrated low coping strategies (depicted in Figure 9). Item response analyses further indicate that fewer numbers of household beneficiaries reported frequently using consumption soothing coping strategies including a) rationing coping strategies (limit portion of meal time, reduce number of meals/day), b) dietary change 22 See Field Methods Manual developed by USAID/WFP/CARE (2008): available at: http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp211058.pdf 23 See WFP Mvam BULLETIN # 10: January 2015 for Bombali, Tonkolili & Koinadugu districts. Available at: http://documents.wfp.org/stellent/groups/public/documents/ena/wfp272359.pdf 11.2 11.4 11.1 11.2 11.3 10.9 11.0 11.1 11.2 11.3 11.4 11.5 All households (587) Bombali (n=246) Tonkolili (n=341) Cash beneficiaries (n=526) Seed Voucher beneficiairies (n=61) Computed rSCI HHs in Districts per intervention rSCI computed from standard severity weighting 17 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone coping strategies (rely on less expensive or less preferred food) and c) reduced food quantity for family members coping strategies (restrict consumption of adult so children can eat more). Figure 8: Percent distribution of coping strategies adopted by household beneficiaries The low rCSI computed from the evaluation is unsurprising, and reinforces validity with the low Household Hunger Scale (HHS) demonstrated by household beneficiaries across beneficiary communities in the intervention districts. Some comparisons with baseline data revealed marked reduction in adoption of food consumption soothing mechanisms. As presented in Table 6, consumption coping mechanisms such as a) consuming or selling reserved grains for farming, b) loaning money or food items from elsewhere and c) begging food items and/or money from friends and relatives proved to have had marked reduction. Using Z-test, the change between baseline and endline (EOP) across coping strategies were tested at α =0.05. Meanwhile there was an overlap between results on proportion of beneficiaries who reported begging for food (34%) and those who ate more than 2 meals a day (84%). While begging for food was analyzed as item response among a number of coping strategies and consumption of meals was analyzed against total number of respondents interviewed, it is certain that some of those beneficiaries who reportedly begged for food, agreed to have eaten more than 2 meals a day. Table 6: Changes in coping strategies adopted by vulnerable households before RESSNER project inception Coping strategies Baseline results (n=111) EoP results (n=256) z-value p-value Result Interpretations 0% 20% 40% 60% 80% 100% Limit portion sizes at meal times? Reduce the number of meals per day Skip entire day without eating? Borrow food or rely on help from a friend or relative? Rely on less expensive or less preferred foods? Purchase food on credit/ take a loan to buy food? Gather unusual types / amounts of wild food or hunt? Harvest immature crops? Send household members elsewhere? Send household members to beg? Restrict consumption by adults so children can eat more? Rely on casual labor for food? 18.1% 20.9% 78.9% 30.2% 17.9% 30.4% 34.4% 38.6% 59.5% 74.0% 34.8% 48.9% 37.5% 26.8% 12.5% 40.9% 26.8% 38.2% 40.3% 32.9% 28.1% 18.1% 32.3% 24.5% 22.6% 19.8% 6.5% 22.8% 26.4% 26.2% 15.8% 18.1% 10.8% 6.3% 17.7% 19.8% 17.3% 23.8% 1.5% 5.7% 21.9% 4.9% 8.9% 10.3% 1.3% 1.7% 14.4% 5.9% 4.6% 8.7% 0.6% 0.4% 7.0% 0.2% 0.6% 0.2% 0.2% 0.0% 0.8% 1.0% % of HH beneficiaries Coping Strategies Never Seldom (1 day/week) Once in a while (2 days/week) Pretty often (3-6 days/week) Almost everyday 18 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Sold our last livestock 26% 15% 2.8 0.005 p<0.05 Significant change Sold or barter our agricultural tools 19% 20% 0.2 0.8102 p>0.05 No significant change Consume or sold our reserved grains for farming 60% 40% 3.9 0.0001 p<0.05 Significant change Reduced our daily food consumption 79% 74% 1.1 0.2698 p>0.05 No significant change Consumed or sold our remaining poultry 7% 23% 13.2 0.000 p<0.05 Significant change Sold or barter our cooking utensils 23% 10% 3.8 0.0001 p<0.05 Significant change Sold or barter our furniture 9% 11% 0.6 0.53348 p>0.05 No significant change Used our business income and/or savings 40% 44% 0.8 0.4396 p>0.05 No significant change Loaned money or food items from elsewhere 63% 49% 2.7 0.0073 p<0.05 Significant change Begged food items and/or money from friends/relatives 66% 34% 6.3 0.0001 p<0.05 Significant change Relied on in-kind/in-cash remittances from elsewhere 28% 34% 1.2 0.2217 p>0.05 No significant change Suspended school attendance by children 32% 12% 5.3 0.0001 p<0.05 Significant change 3.1.4 Economic and Livelihoods Activities Evolved from Project Intervention Household beneficiaries interviewed were observed to have been engaged in various economic and livelihoods activities. Accordingly, such activities have served as an exit strategy once the cash and seed voucher supports ended. In particular, cash transfer beneficiaries have used the cash received to invest in five key economic activities- with agricultural inputs (78%), farm labor (59%) and trading/business (54%) receiving the greatest attention (see Figure 10). Interestingly about 79 percent of cash transfer beneficiaries engaged in personal interviews claimed their investment will sustain them even when no cash assistance is given. Figure 9: Key areas of investment from cash received by household beneficiaries 78% 59% 54% 19% 37% 0% 20% 40% 60% 80% 100% Agricultural inputs Farm labour Income generating activities… Animals Savings (VSLA, ROSCAs) % of HH beneficiaries Key areas of Investment Sustaining Cash Investment “Farming has been our major area of investment since we started receiving cssh transfer program. Our investment in agriculture has been really successful as we are realizing an increase in production which in turn has led to increase in our revenue. The cash crops we cultivate can take just about three months to be harvested- which gives us the opportunity to cultivate them at least three times a year. Having frequent revenue is therefore making the growth of investment quite simple. Animals we bought for rearing will also be producing as soon as possible....and the poultry will be another increment on the amount we received from the cash transfer.” FGD Participant, Matinka, Makari Gbanti Chiefdom, Bombali district 19 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Additionally, a number of household beneficiaries are also reportedly engaged in some other economic activities including savings (37%) and animal rearing (19%). Figure 11 shows examples of investments made by beneficiaries during field evaluation exercise. Figure 10: Examples of beneficiary investment noted during field interviews Various testimonies documented from both FDGs sessions and personal interviews claimed that most beneficiaries are well prepared to cope with future shocks. At the FGD sessions, participants highlighted engagement in VSLA, agriculture, animal rearing and petty trading as common preparedness plans gradually building up resilience to future shocks/uncertainties. “The cash transfer program has indeed prepared us well to respond to any unexpected problem that we may face in the future. This is because we now have something that we can easily sell to settle whatever problem we may have in case money is not available at that moment. For minor problems, we can simply sell a pair or two of our poultry to the nearest town to settle that problem, or we can even sell a goat if the problem is very serious. If it occurs during the harvest season then it will be very easy for us because we can just sell some of the product we have harvested.” FGD Participant, Mombain Village, Safroko Limba, Bombali District Others highlighted agreed minimum contributions such as Le5,000 towards VSLA. Accordingly these are monies set aside to respond to any misfortune that may arise in the future. “One of the essences of the village saving box we created from the benefit received is to attend to emergency issues. Should a person has an emergency issue, he will request a loan from the box, the money will be given to him/her to solve his/her problem and repay the money later with a small amount of interest. This has made us stronger and less worried since we know there is always a place to take money when it comes to unforeseen eventuality.” 20 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone FGD Participant, Mafundu Community, Kunike, Tonkolili Dsitrict Training was further observed to have served as a powerful motivator for the sustainability strategies adopted by both cash and non-cash beneficiaries. As shown in Figure 12, those beneficiaries (both cash and non-cash beneficiaries) who received training and who are engaged in major income generating activities almost doubled those who never received training; and this ratio was found to be consistent across main income generating activities reported. Figure 11: Access to training versus engagement in income generating activities The analyses revealed that about 63 percent of all beneficiaries interviewed were reportedly trained in savings, income generating activities and agronomic practices (see Table 7). And the FGDs sessions clearly revealed that training was well delivered. Most focus group participants could clearly explain the various training they received in the course of the intervention. “We were trained on how to grow certain crops like groundnuts, cassava, pepper and krain krain. They also taught us the different types of soil and the crops that are suitable for each. They also trained us on how to prepare manure for our crops. This training has really increased our knowledge on the different types of crops that we are growing here and hence it has led to increase in production of most of our products” FGD Participant, Mayoloh Village, Paki Masabong Chiefdom, Bombali District Table 7 presents the various training reportedly received during the course of project implementation. Table 7: Training received by both cash and seed voucher beneficiaries Beneficiaries Training received by beneficiaries All (N=587) Cash transfer Seed Voucher Received training n=367) 63% 58% 100% Crop diversification 87% 85% 100% Vegetable gardening 95% 94% 100% 353(96%) 201 (55%) 342 (93%) 201 (55%) 186 (51%) 122 (33%) 182 (50%) 95 (26%) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Engaged in income generating activities Petty trading Agriculture VSLA % of HH beneficiaries Training and income generating activities Received training Did not receive training 21 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Seed multiplication 90% 88% 100% Post-harvest management 84% 81% 100% Nutrition 89% 88% 97% EVD prevention, Health, sanitation and hygiene 90% 88% 100% Business development/Savings 82% 66% 93% Others 0% 0% 0% 3.1.5 Multi-Indicator Comparisons: Analyses of Correlations between Indicators Multi-indicator comparisons are necessary to assess associations between various indicators and their overall influence on the outcomes from the evaluation. The Pearson’s Rho, otherwise product moment correlation coefficient was used to identify these associations. Various indicators including proxy measures of food insecurity (HHS, HDDS), coping strategies, and expenditure were correlated to note the strength of association with each other, and their influence on the RSCI outcomes. The following results were noted from the Person’s correlation coefficient analyses compared at 0.01 level of significance (also see Table 8 for details): ▪ Household Hunger Scale (HHS) has strong correlation with both Coping Strategy Index Score (r=0.649, p<0.01) and Reduced Coping Strategy Index (r=0.508, p<0.01). This implies increased hunger may have influenced households to engage in some predictable negative coping reactions. ▪ Expenditure on food has positive correlation (though weak) with HDDS (r=0.058, p>0.01); even though this correlation is weak, it is stronger than correlation between expenditure on education and HDDS (r=0.017, p>0.01), and healthcare and HDDS (r=0.005, p>0.01). This implies that increase in expenditure on food items had strongly improved on households’ diversity in dietary intake. ▪ Comparison with individual coping strategies adopted by households with other indicators further proved that, a) reduction in number of meals per day have very strong association with rCSI (r=0.813, p<0.01), b) skipping entire day without eating strongly influence HHS (r=0.589, p<0.01), c) increased practice in borrowing food also contribute to increased rSCI (r=0.618, p<0.01), and d) reliance on less expensive food, gathering unusual wild food, purchasing of food on credit and harvesting immature crops, all influence increase in both HHS and rCSI. Table 8: Table showing results of multi-indicator correlation coefficient analyzes Description CSI HDDS HHS rCSI (FFP recommended) rCSI (Int'l Standard) CSI 1 .255** .649** .900** .904** HDDS .255** 1 .163** .223** .231** HHS .649** .163** 1 .497** .508** rCSI (EFSP recommended) .900** .223** .497** 1 .971** rCSI (Int'l Standard) .904** .231** .508** .971** 1 Expenditure on food -.130** .058 -.114** -.113** -.116** Expenditure on healthcare .146** .005 .160** .197** .169** Expenditure on education .084 .017 .122** .100* .099* Total expenditure .000 .040 .025 .033 .021 Limit portion sizes at meal times .664** .161** .287** .766** .728** Reduce the number of meals per day .757** .198** .376** .880** .813** Skip entire days without eating .513** .153** .589** .261** .305** Borrow food or rely on help from a friend or relative .615** -.018 .453** .543** .618** Rely on less expensive or less preferred foods .758** .178** .520** .854** .790** Purchase food on credit/ take a loan to buy food .613** .027 .513** .483** .506** 22 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Gather unusual types / amounts of wild food or hunt .750** .282** .567** .550** .564** Harvest immature crops .711** .319** .513** .511** .520** Send household members elsewhere .526** .246** .375** .323** .386** Send household members to beg .492** -.080 .373** .274** .328** Restrict consumption by adults so children can eat more .734** .396** .459** .722** .792** Rely on casual labor for food .556** .084 .341** .334** .360** Sold last livestock .285** .247** .371** .260** .231** Sold or barter our agricultural tools .492** .205** .655** .271** .323** Consumed or sold our remaining poultry .206** -.056 .438** .056 .092* Sold or barter our cooking utensils/amenities .184** -.108* .467** -.020 .050 Sold or barter our furniture .195** -.122** .421** .022 .091* Used our business income and/or savings .175** -.133** .186** .160** .190** Relied on in-kind/in-cash remittances from elsewhere .366** .109* .503** .343** .334** Suspended school attendance by children .208** -.145** .485** .030 .111* Total asset .192** .288** .252** .088* .089* Total # of investment -.208** -.047 -.196** -.151** -.126** Used cash for daily food consumption .053 .040 -.073 .034 .022 Invested in the education of children -.001 .046 -.080 .007 .016 Invested in health .015 -.213** .054 -.006 -.039 Purchased land for building -.038 .034 -.047 -.062 -.042 Purchased land for agriculture -.042 .082 -.014 -.008 .039 Shared cash among household members -.055 .103* .027 -.012 -.024 Invested in business -.305** -.108* -.450** -.213** -.200** Invested in savings groups (SGs) -.189** -.111* -.052 -.161** -.129** **. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed). 3.1.6 Achievement of Targets: Target-Actual Comparison of Baseline and Endline/Project Performance Evaluation Results The results from analyses of the evaluation findings present clear indications that the RESSNER program achieved its intended purpose to a larger extent. The performance rating of specific indicators set by the project shows that 81 percent of all targets set to track progress in implementation were achieved (see Table 9). Table 9 : Extent of achievement of purpose of the cash transfer intervention Design summary/Results Indicators Baseline values Targets Endline/ PPE values z-value24 ρ-value Result Project Purpose: To reduce the negative coping strategies of extremely poor HH in EVD affected communities % of household beneficiaries reporting negative coping strategies (sale of household items, borrowing cash, absenting children from school). 80% 30% 40% 7.7 0.0001 ρ <0.05, highly significant change Intermediate Outcome1: Increased access to cash for purchase of basic food % of household beneficiaries with increased number of meals per day as compared to baseline 26% 70% 84% 12.6 0.0001 ρ <0.05, highly significant change 24 See EpiTool for z-test available at: http://epitools.ausvet.com.au/content.php?page=z-test-2 23 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone items for selected household beneficiaries % of household beneficiaries who report satisfaction with regards to relevance, timeliness and accountability of cash transfer 0% 75% 91% 20.2 0.0000 ρ <0.05, highly significant change % of household beneficiaries reporting increased consumption of food groups (consumption of more than 6 food groups) 41% 70% 94% 14.1 0.000 ρ <0.05, highly significant change % of household beneficiaries reporting increased average daily income from current base income (Le10,000) 33% 80% 66% 6.5 <0.0001 ρ <0.05, highly significant change % of total average daily cash spent by household beneficiaries on food items 47% 60% 59% 2.3 0.0203 ρ <0.05, significant change % of total average daily cash spent by household beneficiaries on non-food items (incl. health & education) 35% 30% 40% 1.0 0.3267 ρ >0.05, no significant change % of community members reporting increased knowledge of nutrition and food consumption patterns 13% 80% 88% 16.6 0.0000 ρ <0.05, highly significant change Output 1: 3,600 Registered HH beneficiaries are provided with unconditional cash grant # of HH in EVD-affected communities receiving cash disaggregated by age and sex (& disability) 3,600 HH (21,600) 100% 100% Number and value of cash distributions made by the end of the project USD972, 000 @US$30 /Mo USD97 2,000 @US$3 0/Mo USD972,000 @US$30/M o Intermediate Outcome 2: Improved ability of HH to recover from impacts of EVD % of household beneficiaries reporting they have re￾engaged in agricultural activities 9% 80% 89% 17.7 0.0000 ρ <0.05, highly significant change # of household beneficiaries who are able to save seeds for the following planting season 0% 75% 95% 17.7 0.000 ρ <0.05, highly significant change % of beneficiary h/h engaged in savings activities (VSLAs/ROSCAs) 15% 40% 59% 8.4 0.0000 ρ <0.05, highly significant change Output 2.1: Basic agricultural inputs are provided for 900 poor farming HH # of household beneficiaries who received agriculture inputs for the next planting season 0 900 900 Output 2.2: 4,500 Household beneficiaries are provided with Nutrition and EVD prevention training and sensitization # of household beneficiaries who participated in the nutrition and EVD prevention training and sensitization 0 4,500 8,450 Observably, achievement of targets was particularly observed for indicators including a) % of beneficiary households with increased number of meals per day, b) % of beneficiary households engaged in savings 24 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone activities, c) % of total average daily cash spent by beneficiary households on food items, d) % of beneficiary households reporting consumption of more than 6 food groups, and e) % of beneficiary households with increased knowledge of nutrition and food consumption patterns. Although the targets for some indicators were notably not met, changes made in the baseline values were statistically proven to be significant for almost every indicator (see Table 9). For instance, the target of outcome indicator such as ‘percent of household beneficiaries reporting increased average income from base income (Le10,000) 25’ was not achieved, but the percent increase was tested to be statistically significant. Notably, the low rate of achievement of the average income threshold target was partly due to the consideration of the high inflation rates from base period (February 2016) to current period (September 2013)26 . However, household beneficiaries demonstrated potentials for improved ability to recover from the impacts of EVD. In particular 89 percent of household beneficiaries reported to have reinvested cash received in agricultural activities and 59 percent are engaged in savings activities- which will consequently increase their income and prepare them to respond to future shocks. The achievement particularly observed from key outcome and output indicators, could be strongly attributed to the impressive results on Household Hunger Scale (HHS), Household Dietary Diversity Score (HDDS) and Reduced Coping Strategy Index (rCSI). 3.2 Procedures and Processes in Project Implementation 3.2.1 Accountability, Timeliness and Budget Utilization Accountability Mechanism of the RESSNER Program Accountability was observed to be a core component of the RESSNER Program implementation; and had followed consistency since project inception. The participatory community-driven approach adopted by the project, and involvement of key district stakeholders including the Anti-Corruption Commission (ACC), NaCSA and District/Local Council representatives from prelisting to beneficiary identification, selection and cash disbursement have in combination being an effective demonstration of an accountable and transparent system. 25 Exchange rate from US$ to SLL increased by 54% from Feb’16 to Sep’17. Therefore average daily income used for comparison with base income was approximately Le 20,000 26 Exchange rates were US$1.00=SLL4,027 (as at 29 Feb 2016) and US$1.00=SLL7,522.96 (as at 30 Sep 2017). See Ouanda forex website: https://www.oanda.com/currency/travel-exchange-rates 25 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Figure 12: Beneficiaries displaying payment vouchers during field interviews At the beneficiary pre-listing stage, Community Identification Committees (CICs) were formed and allowed to lead the pre-listing process using basic pre-defined criteria. This was followed by a validation exercise, where beneficiary households were further screened through cloud-based Open Data Kit (ODK) system. This method proved to be an innovative community driven approach that promoted local ownership, and ensured the right beneficiaries were selected, and given the due amount of benefits. Interviews with a range of stakeholders involved in the RESSNER program proved that cash disbursement was always done in the presence of various stakeholders including the Anti-Corruption Commission (ACC), District Council Officials, and community leaders. Most importantly, the program had notably ensured that beneficiaries are informed about the amount given to them at the time of disbursement. Further beneficiaries were issued with payment vouchers (as depicted in Figure 13) that are punched each time they received cash. Also noted was that the Grievance Redress Mechanism (GRM) was very active in the course of implementation, and Anti-Corruption Commission (ACC) witnessed almost every round of cash disbursement done. As claimed by the Presidential Delivery Team (PDT) Lead for Tonkolil, ‘...the payment process was credible, given that all actors have to be present at the time of disbursement’. Similarly, non-cash beneficiaries targeted by the RESSNER Program such as seed voucher beneficiaries used vouchers for access to seed assistance. It was also noted that seed fairs were done with the presence of District Council Monitors from the devolved Ministry of Agriculture Forestry and Food Security (MAFFS), and Seed Certification Team. This was to ensure that viable seeds are available, and are accessed by the right beneficiaries. During the seed fair, beneficiaries were reportedly given the opportunity to choose the variety of seeds, particularly rice and groundnuts. Also observed was that the beneficiary organization (CARE) largely demonstrated accountability to the local staff and implementing partners. In particular, implementing partners were allowed to executive a number of activities including community mobilization activities (to get beneficiaries ready for each round of cash transfers), awareness raising on project and weekly monitoring activities in the target communities. Timeliness of the Intervention The evaluation documented various views on timeliness of the RESSNER program. Both beneficiaries and other project participants argued that the intervention was indeed timely. For instance, the program ensured that rounds of cash disbursement were done in lean periods- that is months like July/August- when extreme hunger is felt across communities, and extremely poor households are mostly affected. Notably cash disbursement in lean periods may have averted extreme hunger, and had prevented extremely poor households from adopting negative coping behaviors. “The cash transfer program was timely, and has helped us greatly in reducing the negative coping strategies we used during Ebola. Having food in our various homes was a major problem. Sometimes we could spend the whole of the day without eating and even when we happened to eat at night the food would not be enough even for the kids. We used to borrow money to buy food and those creditors would embarrass us for failing to pay our debts. This was very common during planting season when no new crops have been harvested and when we have spent all what we had on farming activities. But through the intervention of this cash transfer, we have been able to set aside certain amount of money for such situation...” 26 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone FGD Participant, Kamaworain Village, Biriwa Chiefdom, Bombali District Seed assistance, as a component of the RESSNER Program took seasonality into consideration, and as the evaluation noted, agronomic seasonal calendars were used to deliver the variety of seeds. One of the examples noted was that leguminous crops like groundnut grows well in April/May, and therefore such seed was delivered in April/May. While May was claimed as an appropriate month for growing groundnut, other process that guarantee quality seed started few months before delivery. Accordingly, Seed Vendors were identified in March and screened-which was an appropriate timing to organize seed fairs in the planting season. In April, germination test was done and beneficiaries were given training on agronomic practices. Seed Certification was done by the Sierra Leone Seed Certification Authority (SLeSCA) before vendors were allowed to participate and sell seeds during the seed fairs. Notably the seed certification process was to assure purity, viability and germination rate of the seeds. Budget Utilization Notably a total of approximately US$4.55 Million was allocated for implementation of the entire RESSNER Program. Overall 96 percent (US$4.37 million) of the total allocated budget was used to reach the intended number of households targets including Phase I targets (4,500 households), Phase II targets (3,600 households) and Seed Voucher targets (900 households). Ideally, the project assumed an average of 6 individuals per each vulnerable household targeted, estimated from the Sierra Leone 2004 Population and Housing Census data. This means, a minimum total of 54,000 individuals were directly reached. Hence, approximately US$81 was used per individual household member in the course of RESSNER Program implementation; and 96 percent of the total grant was used to provide improved access to food by 94 percent27 of 54,000 members of vulnerable households targeted across the implementation areas. Observed cost reduction modalities Every possible effort to reduce cost, while optimizing quality outcomes was observed to have been made in the course of the RESSNER Program intervention. Community clustering was one mechanism used by the program to reduce cost and improve on effective cash disbursement. Another was to do quarterly cash disbursement from the original plan of monthly disbursements. For instance, in Phase II of the RESSNER program implementation, 106 beneficiary communities were covered- hence making payment in each of these communities in less than 7 days had meant increase in human resources and budget allocation towards human resources and logistics, or otherwise longer pay period and increased logistic costs. Cognizant of this issue, a total of 72 pay points was identified for cash disbursement. Clustering was done in consultation with the community, considering the distance and proximity of communities to the pay point. The use of local produce for seed fair also reduced cost of transportation of highly expensive (sometimes low quality and viable) imported seed varieties. This had also meant encouraging local seed growers to engage in large scale production, which in turn would increase local farm produce in the intervention districts. Further noted was that, initial modalities for payment in the first phase of RESSNER Program implementation had some flaws that undermined efficiency in implementation to some extent. For instance, using online payment through Airtel payment agents was unsuccessful, and had meant waste of resources 27 Proportion of total targeted households reporting high dietary diversity was approximately 94 percent (see Figure 3). 27 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone for service fees. While this was noted in the mid-term evaluation, the end line evaluation however observed such issue was resolved prior to the start of RESSNER Phase II; hence only offline payment, using one vendor was successfully used in phase II of the RESSNER Program intervention. The decision to discontinue the service of Airtel online payment modalities resulted in total savings of about US34,000 transfers fees to service providers. CARE International was also observed to have instituted modalities for sound management of project budget at both country and district office levels in Sierra Leone. Strict financial and procurement rules at the country office are enforced for all projects; and this covers both restrictive or unrestricted donor projects. Accordingly, CARE has two types of payment- the online payment and offline payment. In particular, the online payment goes through a rigorous pay cycle, and uses a cloud system where final approval of requests is made remotely online by the budget holders/authorized bank account signatories. This process require approval at each stage including raising Purchase Request (PR) and Purchasing Order (PO); and approvals are made by Procurement Department, Finance Department and Country Director after thorough review and confirmation of all necessary documents that meet the minimum requirement of CARE and the donor. The offline payment however, does not go through Purchase Request (PR) and Purchase Order (PO) stages. These payments include accommodation for project staff, medical bills, travel expenses claim, etc. However, such payments are supported with sufficient justification including emails with appropriate charging details attached. But also payments for these claims are processed using people soft grants and Financial Management System; and such payments are approved online using the pay cycle. Most importantly, approvals are often delayed for further project activities without prior submission of reports and liquidation for previous activities. This has meant the RESSNER project budget and activities were closely monitored by all actors to ensure sound management of budget and human resources. 3.2.2 Participant’s Perception about the Procedures and Processes in Project Implementation Participants’ perceptions (including perception of project staff and beneficiaries) about field project implementation processes and procedures were also captured in the evaluation. Overall, satisfaction level of beneficiary about the cash-based assistance was high- with about 91 percent claiming to be either satisfied (45%) or very satisfied (46%) (see Figure 14). However, 9 percent of beneficiaries had some reservations about the benefit, and most of them had expressed concerns that cash was small to cover family problems. These views might be valid, considering variations in household sizes across beneficiary communities. For instance about 55 percent of all cash transfer beneficiary households interviewed report had over 6 household members (threshold considered), and reportedly shared an average of about 9 households during the cash transfer. This means, while the RESSNER Program meets exactly the actual target of 8,100 vulnerable households, the number of individuals who directly benefited from the cash transfers far exceeds the expected number of household members- that is a total of 48,600 individuals (27,000 individuals in Phase I and 21,600 individuals in Phase II). Notably, the household average of 6 members per households estimated in Sierra Leone 2004 census was used for the RESSNER Program, which was outdated. 28 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Figure 14: Satisfaction level expressed by beneficiaries for cash-based support Views expressed by beneficiaries about walking distance and waiting time proved that clustering of communities was detrimental for most beneficiaries in course of the project. Content analyses of the open ended questions showed that about 67 percent of all beneficiaries interviewed expressed concerns about either the long waiting time or walking distance experienced during field implementation. As expressed by most beneficiaries interviewed, there was lot of time wasting, and sometimes beneficiaries had to wait whole day without receiving cash. Even though some beneficiaries claimed to have walked in groups (for those who could not afford transport), long walking distance (of a reported average of 4 miles) was further associated with risk and high cost of transportation. These are concerns that would have required attention￾giving that most beneficiaries targeted were vulnerable people including the aged and disabled persons. As noted from evaluation, a relatively high number of the most vulnerable groups such as the aged and disabled persons would have been captured as beneficiaries during the cash transfers. This was evident from the number of persons in these categories captured for interviews through random selection. As depicted from Figure 15, about 57 percent of all respondents interviewed were either aged28 (37%) or disabled persons (20%). Meanwhile, the risk associated with a relatively long working distance was reportedly compromised with community clustering to reduce cost and manage implementation time. 28 Aged refers to beneficiaries of age 60 years and above 46% 45% 8% 1% Satisfaction level of beneficiaries about the cash-based support (N=526) Very satisfied Satisfied Somehow satisfied Not satisfied Aged (60 yrs and above) Disabled persons Disabled & aged 192 (37%) 105 (20%) 50 (10%) Number (Percentage) Vulnerable groups 29 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Figure 15: Percent of most vulnerable persons interviewed in the evaluation Furthermore, triangulation of expressed beneficiary concerns about the long waiting time with responses from project staff proved that this as an issue experienced at the initial stage of the RESSNER project implementation. As noted from the project staff, there were indeed process delays that were deeply rooted from poor management practices of financial service providers (FSPs) particularly Airtel. Accordingly Airtel was contracted to carry out online payment, but this vendor faced a number of uncontrolled circumstances during the first phase of implementation. In particular, lack of own phone by beneficiaries, poor network coverage, lack of cash out points and high illiteracy rate among beneficiaries (which makes SMS reading difficult) contributed to an ‘unnecessarily long payment process’ that took about three months to complete. Hence this situation was reportedly resolved by discontinuing the service of Airtel during the first phase of implementation. Furthermore, question about whether beneficiaries were pressured in any way to forcefully give money was almost unnecessary. Almost all beneficiaries interviewed claimed they were never asked to give out money. However, about 7 percent of all beneficiaries interviewed purported to have missed out cash payments due to strict conditions. Most of the instances reported were related to absenteeism due to unavoidable circumstances such as travels and sickness. Meanwhile, payment for defaulters were not definitely withheld, but were paid in next round of payment- mostly after 2 months. 3.2.3 Gender Implications of Project Intervention Marked reduction in economic activities such as VSLAs activities by small-holders and petty trading over a longer time frame was highlighted in the initial program design as a critical issue that required attention. This situation (as discussed in the initial program design) was due to closure of local markets, and diminished demands during the Ebola outbreak. Petty traders and small-holders could therefore not afford for basic food needs, and also had limited financial capacity to restart economic activities and activities hinged on access to credit including petty trading and agriculture. In addition VSLAs members of poorer households could not access loan due to limited financial capacity and drop of VSLA size. Cognizant that 60 percent of VSLAs members were women who were mostly affected by the Ebola, and that severe loss of income by women would have led to major constraints by poorest households to cover basic food needs; the project design was expected to have captured more women. Hence, the evaluation noted that more women were captured as registered beneficiaries than men (see Figure 16). 30 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Figure 16: Gender distribution of registered beneficiaries by project component and survey data Results from the endline evaluation revealed significant increase in female-headed households engaged in key economic activities such as petty trading and VSLA, than male-headed households; and gender disparity in economic activities has significantly reduced. For instance, the proportion of female-headed households engaged in petty-trading has increased by 25 percent points than male-headed households (experiencing a change by only 3% points). Also even though engagement in VSLA activities markedly increased in both male- and female-headed households, the increase is higher for female-headed households than male-headed households. (See Figure 17). Figure 17: Distribution of key economic activities by gender Arguably, the project (according to diverse views of respondents) has not greatly influenced changes in decision-making on use of cash resources and expenditure within beneficiary households. Overall, 60 -10% 10% 30% 50% 70% 90% Phase I (n=4500) Phase II(n=3600) Seed Voucher (n=900) Endline sample(n=526) 13% 28% 40% 20% 87% 72% 60% 80% Percentage Beneficiaries by gender Gender distribution of registered beneficiaries Males Female 41% 34% 44% 59% 3% 25% 97% 91% 90% 88% -8% -3% 21% 10% 52% 46% 32% 35% Male (n=34) Female (n=77) Male (n=105) Female (n=421) Male Female Baseline (111) Endline (526) Percent Change Percentage distribution Project timeline Key economic activities by gender VSLA Agriculture Petty trading 31 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone percent of all cash transfer household beneficiaries interviewed alluded that the project has not changed patterns in household decision-making on cash resources and expenditure (see Figure 18). However, while this is a short-term emergency intervention, marked changes in decision-making is rather not expected from the RESSNER Cash Transfer Program. Meanwhile, the result on decision making generated from the evaluation will serve as springboard for any further research in the future. Figure 18: Change in decision-making in use of HH cash resources and expenditure Comparative analyses on decision-making between male- and female-headed households also revealed female-headed households demonstrating high independent decision-making by women on use of cash resources and expenditure than those in male-headed households. Further revealed was that shared decision￾making among other household members on cash resources is highly distinct in female-headed households than in male-headed households. For instance, while 70 percent of female-headed households claimed strong involvement of women in decision-making on use of cash resources and expenditure, only 25 percent of women were reportedly involved in similar decision-making in male-headed households. This result clearly shows that more women were empowered by the Program for control over cash resources than men￾considering the overwhelming proportion of female-headed households targeted by the RESSNER Program (as shown in Figure 16). Figure 19 depicts the dynamics in economic decision making in both male- and female- headed households. 40% 44% 39% 60% 56% 61% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%100% Project changed pattern of control of cash resources and expenditure in HH Male (n=105) Female (n=421) % of HH beneficiaries Gender Change in decision-making in the use of cash Yes No 32 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Figure 19: Economic decision making in both male- and female- headed households 3.2.4 Lessons Learned from the process of RESSNER implementation Box 1 below presents major lessons learned from the RESSNER Program intervention across communities in the two intervention districts. 62% 20% 5% 2% 11% 0% 43% 28% 15% 1% 12% 0% 2% 10% 20% 30% 40% 50% 60% 70% Self Partner/spouse Self and partner/spouse jointly Other household member Self and other household member (s) Partner/spouse and other household member(s) % of Male and Female beneficiaries HH decision makers Male (n=105) Female (n=421) 33 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 3.3 Collaboration and Coordination Findings on collaboration and coordination documented a) coordinated activities with national and local government authorities and how this address the issues of parallel systems and structures, b) coordinated activities of implementing partners (IPs) with other FFP implementing partners and how this enhanced results, and c) other activities in the intervention districts that impact results (leveraging funds, enforced or complement implementation, and sustainability). Coordination with National and Local Government Actors The RESSNER program was observed to have maintained compliance with National Standards and Policies on Social Safety Net by building strong synergy with existing systems and structures on Social Safety Net (SSN). This was enhanced by the strong involvement of the National Commission for Social Action (NaCSA) in the implementation process. The Commission is recognized as a lead agency on social protection programs and has carried out a number of social safety net interventions including cash transfer (both conditional and unconditional) for vulnerable households in Sierra Leone. It was also observed that the agency serves as a repository of all poverty data and Geo Referencing System (GRS) for appropriate targeting of poor households across all administrative districts in the country. The involvement of the agency had therefore served as a window of opportunity for leveraging, as well as driving towards national ownership. In particular NaCSA was reportedly involved in preparing data calculations of potential Box 1: Lessons Learned from the process of RESSNER Program implementation  The Consent Form approach for absenteeism was an appropriate strategy adopted during the second phase of RESSNER Program implementation to take care of backlog issues reportedly experienced by some beneficiaries in the first phase. This approach allowed for registered beneficiaries to present a next-of-kin that will represent them in times of unexplained circumstances; and therefore provide them with an opportunity to receive cash on the day of disbursement without having to wait for another 2 months before receiving backlogs. The Consent Form required the name, photo and signature of the next-of-kin, signatures of the beneficiary and community representatives.  Constant feedback from the community was proven as an effective way of service delivery, and had minimized the long waiting time experienced by project participants in the first phase of project implementation.  Online payment modality adopted in phase I of project implementation proved ineffective; and was a powerful lesson that meant online mobile payment modality would not be successful for cash transfer intervention in areas with poor mobile network coverage, and high illiteracy among beneficiaries.  Involvement of the Community Identification Committees (CICs), community authorities and other key district actors in the selection and payment processes is also a powerful lesson for effective accountability mechanism and reduced risk of corruption.  The seed voucher modality, using locally produced seeds and seed fairs where beneficiaries choose among crop varieties was a very innovative idea. This encourages increase in local growers, and is a means of promoting/ domesticating the country’s Local Content Act at community level. It is also a cost-effective approach for similar future intervention.  Lessons learned from gender analyses on the use of household cash resources and expenditure is that, economic empowerment of female-headed household is more likely to increase women’s independent decision making on economic resources, while not compromising economic decision making by other household members. 34 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone beneficiaries (extremely poor households) at chiefdom level; and also providing training support for field staff in relation to beneficiary selection, establishment of the cash transfer system and external monitoring of project start-up and implementation. Whilst NaCSA notably maintains a database of all cash transfer beneficiary households (including those covered by NaCSA and other cash transfer partners), the direct involvement of the agency therefore addressed issues of parallel systems and structures and had undoubtedly led to avoidance of duplication of efforts and waste of resources. This was demonstrated during the process of chiefdom level selection and allocation of resources. At this stage, consideration was made for other Cash Transfer Programs (NaCSA, SNAP/ACDI/VOCA) and therefore, sections that have been targeted by these programs were excluded. This therefore allowed for proper integration of the RESSNER program into existing Social Safety Net (SSN) mechanism; which was reported as part of the modalities that helped in some ways, in the achievement of the national target of the President’s Recovery Priorities on Social Protection during the course of implementation. “...The Commission had to reach 59,000 extremely poor and vulnerable households with conditional and unconditional cash transfers, to achieve targets set by the President’s Recovery Programme (PRP) on Social Protection for Ebola. Indeed we met the target through the 14% contribution of the RESSNER project to the programme by then; and this greatly helped NaCSA to receive the national prestigious Gold Star award as the only Government Agency that achieved its target as set... We are therefore always thankful to CARE for working with NaCSA on the Social Protection Pillar of the Agenda for Prosperity” Staff, NaCSA, Bombali District, Northern Region The RESSNER program was also observed to have included GIS capacity building support to NaCSA by providing GIS equipment (hardware and software) and also recruiting and training a GIS Officer observed to be deployed at NaCSA country office. This support has strengthened the institutional capacity of NaCSA to undertake GIS mapping of beneficiaries not only for CARE but for all organizations implementing the social safety net interventions, and similar future interventions. While the GIS equipment would be used to produce data and mapping of beneficiaries as a first priority, it was also noted for all other priorities for establishing in-house system within NaCSA for national interventions- thus building strong institutional sustainability for vulnerability assessment and mapping across the country. Further observed was that the RESSNER program took full consideration of local resources and capacities in both design and implementation. Notably, the program was well connected, and particularly engaged local institutions including sub-national government institutions (District Councils) and community structures (opinion leaders). This strategy was observed to have catapulted local buy-in and ownership, and have largely built and strengthened the capacities of District Councils, and community structures to respond to similar emergency humanitarian interventions more effectively in the future. Such strategy was testified by a District Council Official in Bombali District ‘as a model for participatory community-driven projects which pooled local resources and ensured that every actor was involved in the process of implementation...’ Hence this process served as a knowledge brokerage and in part created a way by which sustainable knowledge was built for local intervention on cash transfer programming. ‘...We were involved in the CARE Cash Transfer Program, than any other NGO programming in our district. We were given the opportunity to closely monitor the project from the pre-listing stage, to the beneficiary selection and cash disbursement and post cash disbursement stages. Given that 35 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone unconditional cash transfer is a new development paradigm in our district, this intervention is so exemplary to us- and therefore serves as a knowledge brokerage for us, by tremendously increasing our understanding of the processes involved in cash transfer interventions…” Deputy Chair, Bombali District Council, Bombali district Coordination with FFP Partners and other District Actors Collaborative partnership with FFP partners and other organizations that have carried out similar interventions was also observably considered as a core component of the RESSNER project design and implementation. At national level, FFP partners’29 meetings are periodically held to share approaches, updates and challenges for learning in the unconditional cash transfers programming. This has ensured smooth alignment; and therefore overlapping situations (in which interventions duplicate efforts) were duly avoided. At district level, a number of coordinated activities were reportedly held in the course of project implementation with similar aim of shared learning (including best practices) and delineation of failures for remedial actions. Specific coordinated activities notably organized at the district level included sector meetings, district forums, national recovery meetings and After Action Review (AAR) meetings. The District Sector Meetings are conducted by NaCSA, ACC and District Council every two weeks, and are especially organized by NaCSA. As inter alia noted, NaCSA was recognized in the project as a semi￾autonomous government agency that has the mandate to coordinate all Cash Transfer (CT) or Social Safety Net (SSN) interventions across the country. In particular, the sector meetings are organized to provide technical support for all SSN-related interventions, but more so guidance to avoid duplication of efforts, as well as compilation of sectoral updates for national reports. The District Council on the other hand is required to align district level service delivery, and had therefore organized district forums on a quarterly basis to discuss and compile updates on program interventions and other activities in the district. Further noted was that the Presidential Delivery Team (PDT) had organized national recovery meetings in the two intervention districts. The PDTs are Presidential Working Groups located in every district to monitor and track progress in all development interventions (especially those related to Ebola) in every district to monitor and track progress in all Ebola response interventions carried out at both district and local level. While the project design did not capture the PDTs as collaborative partners, they were reported to be fully involved and have closely monitored processes and progress in the cash transfer project. Post distribution activities such as the Post Distribution Monitoring (PDM) and After Action Reviews (AAR) were observably conducted periodically in the course of program intervention. In particular, the After Action Reviews were conducted to review and report on the RESSNER project activities, and processes involved, as well as to identify lessons learned and recommendations for other related future emergency response programming. Each AAR was observed to have mobilized an average number of 42 key players drawn from project communities, district and national level institutions interested in the cash transfer programming. Opportunities for Leveraging Funds, Complementing Implementation and Sustainability The program was also observed to be appropriately streamlined with other interventions in the targeted communities, which in part influenced results and created an avenue for leveraging funds, complementarity and sustainability (at phase out). For instance, whereas the RESSNER project design did not make adequate 29 FFP partners include USAID national implementing partners for the EFSP/FFP including CRS, CARE, World Vision, Save the Children and ACDI/VOCA 36 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone provisions for training activities in terms of sustaining both cash and non-cash benefits received, the Epidemic Control and Reinforcement of Health Systems (ECRHS) project of CARE in the same districts was used to include RESSNER beneficiaries in trainings on VSLA, agronomy, crop diversification and vegetable gardening. This according to a local project staff interviewed helped in leveraging funds, and contributed towards creating an exit point for the RESSNER project. Further reiterated by a District Council participant of the RESSNER program was that attending sector meetings with MAFFS facilitated the incorporation of RESSNER beneficiaries into their Farmers’ Corporation and VSLA programs. Box 2 below outlines lessons learned from findings on project collaboration and coordination. 4 CONCLUSION AND RECOMMENDATIONS Findings from the evaluation undoubtedly revealed that the unconditional cash transfers (UCT) program is an excellent modality for emergency humanitarian response targeting poor and vulnerable households. Whereas the RESSNER cash transfer program achieved its aim of providing support for immediate access to cash for basic food needs, it also proved that the modality could promote economic independence, high dietary diversity and improved nutritional status for extremely poor and vulnerable households. This was evidenced from outcomes such as reduced negative coping strategies, high dietary diversity scores, low household hunger scale, expanded expenditures and improved economic activities (as sustainability strategies) among extremely poor households across communities in the two intervention districts. While cash disbursement served as a pivotal activity of the program, these outcomes were also influenced by a number of best practices that require incorporation into any similar humanitarian health emergency response. However the program was not implemented without flaws that also served as lessons learned for future programming. 4.1 Recommendations Cognizant of the evaluation results, the following recommendations have been made and require critical considerations during any future humanitarian and/or health emergency response: Collaboration and Coordination ▪ Multi-sectoral meetings through district forums, after action reviews, working groups should always be core components in humanitarian and/or health emergency response. Findings from the evaluation proved these as very important strategies for shared learning, capacity building, gap filling (which takes care of duplication of efforts) and sustainability after phase-out of humanitarian response. Box 1: Lessons Learned from Collaboration and Coordination in RESSNER Program Implementation The following are key lessons learned from findings on collaboration and coordination of RESSNER program intervention:  The multi-sectoral meetings conducted through district forums, working groups, after action review groups are innovative strategies for shared learning, capacity building, gap filling (thereby ignoring duplication of effort), and sustainability after phase-out of humanitarian response.  Mobilizing local actors and allowing them be a complete part of the implementation process of the program proved to have enhanced coordination, and promote local buy-in. This could be a model of participatory monitoring approach that showcases a credible and transparent way of implementing community-driven programs. 37 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone ▪ Private Sector engagement was observed to be very minimal in the RESSNER Program intervention, yet this is very important in fostering sustainability of cash-based interventions. Future cash transfer programming must ensure that the private sector forms a core component of the sustainability strategies. In particular, financial services institutions (FSIs) and Micro-finance institutions (MFIs) are critical in ensuring sustainability of cash-based interventions. It is therefore recommended that these institutions be involved in future cash transfers intervention to ensure that beneficiaries receive quality training on business development and management; and are linked to credit and savings opportunities for sustainability after phase-out. ▪ Considering local actors and related community structures as part of the cash transfers intervention proved to be a model of participatory monitoring approach and must be replicated in future emergency response. This approach fosters credibility and transparency, and can minimize suspicious tendencies in the course of implementation. ▪ Involving all actors in a cash transfer program, such as Community Identification Committees (CICs), community authorities, and other district actors particularly in the identification, prelisting, selection and payment of beneficiaries was a powerful lesson learned from the evaluation findings; giving that it builds trust and local buy-in, and can promote effective accountability mechanism, while also reducing the risk of corruption. ▪ Two-way communication approach between the project team and beneficiaries in an emergency response program is imperative. Part of the problems associated with long waiting time reportedly experienced by project participants at the initial stage of the RESSNER Program was lack of communication and feedback loop between the project team and beneficiaries. Creating communication and feedback loop in Phase II of the intervention proved to have resolved the problem of long waiting time initially experienced by project participants. Hence this approach proved to be an effective component in community service delivery, and must serve as lesson learning for future emergency response programming. ▪ Developing and formalizing Consent Form for absenteeism by beneficiaries proved to be very effective in the RESSNER Program intervention, and must be observed as an important part of future cash-based programming. This approach was part of the lessons learned from the RESSNER Program, and is therefore useful as part of the solutions to address backlog issues that keeps eligible beneficiaries from receiving cash for an average period of eight weeks due to absenteeism. This approach must allow registered beneficiaries to present a next-of-kin that will represent them in times of unexplained circumstances; and therefore provide them with an opportunity to receive cash on the day of disbursement without having to wait for another 2 months before receiving backlogs. The Consent Form required the name, photo and signature of the next-of-kin, signatures of the beneficiary and community representatives. ▪ Mapping intervention areas by mobile network coverage and assessing the capacity of financial service providers (FSPs) are critical in cash-based interventions, and can avert unnecessary waste of time and resources. Lessons learned from the intervention proved that the financial service provider (Airtel) contracted to disburse cash through online payment lacked the capacity to disburse cash both online and within the recommended pay period due to poor mobile network coverage and lack of nearby cash-out points. This resulted in an unnecessary expenditure of over US$74,000 towards transfer fees in Phase I of the program intervention. While the online payment was discontinued in Phase II, savings of approximately US$39,000 was realized from transfer fees at the end of implementation. Cognizant of lesson learned, it is recommended that future cash-based programming assess the capacity of financial 38 End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone service providers, as well as mapping of target communities for mobile network coverage prior to any subsequent contract agreement with service providers. Sustainability ▪ Building synergy with existing community-based programs should be an important part of sustainability strategy in humanitarian and/or health emergency response programming. This can promote leveraging and complementarity during implementation, and may lead to continuity after phase out. ▪ Training of beneficiary households proved to be an important driver of sustainability of cash￾based emergency programs; and it is also a means of building community resilience to future shocks/ uncertainties. It is therefore recommended that every humanitarian response considers training as part of the complementary activities in its programming and implementation. ▪ The Seed Voucher modality, using local produced seeds and organizing seed fairs, where beneficiaries could choose among crop varieties is an innovative idea that should be adopted in any seed multiplication program. This encourages increase in local growers, and can serve as a means of promoting/domesticating the country’s Local Content Act at community level. Beneficiary targeting and further research ▪ The RESSNER Program intervention proved that targeting more female-headed households in cash-based programming increases the chances of more women to make independent decisions over the use of cash resources. It is therefore recommended that due considerations be made to target more vulnerable female-headed households in future cash-based emergency responses. This increases the chances of women to be more empowered in decision-making- giving that it gives them the freedom to make decisions. ▪ In line with USAID FFP partners for sharing knowledge and learning, it is recommended that joint decisions on methods, approaches, indicators, etc. needed for Monitoring and Evaluation are made prior to baseline assessments. Observably, the USAID FFP partners agreed on key sets of indicators developed in a series of working group meetings to be used for the endline evaluation. While these indicators were important particularly those needed to analyze food access, some of the indicators were observed to be absent in the baseline assessment done by some FFP partners. For instance, the CARE baseline assessment for the RESSNER program lacked Household Hunger Scale (HHS) indicators recommended by FFP partners. Comparison of endline evaluation result on HHS with existing data therefore proved difficult. The situation is however understandable, given that the cash-based program was an emergency intervention, and therefore had limited sets of indicators for monitoring and evaluation. Meanwhile, consistency is needed between indicators for baseline and subsequent follow-up assessments, as well as across FFP partners. Indicators for assessment of similar future interventions should therefore be clearly set by partners from project inception to allow for consistency and comparability. ▪ Similar future cash transfer program must ensure that community clustering should be consciously done to reduce longer walking distances to payment points. Where many communities are targeted, clustering communities within a range of about 2 miles can be reasonable to allow vulnerable beneficiaries such as disabled persons and the aged to walk long distances to pay points. For fewer communities, considering all communities for payment is otherwise recommended. i End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone ANNEXES ii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Annex I: Terms of Reference SCOPE OF WORK Final Program Performance Evaluation Under Sierra Leone Emergency Food Security Program (EFSP) Funded by USAID/FFP AID-FFP-G-15-00074 Rapid Ebola Social Safety Network and Economic Recovery (RESSNER) Project Implemented by CARE INTERNATIONAL SIERRA LEONE iii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone A. Background The Ebola Virus Disease (EVD) in West Africa began in Guinea in February 2014 and quickly spread to neighboring countries Liberia and Sierra Leone. Once the disease had entered Sierra Leone the lack of basic infrastructure including facilities, logistics, and communications resulted in a rapid spread of the virus to every district in the country. In response to the sudden and rapid spread of the disease, the Government of Sierra Leone declared a State of Emergency in July 2014, and restrictions on all types of public gatherings and the closure of schools and markets were codified into law. During this emergency period, the movement of people was restricted, households and villages were quarantined, farmers were unable to attend to their farms, and small business owners were unable to conduct their business as usual or resupply their stocks of goods. Due to their inability to pursue their normal livelihood activities during the state of emergency, many households lost their basic livelihood resources during the EVD emergency and were unable to recover and stabilize themselves to pre￾Ebola levels once the State of Emergency was lifted. As evidence, in September 2015 a joint WFP FAO Comprehensive Food Security & Vulnerability Analysis reported that 54.2% of rural households reported less income than previous years. Following the EVD emergency, other important socio-economic factors, such as healthcare systems and local markets, also faced slow progress toward recovery. By the time Sierra Leone was declared Ebola free by the World Health Organization on November 7, 2015, the country had experienced 14,601 cases of EVD resulting in 3,955 fatalities. To address these economic and social impacts of Ebola in Sierra Leone, USAID/FFP awarded funding to CARE International in August 2015 to implement an Ebola response and recovery program called Rapid Ebola Social Safety Network and Economic Recovery Program (RESSNER), with direct unconditional cash transfers to extremely poor households affected by EVD. B. Purpose of the evaluation The purpose of the performance evaluation is to provide answers to a specific set of research questions developed to investigate the extent to which the Rapid Ebola Social Safety Network Economic Recovery (RESSNER) project contributed to restoring food security, improve economic recovery and reducing the negative coping mechanisms of EVD affected participating households in Tonkolili and Bombali districts. In addition, the evaluation will also determine the effectiveness and appropriateness of cash transfer as an approach towards responding to a health related emergency as an alternative to providing food items. The evaluation findings will contribute to the body of knowledge concerning the use of direct cash transfers rather than food/commodity distribution as a recovery strategy following these types of extreme health emergencies that have widespread impacts on local and national economic and social structures. It will also document learnings on CARE’s implementation approaches, and where appropriate, inform future programming of similar intervention. It is expected that the key beneficiaries will be consulted during the evaluation which include the direct households who benefitted from the Unconditional Cash Transfer and Conditional Seed Vouchers, the Community Identification Committee (CIC), as well as the community leaders. CARE staff and implementing partners will be requested to provide information relating to the role and general impression of the project. Other respondents to be reached will include stakeholders including the National Commission for Social Action (NaCSA) the district council and the Anti￾Corruption Commission. The evaluation will aim to assess the extent to which the project has achieved its intended objectives. It will also assess how relevant, efficient and sustainable the project was in converting resources into activities, outputs and outcomes and the changes it brought about for direct beneficiaries. The evaluation will also reflect on the results and from the analysis, draw out lessons to be learned from the project. The specific objectives of the evaluation will include the following: iv End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 6. Assess comparative achievements (to the baseline) and changes made by the project 7. Determine efficiency and effectiveness of project implementation. 8. Draw out lessons to be learnt- what worked well and what didn’t work so well, why and how they were mitigated 9. Assess the interaction and impact of the project on the various community and government bodies and stakeholders 10. Explore community’s acceptance, perceptions and attitudes towards the project. C. RESSNER Project overview RESSNER project was implemented in two Phases: Phase 1 implemented between August 2015 – December 2016, and Phase 2 implemented between January – December 2017. The project is implemented in 9 Chiefdoms in Tonkolili and Bombali districts. Bombali and Tonkolili Districts were two of the districts severely affected by the Ebola epidemic, which had severe and negative impact on the economic, livelihood and food security of the population. Many households experienced difficult and increased food security and livelihood due to the restriction of movements and other measures adopted to fight the EVD outbreak. The situation resulted in adopting negative coping mechanisms to survive. Project Goal, Objectives and Intermediate results: The main goal of the project is to support the local economic recovery of EVD affected communities in Bombali and Tonkolili districts. Specific objective: to reduce the negative coping strategies (such as selling household assets, child labor, begging, and skipping meals, etc) of 4,500 extremely poor HHs in EVD affected communities. The main outcomes of the project includes: a) Increased access to cash for the purchase of basic food items for selected household beneficiaries b) Improved ability of household beneficiaries to recover from the impact of the EVD. The intervention of the project include the following: Phase 1: Unconditional cash transfer of USD 30 per month to 4,500 Households (HHs) (27,000 individuals, estimated 6 per HH) for the months of January to October 2016. Phase 2: Unconditional cash transfer of USD 30 per month to 3,600 HHs (21,600 individuals) for the months of January to October 2017; and one off conditional seed voucher of USD 30 and training on improved agronomic practices and crop diversification: 900 HHs (5,400 individuals); and community trainings and sensitizations on Ebola￾related health and Nutrition, sensitization and hygiene for the entire 4,500 households. The evaluation will target samples from this three categories of households. That is, those targeted by the first phase of the project, households targeted by phase two and those that were targeted by both phases. For Phase 2 beneficiaries, out of 3,600 HH cash beneficiaries, 533 HHs were from Phase 1 that were carried over to Phase 2. The 900 HH beneficiaries of conditional seed voucher and training on crop diversification and agronomic practices are independent of the 3,600 unconditional cash beneficiaries. D. Evaluation Questions 1. To what extent did the RESSNER project contribute towards restoring food security, improve economic recovery and reducing the negative coping mechanisms. In answering this question, the evaluator is expected to address, but not limited to the following: a. reduced negative coping strategy index (measured using a harmonized tool shared between other USAID funded FFP ebola recovery projects & using this indicator) b. Household hunger scale index, household dietary diversity score v End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone c. Participation of participating households in other economic or livelihood activities e.g. VSLAs, petty trading, livestock etc (this can also serve as a proxy to measure the sustainability of the project outcomes). Expenditure on food and non-food (measured using a harmonized tool shared between other USAID funded FFP ebola recovery projects & using this indicator) 2. To what extent were the processes and procedures used by the project for delivering cash transfer and seed voucher to participating households effective, efficient and sustainable? In answering this question, the evaluator should address the following: a. Accountability, timeliness and cost of delivering the cash and non-cash assistance to participating households. b. Assess the participants’ perceptions about the cash transfer and non-cash delivery processes and procedures c. Explore the gender implications of cash transfer and non-cash assistance d. What can we learn from the approaches, processes and procedures used? 3. To what extent did the collaboration and coordination with public and private sector stakeholders contribute to effective delivery of cash assistance during the emergency and recovery phases? In answering this question, the evaluator should address the following: a. Compliance with national standards, policies on social safety net b. Contribution to the President’s priorities on social protection c. Connectedness and coherence with other actors such as NGOs, Donors, UN organizations (e.g. WFP, FAO, UNDP), multilateral agencies, government agencies (e.g. NaCSA, Min of Social Welfare, etc. 4. To what extent was gender concerns addressed throughout the program? In answering this question, the evaluator should address the following: a. Addressing gender inequalities b. Gender relationships within the households and the wider communities. c. Power relations and relationships between men and women as a result of the project d. What can we learn about the intended and untended outcomes (that is negative and positive) related to gender because of the project? 5. To what extent did the cash transfers have unintended outcomes (positive or negative) in inter-community and intra-community relations, between cash receiving households and non-cash receiving households? E. Scope of the evaluation The evaluation covers exclusively the Rapid Ebola Social Safety Network and economic Recovery (RESSNER) Project implemented from August 15, 2015 to December 31, 2017. The evaluation will focus on the geographical areas of the 9 chiefdoms in the Bombali and Tonkolili Districts. The chiefdoms in Bombali include: Biriwa, Libisaygahun, Makari Gbanti, Paki Masabong and Safroko Limba. The chiefdoms in Tonkolili include: Gbonkonlenken, Konike Sanda, Konike Barina and Kholifa Mabang. 1. Methodology The evaluation will use mix methods of quantitative and qualitative approach to obtain information from the targeted respondents. These methods is envisaged to solicit relevant information from key stakeholders that will be used to assess the performance of the project. Focus Group Discussion (FGD) and Key Informant Interviews (KII) will be used during the evaluation. CARE will be flexible to discuss potential methodologies with the consultants where possible. vi End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone The targets respondents will include direct project beneficiaries including household heads who benefitted from the Unconditional Cash Transfer and Conditional Seed Voucher. For the unconditional cash transfer, the evaluation methodology and sampling will include three categories of beneficiaries including households targeted by phase 1 only, those targeted by Phase 2 and those reached by the two phases. Other respondents will include RESSNER staff and local NGO implementing partner staff directly involved in the implementation of the project. Other stakeholder including government agencies involved in the project will be consulted as well. Desktop Review will done in order to get a better understanding of the project. Existing documentation and reports relating to the project will be reviewed thoroughly. Project documents and reports including proposal, quarterly project reports, Post Distribution Monitoring Reports etc. will be provided to the consultant by the appropriate project staff. 2. Deliverables: The full process of final evaluation will commence upon award with development of the tools and the work plan for the final evaluation. The evaluator(s) will prepare: 1) an evaluation work plan (including evaluation methods and tools etc.); and 2) an evaluation report in accordance with standards identified in the award document. Below are the main deliverables of the final evaluation: a) An inception report (prior to any field work) including a detailed evaluation methodology, sampling, evaluation question matrix, proposed data collection tools and analysis approach, and evaluation work plan (with corresponding timeline). The evaluation plan will include the methodology and tools and the logistics, staffing, and field work plans. b) A presentation of initial findings to the Sierra Leone staff (using PowerPoint) for discussion and feedback prior to development of the draft report. The evaluation will also be prepared to present to the host government, USAID / FFP and/or others as requested by CARE prior to leaving Sierra Leone. c) Evaluation first draft report to CARE for internal review d) Evaluation final draft report due to CARE e) Submission of database and datasets in compliance with USAID Food for Peace emergency project requirements. 3. Roles and responsibilities The entire evaluation process will be led by the consultant. The lead consultant will be responsible for identifying recruiting and training enumerators preferably within Sierra Leone and from the project districts. The consultant will also be responsible for developing a sound research methodology, planning and conducting a consultative evaluation, managing the data collection, as well as writing up the reports, presenting the findings and recommendations. An evaluation steering committee made up of the Project Manager, Senior Programme Officer, Livelihood Adviser and the Project MEAL Coordinator, and the key contact will be the Project Manager. . The group will guide and review the inception and draft reports. The table below provides more details Consultant RESSNER Project Manager CARE Country Office 1. Develops an inception report, detailing the methodology￾stakeholders to be interviewed, tools to be developed, time frame for the evaluation 1. Provides all required background materials to the consultant in a timely manner. 1. Review Evaluation consultant’s qualifications or specialized knowledge or experience required. 2. Holds the overall management responsibility of the evaluation, including designing and carrying out the evaluation, drafting the 2. Manages the consultancy contract; monitor adherence to specified deadlines; facilitating 2. Read and provide comments on the proposal plans submitted by the consultant (especially the proposed vii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Consultant RESSNER Project Manager CARE Country Office final report and debriefing the project team and key stakeholders. access to required information, communities and stakeholders. research methodology, the information gathering techniques used and the suggested target dates); 3. Liaises with Project staff throughout the process, providing weekly updates and seeking their input and advice where necessary. 3. Provides guidance throughout all phases of execution, approving all deliverables, and facilitating access to any documentation (or any person) deemed relevant to the evaluation process. 3. Review and comment on analysis and draft report submitted by the evaluator i.e. preliminary reports and the final report, 4. As a condition of entering into a consultancy agreement, the evaluator and research assistants must sign the Protocols and Policies and abide by the terms and conditions thereof. 4. Provides feedback to draft data collection tools and reports F. Timeframe: The final evaluation will commence by 2nd week of September 2017 for maximum of 25 working days and the first draft report should be made available followed by 15th October 2017. The final report expected by 25th October 2017. Due Date Activity 28 August 2017 Advertisement sent out. 11 September 2017 Review of Evaluator(s) 12 September 2017 Selected Evaluator(s) and agreement process finalized Consultant will begin the work and review all project documents and reports 18 September 2017 Submission of inception report which includes survey workplan, data collection tools/questionnaire, sampling determination, logistics, etc 19 September 2017 Review of inception report by CARE and/or USAID-FFP 19 September 2017 Feedback on inception report to the consultant 21 September 2017 Evaluation team incorporates comments and submit final inception report and evaluation tools 22 September 2017 Discussion with CARE Project team and Field preparation 25-29 September 2017 Field work/data collection and analysis 11 October 2017 Presentation of preliminary findings with CARE RESSNER staff and management; USAID-FFP and other relevant partners 15 October 2017 Submission of draft report for internal review 17 October 2017 CARE provides feedback on draft report for incorporation 25 October 2017 Submission of final report to CARE 31 October 2017 USAID provides feedback on the report 15 November 2017 Submission of Final Evaluation report G. Proposal Preparation and Submission Requirements 1. Instructions for Proposal Preparation The selection committee will evaluate the applications/proposals based upon their written technical and cost proposals. Each section will be evaluated according to the criteria for evaluations in Section B. Applicants are expected to examine the specifications and all instructions in the RFP. Failure to do so is at the Applicant’s risk. Interested Applicants must provide the following: viii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone a. Capability and Technical Experience Statement Demonstrate capabilities and technical experience by providing the following: • Organization Overview: Give a brief, general overview to demonstrate the Evaluator’s overall qualifications to fulfill the requirements of the Evaluation • Capabilities Statement: Provide a comprehensive discussion of the Evaluator’s capabilities with emphasis on the team that will be assigned to the work detailing technical area(s) covered and experience (CVs should be included as an attachment to the technical proposal), Level of resources and support anticipated from head office and any current/recent experience conducting similar evaluations. • Evaluator’s understanding of the SOW: Description expressing understanding of SOW. • Project Approach: Provide a detailed proposal (no more than 5 pages) explaining how the Evaluator intends to achieve the expectations set forth in Section B, including specifications of any data collection equipment required. • Partner/Subcontractor: Give a description of any partner organization or subcontractor that you might contract with and a description of the division of level of effort and responsibility between your organization and the partner or subcontractor. • Samples of the two most recent study reports authored by the Evaluator(s). b. Project Staffing Identify the project staffing and the percentage of the time each will spend on this activity. Include no more than a halfpage biosketch for each individual considered essential for the successful implementation of this contract. Desired profile of the evaluation team: • Prior experience conducting rapid assessments of USAID/FFP food security and cash transfer emergency recovery programs to evaluate the efficiency and effectiveness of program systems, processes, and procedures. • Lead consultant should have at least 8 years’ experience evaluating emergency/development programs with some of them including cash transfers. • Team will include relevant expertise in statistics with analytical skills in evaluations and impact assessment of humanitarian programs particularly cash-based programs Including competency on the utilization of Statistical Packages Software such as SPSS or STATA. • Experience with the use of focus group and key informant interviews as the primary tools for evaluative research • Ability to effectively integrate quantitative data with the results of qualitative research • Knowledge of USAID/FFP regulations, evaluation procedures, and evaluation reporting c. Cost Proposal Applicant/Evaluators will submit a proposed budget with their proposals in a separate, sealed envelope (or separate file, if submitting via email) labeled “Budget Proposal.” The proposed budget will have sufficient detail to allow evaluation of elements of costs proposed. Budgets should be submitted in the currency in which your organization is located and will be paid; please label your budget with the name of the currency. CARE reserves the right to request any additional information to support detailed cost and price. Applicant/Evaluator will also submit detailed budget notes accompanying their detailed budget. Proposals submitted without budget notes may not be evaluated. ix End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Budget for the whole consultancy service will be presented in 3 parts. The first part will include the consultant cost (Fees, per diem for field (hotel, meals & incidentals), transportation (overseas & local), and overhead if any). Second part is for enumerators cost, which will be provided on a “per enumerator per day.” The third part should present costs for supplies and logistics (for example, paper, photocopy costs, internet connectivity, equipment for anthropometric measurements, venue & food cost for any training). d. References Please include three client references and contact information. References should have worked with you or your organization within the past two years in connection with the countries or regions (and if possible, subject matter) applicable to this RFP. H. Instruction for Submission of Proposal The technical and cost proposal shall be sent via email to: Hortense.Kessambo@care.org and clearly indicate in the subject line the Project name (page 1). All applications/proposals shall be received no later than the submission deadline by COB of 5 September 2017. I. Criteria for Evaluation CARE International will evaluate proposals based on a best-value determination; Applicants should submit their most competitive price proposal. Proposals will be evaluated using the following criteria: The Applicant will be selected based on the evaluation of both technical and financial scores. Evaluation committee members will review the technical proposal first and then will review the financial proposal of technically eligible firms. The ratio of technical to financial scoring will be 70:30. The following are the evaluation criteria for the Applicant’s technical proposal. The technical proposal points will be broken down as follows: • Applicant’s previous experience regarding donor, similar work and country (Sierra Leone): 20 pts • References recommendations: 10 pts • Appropriateness of sample methodology presented: 15 pts • Team member composition and relevancy: 20 pts • Time-bound rollout plan: 10 pts • Data collection, quality assurance, data management plan & smart tools/sound equipment usage: 15 pts • Analysis plan: 10 pts Cost proposal reviews will consider: The cost proposal will be evaluated on an overall scale of 100 points. • Cost reasonableness and allowability of the proposal • Cost-effectiveness to achieve deliverables • Compliance with CARE International and USAID policies and prescribed payment schedule; and • Appropriate structure of cost proposal, which should include: Summary of Costs, Breakdown of Staff Remuneration, Travel and DSAs, miscellaneous, and overhead costs if any, as well as detailed budget notes depicting their budget. The evaluation committee will review the technical proposal based upon the technical criteria listed above. The cost proposals will be reviewed to ensure they are complete and free of computational errors. The committee will also assess the reasonableness of costs and the cost-effectiveness of the budget, and will determine whether the costs x End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone reflect a clear understanding of project requirements. A contract will be offered to the responsible Applicant whose proposal follows the instructions and is judged to be the most advantageous to CARE International. J. Contract and payments CARE International Sierra Leone will sign a consultancy contract with the Evaluator. 30% of the total contract sum will be paid upon submitting satisfactory inception report and draft tools and the remaining amount upon submission and approval of the final report and all deliverables. CARE International will only pay for the entire consultancy fee (as per the contract) after the full submission and acceptance of the final report, corresponding annexes and other related deliverables by CARE International in Sierra Leone Annex II: Checklist for USAID FFP Partners’ Recommended Questions Questions Covered Yes No Evaluation Question 1 - To what extent are households that participated in the EFSP achieved food security, and improved resilience, as measured by the household hunger scale, household dietary diversity score, reduced coping strategies index and income and livelihood? x 1a) What are some of the activities you have undertaken to keep or grow the cash transfer money? x 1b) Do you believe these activities will continue when you are no longer receiving cash? x 1c) Are you purchasing goods and services at prices similar to before cash transfer? If not, why do you believe prices have increased? x 1d) Do you believe the receipt of cash was effective in reducing negative coping strategies as a result of Ebola? x 1e) How has the EFSP program prepared you to cope with future shocks? x Evaluation Question 2 - To what extent were the systems, processes and procedures used by EFSP for delivering cash transfers and non-cash assistance to participants were effective, efficient, and sustainable during the period of program implementation? x 2a) What mechanisms were used to ensure that the right beneficiaries were targeted in all phases? x 2 b) Was there a change in approach or modalities due to emerging conditions/situations? If yes, can you please describe? x 2c) During implementation, what were the mechanisms used to ensure beneficiaries, the public, and management were fully informed about what was happening at field level? x 2d) What was your experience with the cash distribution process such as waiting time, walking distance, safety and grievance redress? x 2e) Have you ever been pressured by local authorities (Town/Paramount chiefs, District Councilors); CICs, extended family or community members to give money you did not want to give? If so, how did you handle this? x 2f) Has there been any other kind of conflict over your receipt of cash? x 2g)Where there conditions that made you not to receive the cash at any point in time? How was this situation handled? How long did it take for the partner to respond to your request? x Evaluation Question 3 - To what extent did the collaboration and coordination with public and private sector stakeholders result in enhanced capacity to deliver cash transfers as a form of social protection to vulnerable households and contribute to effective delivery of food assistance during the emergency and recovery phases? x 3a) How did Implementing Organization coordinate activities with national and local government authorities? x 3b) How did implementing partners address the issues of parallel systems and structures? How did this impact results? x 3c) How did Implementing Partner coordinate with other FFP Implementing Partners? How did that collaboration enhance achievement of results? x 3d) How did other interventions in targeted communities impact results (leveraging funds, enforced or complement implementation, and sustainability)? x xi End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Annex III: Personal Interviews Questionnaire # Question Response Code SERIE 100: SOCIO-DEMOGRAPHIC CHARACTERISTICS 101 Are you a beneficiary of RESSNER? [1] Cash transfer Beneficiary [2] food voucher beneficiary [3] Beneficiary with extended cash benefit 102 Age of respondent Age of respondent………………………………………………. 103a Sex [1] Male [0] Female 103b What is the disability status of respondent [1] Non-disabled [2] Disabled 104 Relationship with the household head (do not interview respondent below 18 years) [1] Head of household [2] Spouse to household head [3] Daughter to household head [4] Son to household head [96] Other relative (specify…………………………………………………………………..) 105 Are you or any member of your household engaged in any form income generating activities? [10 Yes [0] No (If ‘No’ Go to 107) 106 What is (are) currently the major source (s) of income generated by this houses? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Regular trading/business [b] crop farming [c] Animal farming [d] Fishing [e] VSLA/ROSCA [f] Civil servant [g] Others (specify…………………………………………………………………) 107 How many people (including you) were in your household before the CARE cash transfer/seed voucher project was introduced in this community? No. of persons…………………………………… Introduction: Hello, my name is____________________. I am collecting information on behalf of CARE International in Sierra Leone on the impact of cash transfer project to poor households in response to the negative effect of the Ebola Virus Disease outbreak on the livelihoods of your households. In particular, CARE would be interested to know about the welfare of your household during the outbreak, how the cash transfer project directly or indirectly impacted on your daily living after the outbreak. Kindly note that your sincere responses to the questions I would be asking will help CARE International in Sierra Leone to better programme their livelihoods intervention in Bombali and Tonkolili district. Be assured that your name will not be disclosed in terms of any sensitive response(s), but would be promoted where ‘golden quotes’ are essential. I would therefore appreciate it most, if you could spare some part of your busy schedule to respond to my questions. Thank You! Respondent’s Name_______________________ Address/ Community name________________ Mobile #____________, ______ Chiefdom___________________ District______________________ Start time of interview__:__ End time__:__ Interviewers Name______________________________ Signature of Supervisor________________, Date__/__/ 2017 xii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 108 How many people (including you) were in your household during the CARE cash transfer/seed voucher project implementation in this community? No. of persons…………………………………… 109 What is the current number of male and female members of this household? [a] No. of males………………………………… [b] No. of females…………………………... 110 What is the current number of household members (including respondent) by type in your households? (write ‘o’ against the category that is not in the household) Respond to All options (Kindly write the numbers in figures) [a] No. of children (less than 5 yrs) in the household [b] No. of children (5-17 yrs) in the household [c] No. of adults (18 years and above) in the household 111 What is your level of education? [1] None [2] Primary [3] Secondary [4] Tertiary SERIE 200: COPING STRATEGIES USED IN RESSNER PROJECT AREAS 201 Did your household do any of the following in the last seven days due to food shortage or lack of money? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Sold our last livestock [b] Sold or barter our agricultural tools [c] Consumed or sold our reserved grains for farming [d] Reduced our daily food consumption [e] Consumed or sold our remaining poultry [f] Sold or barter our cooking utensils/amenities [g] Sold or barter our furniture [h] Used our business income and/or savings [i] Loaned money or food items from elsewhere [j] Begged food items and/or money from friends/relatives [k] Relied on in-kind/in-cash remittances from elsewhere [l] Suspended school attendance by children [m] Others (specify………………………………………………………………….) 202 During the last 30 days, how frequently did your household use the following strategies on a weekly basis in order to access food? Respond to All options: For the first response use the following scale: 1= Never 2= Seldom (1 day a week) 3=Once in a while (2 days per week) 4= Pretty often (3-6 days per week) 5=Almost every day For the second response of the same item us the following scores aligned to each scale attributed to the item. Item [1] [2] [3] [4] [5] a1. 2.3 5.6 6.9 9.2 11.5 b1. 2.9 5.8 8.7 11.6 15.5 c1. 3.7 7.5 11.1 15.6 18.5 d1. 2.3 5.6 6.9 9.2 11.5 e1. 3.0 6.0 9.0 12.0 15.0 f1. 2.3 5.6 6.9 9.2 11.5 g1. 3.1 6.2 9.3 12.5 15.5 h1. 3.1 6.2 9.3 12.5 15.5 i1. 3.0 6.0 9.0 12.0 15.0 j1. 3.5 7.0 10.5 15.0 17.5 k1. 3.1 6.2 9.3 15.5 15.5 l1. 2.3 5.6 6.9 9.2 11.5 [a] Limit portion sizes at meal times? xiii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone [a1] what is the corresponding score as noted above? [b] Reduce the number of meals per day [b1] what is the corresponding score as noted above? [c] Skip entire days without eating? [c1] what is the corresponding score as noted above? [d] Borrow food or rely on help from a friend or relative? [d1] what is the corresponding score as noted above? [e] Rely on less expensive or less preferred foods? [e1] what is the corresponding score as noted above? [f] Purchase food on credit/ take a loan to buy food? [f1] what is the corresponding score as noted above? [g] Gather unusual types / amounts of wild food or hunt? [g1] what is the corresponding score as noted above? [h] Harvest immature crops? [h1] what is the corresponding score as noted above? [i] Send household members elsewhere? [i1] what is the corresponding score as noted above? [j] Send household members to beg? [j1] what is the corresponding score as noted above? [k] Restrict consumption by adults so children can eat more? [k1] what is the corresponding score as noted above? [l] Rely on casual labor for food? [l1] what is the corresponding score as noted above? 203 During the last 3 months, did you adopt any of the following coping mechanisms? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Avoid spending on health care in order to buy food? [b] Reduce expenditure on education/ withdraw children (or other household members) from school in order to buy food? [c] Reduce expenditure on agricultural and livestock inputs in order to buy food? 204 How many meals were served/ consumed by members of your household in the last 24 hours? (write numbers) [a] Children less than 5 years (<5 years) [b] Children (5-17 years) [c] Adults 18 years and above SERIE 300: EXPENDITURE ON FOOD, HEALTHCARE, EDUCATION AND OTHER NON-FOOD ITEMS/ACTIVITIES (FOR SEED VOUCHER BENEFIAIRIES ASK QUESTIONS 303-309) Hint: If the respondent could not easily tell amount spent due to illiteracy, kindly use the ‘beans scenario’ to calculate the household expenditure. That is, present 10 beans and the spending visual aid with pictures. Ask the respondent to do the following: ‘If these beans represent all the money you received last time: take out the amount of the beans that represent the amount of money you spent on food, healthcare and education pictures’. Tell respondent that there is no problem if the household did not spend at all. Note: Any one of the 10 beans represents 10%. Use this percentage to calculate the amount spent per each item. E.g if 1 beans is placed on the food picture, the amount spent should be calculated as 10% x 130,000=Le 13,000 301 How much of the money received from the last cash transfer spent on: Write ‘0’ if no money was spent on the item [a] Food items………………………………………….. [b] Healthcare………………………………………….. [c] School related needs (education)………………………………….. 302 If you did not spend all the money received from the last cash transfer, please state three other items (in order of priority) towards which this household used the money received Indicate with 1,2 and 3 for the first, second and third high ranking item on the list and ‘0’ against all other items [a] Agricultural inputs [b] Farm labour [c] Income generating activities (Trading/business) [d] Animals [e] Home repair [f] Savings (VSLA, ROSCAs) [g] Debt repayment xiv End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone [h] Cultural (wedding, funeral, holidays, etc) 303 What was the total income earned/ generated by this household in the in the last 30 days (kindly include cash received and spent)? Le………………………………………………….. 304 What were this household’s sources of cash earned/ generated in the last 30 days? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Sales from crops produced by household [b] Sales from large livestock (cattle) [c] Sales from small livestock (goats, pigs, sheep, etc) [d] Sales from poultry (chicken, ducks, pigeons, etc) [e] Sales from regular trading/business [f] Shares from savings (VSLAs, ROSCAs) [g] Loans [h] Remittances [i] sales of other household assets (TV, generator, radio, etc) [j] Others (specify…………………………………………………………………) 305 If loan is part of the total income earned/generated by this household in the last 30 days, how much of the total income was loan Amount of loan received in the last 30 days…………………………. 306 How much of the income generated by the household in the last 30 days spent on: Write ‘0’ if no money was spent on the item [a] Food items………………………………………….. [b] Healthcare………………………………………….. [c] School related needs (education)………………………………….. 307 How much of the income generated by the household in the last 30 days spent on other non￾food items? Expenditure on other non-food items: Le…………………………………. 308 Had health- and/or education-related expenses affected your daily food consumption in any way after the cash transfer? [1] Yes [0] No (If ‘No’ Go To question 401) 309 If ‘yes’ to question (308) in which instance/s has health and/or education expenses affected your food consumption pattern? ……………………………………………………………………………………………………………………………………… ……………………………………………………………………………………………………………………………………… ……………………………………………………………………………………………………… SERIE 400: HOUSEHOLD HUNGER AND DIETARY DIVERSITY 401 During the last 30 days, was there a time, because of lack of money or other resources: Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’ and 97 for ‘Don’t know’) [a] You or others in your household worried about not having enough food to eat? [b] You or others in your household were unable to eat the kind of food preferred? [c] You or others in your household ate only a few kinds of foods? [d] You or others in your household eat some foods that you really did not want to eat? [e] You or others in your household ate less (or reduce food consumption) than you thought you should? [f] You or others in your household ate fewer meals in a day? [g] Where there was no food of any kind in your household? [h] You or others in your household went to sleep at night hungry? [i] You or others in your household go a whole day and night without food? 402 During the last 30 days, how many days because of lack of money or other resources…. (Use numbers 0-30 to answer number of days. Write ‘0’ if respondent say ‘never’ and ‘NA’ for not applicable) [a] You or others in your household worried about not having enough food to eat? [b] You or others in your household were unable to eat the kind of food preferred? [c] You or others in your household ate only a few kinds of foods? xv End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone [d] You or others in your household eat some foods that you really did not want to eat? [e] You or others in your household ate less (or reduce food consumption) than you thought you should? [f] You or others in your household ate fewer meals in a day? [g] Where there was no food of any kind in your household? [h] You or others in your household went to sleep at night hungry? [i] You or others in your household go a whole day and night without food? 403 What category of food did your household consume in the last seven days? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Cereals………………………………………………………………………………. [b] Roots and tubers……………………………………………………………. [c] Vegetables………………………………………………………………………. [d] Fruits……………………………………………………………………………….. [e] Meat and poultry……………………………………………………………. [f] Eggs…………………………………………………………………………………… [g] Fish and sea food…………………………………………………………… [h] Pulses/legumes/nuts……………………………………………………… [i] Milk and milk products…………………………………………………… [j] Oils and fats…………………………………………………………………….. [k] Sugar and honey………………………………………………………………. [l] Miscellaneous (Maggie, onions,……………………………………….. 404 What is the number of days that each of the following food categories of food was consumed by your household in the last seven days? [a] Cereals………………………………………………………………………………. [b] Roots and tubers……………………………………………………………. [c] Vegetables………………………………………………………………………. [d] Fruits……………………………………………………………………………….. [e] Meat and poultry……………………………………………………………. [f] Eggs…………………………………………………………………………………… [g] Fish and sea food…………………………………………………………… [h] Pulses/legumes/nuts……………………………………………………… [i] Milk and milk products…………………………………………………… [j] Oils and fats…………………………………………………………………….. [k] Sugar and honey………………………………………………………………. [l] Miscellaneous (Maggie, onions,……………………………………….. 405 How many times did this household eat on average in the last 24 hours? [1] Once a day [2] Twice a day [3] Three times a day [4] More than three times a day 406 Did you or any member of this household ever receive any information or training on nutrition, savings and business development etc during the CARE cash transfer project implementation? [1] Yes [0] No (If ‘No’ Go To question 501) 407 If ‘yes’ to question (406), what particular information or training did you receive? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Training and/or information on crop diversification [b] Training and/or information on vegetable gardening [c] Training and/or information on seed multiplication [d] Training and/or information on post-harvest management [e] Training and/or information on nutrition [f] Training and/or information on health, sanitation and hygiene [g] Training on business development/Savings [h] Others (specify……………………………………………………………………) xvi End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone SERIE 500: PROJECT IMPACT ON HOUSEHOLD LIVELIHOODS AND LOCAL ECONOMY 501 What impact do you think the CARE cash transfer project has had on members of this household (e.g income status/savings, economic empowerment, social status/decision making, etc)? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Project has contributed to increased total household income [b] Project has contributed to increased savings by household members [c] Project has led to business development by some household members [d] Household now has land asset for building due to project [e] Household now has land asset for agricultural purpose [f] project has led to business development by some household members [g] project led to increased sales of household business commodities [h] Some or all members of the households have more hired labor opportunities [i] The household have engaged in large-scale production of farm produces than before [j] Others (specify)………………………………………………………………….. SERIE 600: BENEFICIARIES PERCEPTION OF CARE CASH TRANSFER PROJECT (FOR CASH BENEFICIARIES ONLY 601 What has been your most important investment from CARE cash transfer assistant to this household? Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Used cash for daily food consumption [b] Invested in the education of children [c] Invested in health [d] Purchased land for building [e] Purchased land for agriculture [f] Shared cash among household members [g] Invested in business [h] Invested in savings groups (SGs) [i] Other investments (specify………………………………………..) 602 What significant impact/ benefit have your household had from the investments you mentioned in question (601)? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………. 603 Now that the cash transfer assistance has ended, what activities are you undertaking to ensure that your household copes without further assistance? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… …………………………………………………………………… 604 If you are currently undertaking these activities stated in 603, do you think this will continue? 605 How has the cash transfer intervention prepared you to cope with future shocks? 606 What most significant change have you experienced in your life as a result of the cash transfer assistances (e.g social status/decision-making, economic empowerment, increased income, etc)? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ……………………………………………………………………………….. 607 In your opinion, do you think the cash transfer for food was the most appropriate method adopted for this community? [1] Yes [0] No (If ‘Yes’ Go To’ question (609)) 608 If ‘No’ to question (607) what modality of assistances would you prefer for any food aid assistance to reach you? [a] Food vouchers [b] Direct food supply 609 Which activities of the cash transfer did you like most? 610 What do you think was not right with the cash transfer assistance? xvii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone 611 How do you think future interventions should be done address the problems with the cash transfer assistance? 612 Were there conditions that made you not to receive the cash at any point in time? [1] Yes [0] No (If ‘No’ Go To 615) 613 If ‘Yes’ to 612 how was this situation handled? 614 If ‘Yes’ to 612 how long did it take for the partner to respond to your request 615 Have you ever been pressured by local authorities (Town/Paramount chiefs, District Councilors); CICs, extended family or community members to give money you did not want to give? [1] Yes [0] No (If ‘No’ Go To 617) 616 If ‘Yes’ to 616 how did you handle this? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 617 What was your experience with the cash distribution process such as waiting time, walking distance, safety and grievance redress? 618 How satisfied are you with the support you received from the CARE RESSNER project? [1] Very satisfied [2] Satisfied [3] Somehow satisfied [4] Not satisfied (If very satisfied, satisfied or somehow satisfied Go To 701) 619 If you are not satisfied, why? ………………………………………………………………………………………………………………… SERIE 700: GENDER, INTER- AND INTRA-HOUSEHOLD AND COMMUNITY RELATIONS 701 Did the CARE cash transfer project change the pattern of control of cash resources and expenditure in your household? [1] Yes [0] No 702 Who is the main decision-maker over the use of cash currently earned by your household? [1] Self [2] Partner/spouse [3] Self and partner/spouse jointly [4] Other household member [5] Self and other household member (s) [6] Partner/spouse and other household member(s) 703 Do you or any member of your household have any of the following financial assets Respond to All options (Write 1 for ‘yes’ and 0 for ‘No’) [a] Agricultural land [b] Commercial motorcycle [c] Large livestock (Oxen, cattle) [d] Small livestock (sheep, goat and/or pig) [e] Poultry (Chicken, ducks, pigeon, etc) [f] Power tiller/ tractors [g] Rice mill or other food processing machines [h] Building for rental 704 If the household has one or more of any of the following financial assets, who would you say owes most of the asset? Write the following code for any of the following items: [1] Self [2] Partner/spouse [3] Self and partner/spouse jointly [4] Other household member [5] Self and other household member (s) [6] Partner/spouse and other household member(s) [a] Agricultural land [b] Commercial motorcycle xviii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone [c] Large livestock (Oxen, cattle) [d] Small livestock (sheep, goat and/or pig) [e] Poultry (Chicken, ducks, pigeon, etc) [f] Power tiller/ tractors [g] Rice mill or other food processing machines [h] Building for rental 705 If the household has one or more of any of the following financial assets, who would you say can decide to sell most of the asset? Write the following code for any of the following items: [1] Self [2] Partner/spouse [3] Self and partner/spouse jointly [4] Other household member [5] Self and other household member (s) [6] Partner/spouse and other household member(s) [a] Agricultural land [b] Commercial motorcycle [c] Large livestock (Oxen, cattle) [d] Small livestock (sheep, goat and/or pig) [e] Poultry (Chicken, ducks, pigeon, etc) [f] Power tiller/ tractors [g] Rice mill or other food processing machines [h] Building for rental 706 Who makes the decision to borrow/what to do with money borrowed from any of the following: Write the following code for any of the following items: [1] Self [2] Partner/spouse [3] Self and partner/spouse jointly [4] Other household member [5] Self and other household member (s) [6] Partner/spouse and other household member(s) [a] NGO [b] Informal lender [c] Formal lender (bank, MFI, etc) [d] Friends or relatives [e] VSLA/ROSCA 707 Did the CARE cash transfer bring any conflict between members in the household or within the community? [1] Yes [0] No (If ‘No’ Go To question 801 for seed voucher beneficiaries only)) 708 If ‘Yes’ to question 707, what was the conflict? ………………………………………………………………………………………………………………… 709 If ‘Yes’ to question 708, has the conflict been settled? [1] Yes [0] No SERIE 800: BENEFICIARY PERCEPTION ABOUT THE SEED VOUCEHR (SEED VOUCHER BENEFICIARIES ONLY) 801 If this household benefited from the seed voucher and/or seed fair, do you think the seed voucher support was the best modality compared to other modalities in accessing seeds? [1] Yes [0] No (If ‘No’ Skip To’ 803) 802 If ‘yes’ to 801 what are your reasons? ………………………………………………………………………………………………………………… 803 If ‘No’ to 801 which other modalities for accessing seed did you prefer? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 804 If this household benefited from the seed voucher support, in what ways do you think this support contributed to your standard of living? ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… 805 If you benefited from the seed voucher, what has your household put in place to ensure availability of food and increased income after the seed voucher benefit ………………………………………………………………………………………………………………… ………………………………………………………………………………………………………………… ………………………………………………………………………………….…………………………… xix End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Annex IV: Focus Group Discussions Guide Questions for Focus Group Discussions Target respondents: Cash transfers beneficiaries Instruction: consider mix of respondents, including gender (male and female), disability (involve disabled persons), age (involve old people), single parents. Evaluation Question 1: To what extent are households that participated in the EFS achieved food security, and improved resilience, as measured by the household hunger scale, household dietary diversity score, reduced coping strategies index and income and livelihood? a) What are some of the activities you have undertaken to keep or grow the cash transfer money? b) Are you sure these activities will continue even when the cash transfer support has ended? [1] Yes [0] No c) What is the reason for your response to 1b)? d) Did you experience any change in prices of goods and services you purchased during the cash transfer compare to prices before cash transfer? [1] Yes [0] No, If ‘yes’, e) If ‘yes’ to 1d) what change did you observe (increase or decrease)? What do you think was responsible for the change mentioned? Which goods and services were mostly affected by the change and why? f) Do you think the receipt of cash was effective in reducing the negative ways you used to survive as a result of the Ebola? [1] Yes [0] No, If ‘Yes’ how has the cash received helped to reduce the negative ways you used to survive during Ebola? If ‘No’ why do you think receiving cash did not help to reduce the negative ways you used to survive during Ebola? g) How has the cash transfer program prepared you to cope with future shocks? h) What non-cash component (e.g training, awareness raising, etc) do you think has also helped improve the way you use the cash received? In what ways has this improved the way you use the cash received? If the non-cash components were not effective, what are the reasons for failure? Evaluation Question 2: To what extent were the systems, processes and procedures used by EFSP for delivering cash transfers and non-cash assistance to participants were effective, efficient, and sustainable during the period of program implementation? a) Are you satisfied with the way information was delivered about the identification of beneficiaries, selection of beneficiaries, and delivery of cash during cash transfer in the communities? [1] Yes [0] No If ‘yes’ how well information on cash transferred was relayed? If ‘No’ what was wrong with the information on cash delivery program? b) What was your experience with the cash distribution process such as waiting time, walking distance, safety and grievance redress? How can this be resolved if this program should continue in other areas? 806 In your opinion, did the seed voucher (through the seed fair) contribute to access to good quality seeds for farmer beneficiaries in this community? [1] Yes [0] No (If ‘Yes’ Go to 809) 807 If ‘No’ to question (806) why do you think the seed voucher did not contribute to access to good quality seeds? 809 Should there be problems in the seed voucher support, what are the problems you would most advice to be solved in any subsequent implementation? 810 Was your household able to save seeds for the following planting year after end of seed voucher support? THE END. THANK YOU! xx End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone c) Are there instances where cash beneficiaries are pressured by local authorities (Town/Paramount chiefs, District Councilors); CICs, extended family or community members to give money you did not want to give? If so, how did you handle this? d) Has there been an instance where some kind of conflict emerged over the receipt of cash (within same beneficiary households, between beneficiary and non-beneficiary household, between beneficiary community and non-beneficiary community? [1] Yes [0] No If ‘yes’ kindly explain instances where such conflict occurred. e) Were there conditions that made some beneficiaries not to receive the cash at any point in time? [1] Yes [0] No. If ‘yes’ what were the conditions? How was this situation handled? How long did it take for the partner to respond to the request of the beneficiary who failed to comply with the condition? xxi End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Annex V: Key Informant Interviews Guide Questions for Key Informant Interviews (KIIs): Cash transfer Target respondents: CARE Project Staff and Implementing Partners (MADAM, RODA), NaCSA, Bombali District Council Rep., Tonkolili District Council Rep., President’s Delivery Team (PDT), Anti-corruption Commission (Bombali and Tonkolili), Councilors, Parliamentarians. Evaluation Question 2: To what extent were the systems, processes and procedures used by EFSP for delivering cash transfers and non-cash assistance to participants were effective, efficient, and sustainable during the period of program implementation? a) What mechanisms were used to ensure that the right beneficiaries were targeted in all phases? b) Was there a change in approach or modalities due to emerging conditions/situations? If yes, can you please describe? c) During implementation, what were the mechanisms used to ensure beneficiaries, the public, and management were fully informed about what was happening at field level? d) What was your experience with the cash distribution process such as waiting time, walking distance, safety and grievance redress? e) Were there conditions that made beneficiaries not to receive the cash at any point in time? How was this situation handled? How long did it take for the partner to respond to request who did not follow the due process for receipt of cash? f) How did the cash and non-cash assistance take into account timeliness, cost-effectiveness and accountability to participating households during the implementation period? g) What were the gender implications of the cash and non-cash assistance? Evaluation Question 3: To what extent did the collaboration and coordination with public and private sector stakeholders result in enhanced capacity to deliver cash transfers as a form of social protection to vulnerable households and contribute to effective delivery of food assistance during the emergency and recovery phases? a) How did Implementing Organization coordinate activities with national and local government authorities? b) How did implementing partners address the issues of parallel systems and structures? How did this impact results? c) How did Implementing Partner coordinate with other FFP (food for peace) Implementing Partners? How did that collaboration enhance achievement of results? d) How did other interventions in targeted communities’ impact results (leveraging funds, enforced or complement implementation, and sustainability)? e) How far did the RESSNER Cash transfer project contribute to the President’s priorities on social protect? General Question: What worked well and what did not work well in the RESSNER project implementation, and why? a) What were the best practices observed from the project implementation? b) What challenges or failures were observed and why? c) What changes and/or improvements could be recommended for any replication or future interventions similar to RESSNER cash transfer program? Questions for Key Informant Interviews (KIIs): Seed Voucher Support Target respondents: CARE Project Staff and Implementing Partners (MADAM, RODA), MAFFS, Bombali District Council Rep., Tonkolili District Council Rep., xxii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Evaluation Question 1: How did the seed voucher support contribute to increased resilience of affected communities? a) How has the seed support (through seed fairs) contributed to access to good quality seeds for farmer beneficiaries in target communities? b) What is the significant value of the seed voucher and seed fair scheme compared to other modalities of accessing seeds? c) Did the seed voucher scheme affect non-beneficiary communities in terms of market prices or access to good quality seeds in their communities? d) How did the seed voucher and seed fair scheme benefited the local seed growers/vendors/ sellers? e) How did the projects’ seed support contributed to MAFFS targets and programs at the district/nationwide? xxiii End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Annex VI: List of Stakeholders Engaged # NAME POSITION INSTITUTION PHONE EMAIL 1 Farama J. Bangura Regional Coordinator NaCSA- North 076383740 faramajay@nacsa.gov.sl 2 Christopher Bangura Community-based Specialist NaCSA- Bombali 078585404 christb2013@gmail.com 3 Steven Bangura District Coordinator NaCSA- Tonkolili 076294441 4 Aruna Mans-Davies Team Lead Presidential Delivery Team 078428294 arunamansdavies@yahoo.com 5 Augustine Tarawali Project Officer MADAM 079349444 augustine.tarawali@madam.sl.org 6. Joseph S. Bangura Field Officer MADAM 076720133 jsphmjl@gmail.com 7. Braima B. Jalloh Field Officer MADAM 077243028 8. Ramatu S. Kamara Field Officer MADAM 077809121 ramatusalimkamara14@gmail.com 9. Lucy Fofanah Field Officer MADAM 088265415 10. Yayah T. Turay Field Officer MADAM 088698487 11. Mohamed S. Sankoh Councilor Ward 224 088261060 12. Abu Bakarr Kamara Regional Manager ACC 076492628 yawamie09@yahoo.com 13. John Dito Kamara Deputy Chairman Bombali District Council 078979740 14. Jamie Jusu District Coordinator CARE-Bombali 076588120 15. Mustapha Kamara District Coordinator CARE-Tonkolili 076491203 Mustapha.kamara@care.org 16. Kharifa Kamanda MEAL Coordinator CARE-RESSNER 076682622 Kharifa.Kamanda@care.org 17. Veronica Avila Livelihoods Advisor CARE 030035900 vavila@care.org 18. Lovetta Sannoh Project Management Specialist USAID 099905319 lsannoh@usaid.gov 19. Lahai J. Braima GGrants Coordinator CARE Sierra Leone 076506848 Lahai.Braima@care.org xxiv End of Project Evaluation Report| Rapid Ebola Social Safety Net and Economic Recovery Project in Bombali and Tonkolili Districts in Sierra Leone Annex VII: Bibliography FANTA. (1999). Food Security Indicators and Framework for Use in the Monitoring and Evaluation of Food Aid Programs. Retrieved from http://pdf.usaid.gov/pdf_docs/Pnacg170.pdf FANTA III, U. F. (2011). Household Hunger Scale: Indicator Definition and Measurement Guide. Retrieved from https://www.fantaproject.org/sites/default/files/resources/HHS-Indicator-Guide￾Aug2011.pdf GoSL. (2009). National Draft Framework for Social Protection in Sierra Leone. Retrieved from http://www.fao.org/docs/up/easypol/forum/31/31_National_Draft_Policy_for_Social_Protection_ in_Sierra_Leone.pdf GoSL. (n.d.). President's Recovery Priority. Retrieved from http://www.presidentsrecoverypriorities.gov.sl/ Grobler, W. C. (2014). Food Insecure Household Coping Strategies: The Case of a Low Income Neighborhood in South Africa. Mediterranean Journal of Social Sciences, Vol 5 No 13. Available at: http://www.mcser.org/journal/index.php/mjss/article/viewFile/3552/3493. IFPRI. (2012). The Women Empowerment in Agriculture Index. IFPRI Discussion Paper, Available at: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.303.4947&rep=rep1&type=pdf. USAID. (1999). Sampling Guide (Addendum). Retrieved from https://www.fantaproject.org/sites/default/files/resources/Sampling-1999-Addendum-2012- ENG_0.pdf USAID. (2014). An Unprecedented Response to the Ebola Crisis. Retrieved from https://blog.usaid.gov/2014/09/an-unprecedented-response-to-the-ebola-crisis/ USAID. (2015). Baseline Study for the Title II Development Food Assistance Programs in Zimbabwe. Retrieved from https://www.usaid.gov/sites/default/files/documents/1866/Zimbabwe%20Baseline%20Study%20 Report%2C%20June%202015.pdf USAID. (2016). USAID Office of Food for Peace: 2016-2025 Food Assistance and Food Security Strategy. Retrieved from https://www.usaid.gov/sites/default/files/documents/1867/FFP-Strategy￾FINAL%2010.5.16.pdf USAID. (2016). USAID's Office of Food for Peace Policy and Guidance for Monitoring, Evaluation and Reporting for Development Food Security Activities. Retrieved from http://pdf.usaid.gov/pdf_docs/PBAAF103.pdf USAID, F. I. (2015). Comparing Household Food Consumption Indicators to Inform Acute Food Insecurity Phase Classification. Retrieved from https://www.fantaproject.org/sites/default/files/resources/HFCIS-report-Dec2015.pdf USAID, W. C. (2008). The Coping Strategies Index: A tool for rapid measurement of household food security and the impact of food aid programs in humanitarian emergencies. Retrieved from https://www.researchgate.net/profile/Daniel_Maxwell2/publication/259999318_The_Coping_Stra tegies_Index_Field_Methods_Manual_- _Second_Edition/links/59b66271aca2728472dc5c12/The-Coping-Strategies-Index-Field￾Methods-Manual-Second-Edition.pdf WFP. (2015, January). Mvam Bulletin # 10. Retrieved from http://documents.wfp.org/stellent/groups/public/documents/ena/wfp272359.pdf WFP. (2015). Sierra Leone Emerency Food Security Assessment. Retrieved from http://documents.wfp.org/stellent/groups/public/documents/ena/wfp276779.pdf?iframe