NERRB EVALUATION REPORT Nsanje Early Recovery & Resilience Building (NERRB) Project Report Prepared For Concern Worldwide —Malawi Prepared By Centre for Research & Development Post Office Box 30729, LILONGWE 3, MALAWI +265 (0) 881 203178 crdcnsultancy@gmail.com Date Data Collected From Saturday, 4th August to Monday 6th August, 2018 Date Report Written Wednesday, 08th August, 2018. Copyright © Wednesday, 08th August, 2018 Concern Worldwide. All Rights Reserved 2 i. Table of Contents i. Table of Contents ...................................................................................................................... 2 ii. Acronyms and Abbreviations List ........................................................................................... 3 iii. The Executive Summary .......................................................................................................... 4 1. Project Background............................................................................................................. 9 2. Survey Design and Methodology..................................................................................... 11 3. Survey Findings and Discussion .......................................................................................122 3.1. RELEVANCE..................................................................................................................... 122 3.2. EFFECTIVENESS ................................................................................................................ 155 3.3. EFFICIENCY...................................................................................................................... 200 3.4. IMPACT ............................................................................................................................ 233 3.5. SUSTAINABILITY.................................................................................................................. 26 3.6. Key Lessons and Challenges.......................................................................................... 28 3.7. Project Partnership, Collaboration and Integration ................................................. 301 4.0. Conclusions and Recommendations...........................................................................334 Annex1: The NERRB’s VSLs Case Study .................................................................................... 38 Annex 2: Evaluation Tools and ToRs....................................................................................... 390 Annex 3: Inception Report ...................................................................................................... 390 3 ii. Acronyms and Abbreviations List AEDC Agricultural Extension Development Coordinator AEDO Agricultural Extension Development Officer CDO Community Development Office CIP International Potato Centre CPCs Civil Protection Committees DAC Development Assistance Committee DADO District Agricultural Development Officer DRR Disaster Risk Reduction DVMs Decentralized Vine Multipliers EPA Extension Planning Area FAWs Fall Army Worms FGDs Focus Group Discussions GBV Gender Based Violence HSA Health Surveillance Assistant HWFs Hand Washing Facilities IGAs Income Generating Activities IPM Integrated Pest Management KIIs Key Informants Interviews KPIs Key Performance Indicators M&E Monitoring and Evaluation NERRB Nsanje Early Recovery and Resilience Building OECD Organization for Economic Cooperation on Development OFDA Office of US Foreign Disaster Assistance OFSPs Orange Fleshed Sweet Potatoes rCSI Reduced Coping Strategies Index TAs Traditional Authorities TFI Targeted Farm Input VCPCs Village Civil Protection Committees VDCs Village Development Committees VSL Village Savings and Loans WASH Water, Sanitation and Hygiene 4 iii. The Executive Summary oncern Worldwide commissioned an evaluation of the two-year Nsanje Early Recovery and Resilience Building (NERRB) project to determine levels of improvement in selected indicators across 3 sectors (namely Food Security; Water, Sanitation and Hygiene (WASH); and Economic Recovery) after implementation. Other objectives of the evaluation were to assess level of impact on the beneficiaries’ livelihood status, sustainability of project outcomes and highlighting the project’s efficiency. The consultant, Centre for Research and Development, was mandated to design, conduct and report the end-term evaluation survey by Concern Worldwide in the Nsanje District. Pre- and post- intervention comparisons were used as the framework of evaluation for the project; baseline values for key outcome indicators were collected at project onset and compared with values established at the end. The end line survey employed a mixed methods approach comprising literature reviews, key informants interviews (KIIs), focus group discussions (FGDs) and community observations to collect data that have informed the findings herein. Quantitative data from the sampled households of the beneficiaries was collected by Concern Worldwide with guidance from the Livelihoods Monitoring and Evaluation (M&E) Advisor. A total of 16 FGDs, 22 KIIs, and 4 community observations were included in qualitative data collection. Table 1 below presents the findings of the evaluation for NERRB’s contributions towards an increased ability of vulnerable households to recover from drought and floods occurrences, achieve food security, regenerate their asset base and build resilience to recurrent climatic shocks. The survey findings in Table 1 show performance of a total of 14 outcome level indicators by comparing them before (baseline) and after (evaluation). Clarity on the project’s level of effectiveness is added by the measure of the percentage of the indicator’s targeted change achieved. The baseline and end-line comparison for NERRB indicators has been disaggregated along the three sectors: Food Security, Water, Sanitation and Hygiene; and Economic Recovery and Market Systems. In general, the project area experienced tremendous improvements in the indicator values for WASH and Economic Recovery with un-compelling effectiveness in the indicators for Agriculture and Food Security. C Table1: Baseline and End-line comparison for NERRB Outcome Level Indicators Founded on the evaluation findings on the higher level indicators, the following inferences per sector have been drawn: Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators Sector 3. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 2. WATER, SANITATION AND HYGIENE Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators 1. AGRICULTURE AND FOOD SECURITY Outcome #1: Increased access to agricultural inputs for drought and pest infestation affected households. On assessment of the findings, the evaluation found that farmer’s accessibility to farm inputs improved following introduction of seed fairs and promotion of small scale shallow well irrigation which drastically increased accessibility of year-round access to a water source for crop cultivation activities. In accordance with this, the hunger gap of the beneficiaries also decreased. However, there was no significant change in the meal frequency across the participants. This is likely due to the eating habits and coping strategies that communities currently adhere to. Whilst less households required emergency food assistance throughout the project timeline, the project faced challenges from recurring shocks, such as dry spells, floods and pest infestations over the study period reducing the impact NERRB could have. These recurrent pest infestations also negatively affected the number of IPM practices adopted by the end of the project timeframe and it is believed a number of beneficiaries discarded the ideas when they realised current practices were not effective in controlling the FAW populations. 2. WATER, SANITATION AND HYGIENE Outcome #2: Improved hygiene practices for most vulnerable households involved in irrigation activities. This evaluation has revealed that the greatest improvements for the beneficiaries came in the WASH sector. Whilst only 72% of the targeted households were transporting water safely, 91% of the participants were successfully storing their water safely at their dwellings by the end line evaluation. It is worth noting that the unsafe transportation was largely due to the use of open containers exposing water to unhygienic conditions. These findings highlighted the need to continue expressing the requirement for safe transportation of water, alongside safe storage. There were also improvements in hand washing practices and availability of adequate sanitary facilities across the targeted households. These improvements were likely due to the campaigning and capacity building delivered by Assistant Environmental Health Officers (AEHOs), Health Surveillance Assistants (HSAs) and 36 hygiene promoters. 3. ECONOMIC RECOVERY AND MARKET SYSTEMS Outcome #3: Increased ability of communities to build up resilience to future shocks. With regards to this outcome, it should be acknowledged that the indicators for this sector were already high as the baseline survey was done late following an improvement in status. This could therefore explain some of the minimal changes observed. 7 Appraisal of the findings within this sector show that there has been increase in households owning or accessing livelihoods assets and, when required, their ability to access financial credit. This project saw the establishment of 114 Village Savings and Loan (VSL) groups and provided training in business skills development to the beneficiaries. It is likely that these benefits will potentially help in withstanding any climatic shocks. RELEVANCE, EFFICIENCY, IMPACT AND SUSTAINABILITY With reference to relevance, the evaluation established that all three sectors of the project (food security, WASH and economic recovery) were found to be aligned to the aspirations of the community members. Not only was NERRB relevant to the beneficiaries, it was also consistent with the development priorities of Nsanje district council and national strategy of Concern Worldwide. The survey with regard to efficiency found that the project interventions were done in a cost-efficient manner with the measure of the benefits accruing to the beneficiary population exceeding the measure of the costs. Also, the resource depletion rate and management was satisfactory and most of the activities were implemented on schedule and within the agreed budget. Relating to impact, the evaluation results indicated that the project achieved intended and unintended effects, both long term and short term, and real difference in the lives of the beneficiaries. Nevertheless, the challenge, for instance in agriculture and food security sector, was poor climatic conditions i.e., prolonged dry spells, flash floods especially in Madani and Chilema, and FAWs infestation that characterized the years of project implementation. While in VSL groups, the major setback was that men, owing to lack of interest, weren’t actively participating in the microfinance activities. This was coupled by low savings and increased incidences of loan defaulting. The evaluation, pertaining to sustainability, uncovered that in all 3 sectors of NERRB, there’s a possibility of the continuity of the project benefits even after the exit of the funding since the local structures have been trained and equipped with the relevant knowledge and skills to carry on the activities. In accordance with the survey findings and discussion above, the NERRB evaluation includes a detailed list of recommendations for future project sustainability and replication. This evaluation highlights the need for an improved project design with more achievable time frames and consideration for intervention impacts to materialise. It also 8 addresses the need for further work focussing on Disaster Risk Reduction and preparedness as opposed to risk mitigation interventions in future projects. Further recommendations include ensuring farmers have access to better quality suppliers for agriculture; the inclusion of livestock as risk reduction methods; continued promotion and improvements of sanitation measures; increased capacity training; and Concern Worldwide’s continued cooperation with international organisations to deliver better management of FAW infestation. 9 1. Project Background oncern Worldwide has been working in Malawi since 2002 and in Nsanje since 2007, with a focus on supporting the poorest and most marginalized to build resilient livelihoods and to lead healthy and productive lives. In Nsanje, Concern has been implementing programs on (1) resilient livelihoods, (2) gender equality, (3) gender-based violence and education, and (4) community management of acute malnutrition. In 2015 Concern Worldwide implemented a highly effective emergency response to the floods in Nsanje district, focusing on immediate life-saving support, camp management, protection and livelihoods recovery. According to NERRB Baseline Report, the January 2015 floods affected nearly 80% of the households in Nsanje district forcing people to relocate to safer and more secure places. The households that were hit by floods were also affected by food unavailability. The problem was worsened by erratic rainfall, rendering houses to exacerbating levels of vulnerability to the effects of climate change. Likewise the economic capacity of the affected households was very low. On the other hand, relocation of the flood-hit households meant that access to sanitary facilities for defecation and non-defecation was a challenge. In order to further build resilience of Nsanje residents, the Office of US Foreign Disaster Assistance (OFDA) sponsored action was introduced. The project, which was also commonly identified as “Nsanje Early Recovery and Resilience Building (NERRB)”, was a response to Nsanje’s two major climate change related calamities, namely “floods and dry spells.” The intervention was implemented in Nsanje district targeting the following five Traditional Authorities (TAs):  Makoko  Ndamera  Chimombo  Nyachikadza  Ngabu The NERRB 1 project was executed by Concern Worldwide between July 2016 and July 2017 with an aim of achieving an improved ability of the most vulnerable households to recuperate from the recurrent climatic shocks, i.e., floods and drought. Key activities under this first phase were aimed at supporting vulnerable households with improved crop production, increased hygiene promotion, and economic recovery and market systems. Improved crop production was targeted through improved seeds and the provision of planting materials for both summer and winter crops. The crops targeted C 10 were mainly maize and beans. NERRB 1 also contributed to the enhanced production of legumes i.e. beans using small scale irrigation with treadle pump and shallow well development. In addition to the above, this project also employed innovative extension models to promote longevity of farming within the community. The two models promoted were the “Sustainable Lead Farmer Model” for climate smart extension advice, and the “Irrigation Extension Support Model” for small scale irrigation endeavours. NERRB 1 also looked to focus on Integrated Pest Management (IPM), i.e. interventions against Fall Army Worms (FAWs) which are common in the area. Finally, Women Economic Empowerment was also an area of concentration through Village Savings and Loans (VSL) Groups. This was for increasing access to credit for enhanced resilience of the households to economic shocks. However, the gains made under NERRB 1, were compromised by the FAWs which infested the fields of the beneficiary farmers. A cost extension of NERRB 1 was proposed hence the introduction of NERRB 2. This was deliberately done to consolidate the gains of NERRB 1. While NERRB 1 commenced from July 2016 to July 2017, NERRB 2 was carried out from between July 2017 to August 2018. The USAID funded NERRB 2 put much emphasis on the addition of the elements (that is to say, new interventions) that were NOT included in the previous phase. For instance, choice of crops by farmers was broadened by including sorghum and millet to the list of common beans and maize. Other prominent additions were; awareness raising in hygienic practices, capacity building and business skills training. Essentially, NERRB 2, just like NERRB 1, was primarily a resilience building project that focused mainly on the vulnerable households in terms of three sectors. These were: 1. Agriculture and Food Security (seed fairs for easy access to farm inputs, conservation agriculture, irrigation, IPM for increased productivity), 2. Water, Sanitation and Hygiene (safe collection, transport and storage of drinking water, proper hand washing behaviours, unrestricted access to sanitary facilities), and 3. Economic Recovery and Market Systems (access to financial credit through the community savings and loans schemes). The updated baseline assignment for NERRB 2 was facilitated in November 2017 in order to obtain reliable and realistic baseline figures for the project. Considering that implementation of NERRB 2 ended on 17th August 2018, an end line evaluation was therefore expedited to establish the picture of the overall performance of the project by comparing the evaluation findings with baseline figures. 11 2. Survey Design and Methodology he qualitative survey was implemented in three days beginning Saturday 4th August to Monday 6th August. While quantitative data was collected by Concern Worldwide between 17th July and 23rd July 2018, the consultant embarked on gathering the much needed qualitative data for the assessment of the overall performance of the project. The consultant facilitated Focus Group Discussions (FGDs) mainly targeting two TAs, Ndamera and Chimombo, with the sampling unit comprising of 4 Group Village Heads (GVHs); Fransiku in TA Chimombo and Kalumbi, Madani and Chilema GVHs in TA Ndamera. As for the KIIs, TA Makoko was also included targeting the AEDOs for Zunde Extension Planning Area (EPA). Table 2 gives a quick catch of the sample size framework for the qualitative survey. Table 2: Evaluation Sample Size Framework for the Qualitative Data Elements Focus group discussions Key informants interviews Community observations Sample Size 16 FGDs Fransiku VDC & VCPC, Fransiku Beneficiary Men & Women, Fransiku VS&L Group, Fransiku Non Beneficiary Men & Women, Kalumbi VDC & VCPC, Kalumbi Beneficiary Women & Men, Kalumbi VS&L Group, Kalumbi Non Beneficiary Men & Women, Madani VDC & VCPC, Madani Beneficiary Men & Women, Madani VS&L Group, Madani Non Beneficiary Men & Women, Chilema VDC&VCPC, Chilema Beneficiary Men & Women, Chilema VS&L Group, and Chilema Non Beneficiary Men &Women. 22 KIIs AEDO under Zunde EPA, Fransiku Senior HSA, Fransiku Lead Farmer, Fransiku Village Agent, GVH Fransiku, AEDO under Nyachilenda EPA, Kalumbi Senior HSA, Kalumbi Village Agent, GVH Kalumbi, Madani Hygiene Promoter, Madani Lead Farmer, Madani Village Agent, Madani Senior HSA, Chilema Hygiene Promoter, Chilema GVH, Chilema Lead Farmer, Chilema Senior HSA, Crops Officer, Community Development Officer, Project Manager, M&E Person and Project Support Officer. 4 Fransiku Community Observation Kalumbi Community Observation Madani Community Observation Chilema Community Observation Data Collection Tools FGDs Checklist KIIs Checklist Observation Guide paper T 12 3. Survey Findings and Discussion The evaluation has deliberately organized this section of results and discussion by the core evaluation objectives according to the OECD-DAC. These are relevance, effectiveness, efficiency, impact and sustainability. This has been purposefully done for easy conception of the report and quicker comprehension of the survey findings. 3.1. RELEVANCE elevance gives a summary of the extent to which project beneficiaries feel the intervention addressed felt needs. At its core this is the appraisal of the alignment of the project to the aspirations of the beneficiaries. Two questions were prominent under this section. Firstly, “what problems/ challenges were faced by the beneficiaries before the introduction of the project?” These were pre-NERRB’s challenges that were being faced by the communities. And secondly, “In what way(s) has the project addressed these problems?” In short, the evaluation established that all the three sectors of the project (i.e. food security, WASH and economic recovery) were found to be aligned to the aspirations of the community members. Not only was NERRB relevant to the beneficiaries, it was also consistent with the development priorities of Nsanje district council and the national strategy of Concern Worldwide. The narration of the survey findings and discussion under relevance has been segmented by sectors as follows: 1. Agriculture and Food Security Nearly all respondents (94%) in the16 FGDs that were facilitated indicated that prior to the NERRB action, households were hit by floods due to heavy rains and siltation and they [floods] engendered food insecurity by causing low food production. This resulted in low incomes from the sales of the crop yields. Additionally, 90% of the key informants revealed that as a result of the lack of farm inputs, i.e., improved seeds, farmers were forced to mainly rely on uncertified seeds for their agricultural activities. Consequently, agricultural productivity was very minimal, coupled with persistent dry spells and environmental degradation. When asked, what problems/challenges you were facing before the onset of NERRB, GVH Fransiku, TA Chimombo, had this to say: R 13 “Before the introduction of NERRB project, owing to inadequate extension services on improved (modern) farming technologies, we used to have knowledge gaps in terms of irrigation farming using shallow wells and money maker treadle pumps. Likewise we weren’t doing manure making and this resulted into poor soil fertility. This was further aggravated by environmental degradation. In like manner, we had knowledge gaps as it pertains to IPM and control of the Fall Army Worms (Funye).” In relation to the problems and challenges addressed, the evaluation signified that 90% of the FGDs’ respondents were knowledgeable of the negative situations which NERRB addressed and the modalities through which such challenges were unravelled. 3 out of 4 Concern staff members, i.e., Project Manager, M&E Officer, and Project Support Officer, who were interviewed as key informants disclosed that one way of curbing the beneficiaries’ challenges, 4 seed fairs as of March 2018, were facilitated by giving farmers an opportunity to have unrestricted accessibility to certified farm inputs. Madani Lead Farmer Group members cited that through the seed fairs they received 2 Kilograms of SC 403 Maize seeds, 5 Kilograms of top dressing fertilizer (UREA), 2 kilograms of either sorghum or millet, 1 Kilogram of beans, and 2 sachets of vegetable seeds for winter and/or residual moisture cropping. Additionally, 70% of the key informants divulged that 120 treadle pumps were given to the beneficiary farmers. Subsequently, as of March 2018, shallow well irrigation is being practised with a total of 97 shallow wells being lined enabling winter cropping. This means that 80 hectares of land was now under irrigation. All the four observations that were facilitated uncovered that 53 conservation agriculture clusters comprising 3950 farmers were established. 2. Water, Sanitation and Hygiene 80% of the FGDs made it known that inadequate sanitary facilities for defecation, (i.e. latrines and HWFs), and insufficient sanitary facilities for non-defecation, (i.e. dish drying rack, rubbish pit, two cup system for water storage & treatment, cloth wire and bathrooms), were issues due to relocation following the January 2015 floods. These were considered the core needs of the communities before the introduction of NERRB project. What the FGDs’ respondents stated was resonated by 100% of the hygiene promoters who mentioned that due to knowledge gaps in hygienic practices there were increased cholera cases in the area due to poor hygienic practices. Through the community observation in Chilema, TA Ndamera, it was found that 6312 hand washing facilities (HWFs) were installed which have proved to be effective in reducing cases of waterborne diseases, i.e. cholera and diarrhoea. In a like manner, an 14 interview with the Project Manager found that to curb the problems beneficiaries were facing, the project managed to recruit and train 36 community hygiene promoters (13 females and 23 males). These hygiene promoters were helping in raising awareness of the importance of hand washing with an effective product, i.e. ash and/or soap, and safe collection, transportation and storage of water. An interview with Kalumbi Senior HSA showed that the project engaged the hygiene promoters in the provision of chlorine for water treatment to the targeted households. 3. Economic Recovery and Market Systems All FGDs (100%) with Members of the VSL Groups revealed that preceding to the implementation of NERRB there were increased cases of poor economic status. Households principally relied on scanty sales made from the farm produce resulting in low disposable household incomes. This was coupled with limited access to financial credits which was prevalent in the area. An interview with Village Agents in Fransiku GVH, TA Chimombo, and Kalumbi GVH, TA Ndamera, divulged that before NERRB challenges like knowledge gaps in terms of managing the VSL Groups, a poor saving and investing culture, lack of knowledge in administering small businesses, i.e. failure to run IGAs due to financial challenges, and high economic dependence of women on men were common in the area. Following NERRB action, 100% of village agents said that the project addressed their problems through the recruitment and training of 22 village agents. During 95% of the FGDs, VSL members stated this was achieved through the business skills and management training they received. Further to that, the Nsanje Community Development Officer (CDO), whose office was working closely with NERRB’ VSL sector, disclosed the esablishment of 105 VSL groups as of March 2018 (membership is 1988 - 527 males and 1461 females) which were used to address the beneficiaries’ challenges. During the FGD with Chilema VSL Group Members in GVH Chilema, TA Ndamera, the evaluation established the following as some of the modalities which the project employed in addressing economic recovery problems that targeted beneficiaries were facing:  Promoting sound investing and saving culture among the VSL Group members by encouraging group members to undertake the income generating activities (IGAs).  Provision of business skills and management trainings by Village Agents to VSL groups on VSL methodology and management. 15 3.2. EFFECTIVENESS ffectiveness gives the potency or strength of the project, i.e. the success of the project in producing the intended or desired results. NERRB effectiveness has compared performance of the outcome-level indicators between the baseline values adapted from the project literature and end of project evaluation values. The consultant has tabulated the % change between the end line and baseline values. In brief, the evaluation disclosed that significant strides have been made in the WASH sector. Most of the indicators under agriculture and food security haven’t showed a compelling level of effectiveness but the indicators for economic recovery and market systems sector were on track. In the quest of aligning to the outcomes of the project, the survey findings have been summarized along the three sectors - Agriculture and Food Security, WASH, and Economic Recovery and Market Systems. 1. AGRICULTURE & FOOD SECURITY Outcome #1: Increased access to agricultural inputs for drought and pest infestation affected households. The progress of this sector and its corresponding outcome was measured using 7 quantitative indicators as detailed in Table 3 below. Table 3: Performance Analysis of the Indicators for Agriculture & Food Security Sector E Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators 16 Discussion of the survey findings in Table 3 above can be expressed as follows: i. Owing to the seed fairs, the project made investments in improving farmer’s accessibility to farm inputs as validated by basic agricultural input score of 8.3. Even though food security has improved and the hunger gap has been reduced from a scale of 3.4 to 2.9, there is still no significant improvement in meal frequency— the average number of meals the adults and children under 5 in the household consumed on the previous day. The survey showed that the meal frequency remains almost unchanged at 2.1 (end line) from 2.0 at baseline. The slim end line and baseline gap of 0.1 might be attributed more to the following two key reasons:  Eating habits of the community - most households take their meals around mid￾day and in the evenings, i.e. two meals a day. They spend most of their morning hours (from 5 am to 10 am) in the fields without taking morning meals. This habit largely prevents them from shifting their eating habits to reflect the positive gains in food availability.  Coping strategy - in some interviews respondents suggested that with FAW infestations they didn’t harvest adequate amounts of food. Eating twice a day is a part of the coping strategy to evenly distribute their available food to a possible considerable time frame. By doing this, they’re managing risks of food insufficiency during lean periods. ii. NERRB has by a small margin (-3.8%) reduced the proportion of households in need of emergency food assistance during the lean season (November to March) from 52.1% at baseline to 48.3% at evaluation. NERRB worked on increasing the targeted beneficiaries’ accessibility of farm inputs and promoted shallow well irrigation to enhance household’s ability to withstand the shocks. It is possible this reduction was only small because of recurring shocks such as dry spells, floods and pests throughout the project period, resulting in household’s still needing assistance. Another contributing factor, as indicated through the FGDs, is that there’s a dependence syndrome by Nsanje residents who always await the government and its development partners to give them things. Furthermore, shorter duration of the project may also mean that some outcomes haven’t materialised.  These findings are further compounded when looking at the results regarding the hunger gap. The Hunger Gap (the average number of months that households struggle to meet their minimum daily food requirements in a calendar year from either own production or accessing the market) has registered a narrow reduction of -0.5 from 3.4 at baseline to 2.9 at end line. In the same way, the reduced coping strategies index (rCSI), the household experience of food insecurity in the past 7 days calculated based on negatives and then converted to a positive scale of 0-10, has also only jumped from 7.2 17 to 8.0 with the end line-baseline gap of 0.8. The evaluation concluded that as much as there has been a drive to build resilence of targeted households from food aid through increased productivity, FAWs compounded by prolonged dry spells have been devastating to crop production in the impact areas, causing reduced impacts from NERRBs’ effort. iii. Proportion of respondents who report year-round access to a water source for crop cultivation activities has shifted sharply from 0.3% at baseline to 15.9% at end-line. This is very much attributed to the provision of 97 shallow wells by the NERRB project. Despite this being what appears to be a significant increase, it still falls short of the expected target of 40%. The target has not been met due to reduced water level after speedy recession of water where these wells are constructed. It was a bad year in terms of rainfall pattern that can refill aquifers, thereby limiting refill levels of constructed wells. iv. A before and after contrast approach to the average score of promoted IPM practices shows a significant reduction from 5.3 to 0.7 with the endline-baseline gap of -4.6, falling well short of the intended target value. It is likely that these findings were impacted by the prevalence of FAWs at the time the survey was conducted. There is little research done on effective IPM techniques in fighting against FAWs so they remained a pest throughout the program. The high adoption rate prevalent at baseline suggests that IPM might have been given pre-eminence in NERRB 1 while in NERRB 2 the beneficiaries perhaps discarded them as they realized that they weren’t effective after trying them for some time. The low adoption of IPM could be further clarified by the fact the gains made under NERRB 1 were compromised by the FAWs which infested gardens of the beneficiary farmers. 2. WATER, SANITATION AND HYGIENE Outcome #2: Improved hygiene practices for most vulnerable households involved in irrigation activities. The progress of this sector and its corresponding outcome was measured using 4 quantitative indicators as detailed in Table 4 below. Table 4: WASH Sector Indicator’s Performance 18 This is the area that has made considerable strides within a short period of time. WASH activities are always hard to achieve using the selective target criteria that Concern used. The appropriate way to deal with WASH is to do total catchment area targeting where 100% of households should participate for impact. By-laws are also crucial in making sure that people are following the hygiene and sanitation messages within the catchment area. That being said, according to the evaluation findings, effectiveness of WASH for NERRB Project has been summarised as follows: i. The evaluation uncovered that 91% of those that participated in the survey managed to keep their water safe. The containers were clean and covered. However, when it came to transportation of water from water sources to their homes only 72% were able to transport it using safe (properly covered) containers. It should be noted that the unsafe transportation is mainly influenced by use of open containers. This tendency exposes water to unhygienic conditions. This is a very crucial area that was not taken forward by many of the community members. ii. Concerning hand washing, the survey revealed that 82% of the respondents reported a minimum of 3 critical moments when they are supposed to wash hands at the end of the project against 56% at baseline, superseding the target of 80%. iii. Despite this increase in hygiene practices stated above, the project has only registered a narrow escalation in proportion of households with adequate and available sanitary facilities from 26.7% at baseline to 29.9% at end line. This has been a 3.2% improvement, however, this was not as high as the project had hoped to be. Challenges highlighted during FGDs, like failure/ reluctance by some households in the area to construct latrines owing to negligence, could help to explain the deficit. Another explanation, perhaps, could be that the indicator’s Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators 19 target was too high considering this was a behavioural indicator and behavioural tendencies can take some time to become common practice. 3. ECONOMIC RECOVERY AND MARKEY SYSTEMS: Outcome #3: Increased ability of communities to build up resilience to future shocks. The series of indicators that were used to demonstrate a change made by this sector have herein been compared at baseline and end line, as revealed in Table 5 below. Table 5: Demonstrated Change for Increased Resilience to Future Shocks Below is how the evaluation appraised the effectiveness of economic recovery and market systems. i. The evaluation has unveiled that indicators for this sector were already high as the baseline was done late after an improvement in the status had already taken place. This could help explain the minimal changes observed in Table 5. ii. For example, the survey revealed a slight increase of 8.9% in the percentage of households owning or accessing livelihoods assets with the coping strategies index narrowly moving from 7.7 to 8.5. These numbers may be small as they only represent improvements over the last 8 months of the project with the targets likely too high to be achievable. Despite what appears to only be small increases, nearly all respondents (99%) who indicated possession of improved livelihood assets mentioned now owning a bicycle. This shows that despite numerous climatic-related shocks which the community members face in the area, having a bicycle is regarded as a priority and shows economic growth. iii. Another interesting finding which the evaluation made known was that there was a positive correlation between households’ saving culture and owning improved livelihood assets. This suggests that an increase in VSL activities can have a positive impact on the capacity of household to own improved livelihood assets. Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators Achievement Target Targeted Change Actual Endline Data (from Evaluation) Actual Change % Targeted Change Achieved 1 Basic Agricultural Inputs Index Score 7.6 Index Score 8.6 Index Score Increase of 1.0 8.3 Index Score Increase of 0.7 70.0% 2 Average Meal frequency per day 2 meals 3 meals Increase of 1.0 2.1 meals Increase of 0.1 10.0% 3 Hunger gap (average number of months of food insecurity) 3.4 months 0 months Decrease of 3.4 2.9 months Decrease of 0.5 14.7% 4 Household experience of food insecurity in the past 7 days [as measured by the Reduced Coping Strategies Index (rCSI)] 7.2 Index Score 9 Index Score Increase of 1.8 8 Index Score Increase of 0.8 44.4% 5 % of Households in need of emergency food assistance during the lean season 52.1% 20% Decrease of 32.1% 48.3% Decrease of 3.8% 11.8% 6 % of respondents who report year-round access to a water source for crop cultivation activities 0.30% 40% Increase of 39.7% 15.9% Increase of 15.6% 39.3% 7 Average score for adoption of promoted IPM practices as reported by respondents) 5.3 Index Score 7 Index Score Increase of 1.7 0.7 Decrease of 4.6 -270.6% 12 Percentage of households owning or accessing livelihoods assets 49.3% 70% Increase of 20.7% 58.2% Increase of 8.9% 43.0% 13 Coping Strategies Index (CSI) 7.7 Index Score 9 Index Score Increase of 1.3 8.5 Index Score Increase of 0.8 61.5% 14 % of households who can access financial credit if required 49.0% 80% Increase of 31% 60.2% Increase of 11.2% 36.1% 8 % of households with adequate and available sanitary facilities 26.7% 50% Increase of 23.3% 29.9% Increase of 3.2% 13.7% 9 % of households who transport drinking water in safe storage containers 70.3% 85% Increase of 14.7% 71.9% Increase of 1.6% 10.9% 10 % of households who store drinking water in safe storage containers 52.8% 70% Increase of 17.2% 91.0% Increase of 38.2% 190.7% 11 % of respondents who know at least three of the main critical moments for hand washing 56.2% 80% Increase of 23.8% 82.2% Increase of 26% 109.2% Sector 2. ECONOMIC RECOVERY & MARKET SYSTEMS Sector 3. WATER, SANITATION AND HYGIENE Targets Actual Baseline Data Sector 1. AGRICULTURE AND FOOD SECURITY # Indicators 20 iv. In addition, the proportion of households who can access financial credit if required has increased by the margin of 11.2% from 49.0% to 60.2%. This increase is owing to the 114 VSL groups that were established by the project. 3.3. EFFICIENCY fficiency reports the extent to which the project managed to achieve maximum productivity with minimum wasted effort or expense. Burn rate of the project resources; cost-benefit analysis; and availability of materials on time, in right quantity and quality, has been given unique focus in this area. The following were the key areas of enquiry under efficiency: 1/ “were the stocks of items available on time, in the right quantities and quality?” 2/ “were the activities implemented on schedule and within the budget?” 3/ “were the outputs delivered economically?” 4/ “how were the project’s beneficiaries identified?” 5/ “how satisfied are the beneficiaries with the services such as trainings and products such as materials provided by the project?” In summary, the project interventions were done in a cost-efficient manner with the measure of the benefits accruing to the beneficiary population exceeding the measure of the costs. Also, the resource depletion rate and management was satisfactory and most of the activities were implemented on schedule and within the agreed budget. 1/ “were the stocks of items available on time, in the right quantities and quality?” Touching on the availability of the stocks on time, in the right quantities and quality, the survey findings through 90% of the KIIs showed that the majority of the project items were made available in time, although some challenges did persist. For example, the Project M&E Advisor had this commentary: “The materials like cement, treadle pumps, and farm inputs were available on time. But the challenge was on the water testing kits. The District Health Office delayed in making available the specifications of water testing kits hence the project didn’t manage to procure kits. Subsequently, the project didn’t manage to furnish the hygiene promoters with water testing kits for distribution to the targeted households.” E 21 With regards to the quality and quantity of supplies, the survey respondents were quick to indicate reservations with these factors. For the quantity, approximately all lead farmers (95%) that were interviewed were quick to state that quantity of each of the farm inputs which they were redeeming during the seed fairs weren’t adequate, i.e. 2 kilograms of SC 503 maize seeds, 5 kilograms of top dressing fertilizer, 2 Kilograms of either sorghum or millet, 1 Kilogram of beans and 2 sachets of vegetables were scrimp. However, a FGD with Kalumbi beneficiary farmers unveiled that they were able to use the provided inputs for nearly three-quarters (75%) of their farming land (conservation agriculture plots). All things being equal, it can therefore be argued that an increase in the farm inputs quantity can lead to an increase in the size of the farm land cultivation. In terms of quality, it was reported that some onions and tomatoes seeds weren’t in good quality as some of them had low germination rate. Besides the Nsanje-based agro dealers supplying poor quality seeds at seed fairs, the poor germination rate by some seeds could also be attributed to micro-climatic factors in the area. 2/ “were the activities implemented on schedule and within the budget?” An interview with the Project Manager stated that the activities were done on schedule and within the budget. He further explained, “When there was under-expenditure in one thematic area, they could use the remainder of the budgeted funds for another activity which was under-budgeted so long as both activities were within the same sector.” Prudent management of the project funds might be owed to the availability of the internal auditor who was responsible for conducting an official financial analysis of the project’s accounts. However, 85% of the FGDs with community members disclosed that sometimes project staff personnel didn’t honour the project timelines and schedules as they could show up late during meetings. 3/ “were the outputs delivered economically?” According to the community observations that were conducted by the evaluator, in terms of economic delivery of outputs, the project served its purpose cost-effectively. It was unveiled that the project strived to deliver more outputs using the minimum inputs. A discussion with the Project Finance Personnel showed that community members were encouraged to use their own resources as much as possible, especially the much needed labour. This was also echoed by the Project Support Officer who indicated that while Concern provided cement and metal covers for shallow wells construction, the community members contributed bricks, sand and labour. 4/ “how were the project’s beneficiaries identified?” 22 Taking into consideration that NERRB was a recovery and resilience building intervention, Concern would have liked to target all vulnerable and affected households in the targeted areas. Albeit, resources are scarce and this necessitated Concern to identify beneficiaries for the NERRB project. Survey findings from the KIIs and FGDs that were expedited revealed that the project used the wealth ranking system to identify and list the poorest households who were then selected to be the beneficiaries. It was a pro￾poor action. Alongside this, the project used the community structures with involvement of the local chiefs and district level stakeholders. According to the Nsanje Crops Officer, whose office is under Nsanje DADO, the extension officers in the two EPAs, namely Zunde and Nyachilenda, participated in identification of the lead farmers and follower farmers for the project activities. However, 13% of the FGDs uncovered that some deserving farmers weren’t considered. Furthermore, the evaluation found that among the beneficiary households, people living with disability weren’t significantly included as compared to inclusion of the female headed households. In summary, the evaluation has established that the project targeted ADCs and VDCs, lead famers and their follower farmers, men and women for VSL activities, and vulnerable households for hygiene promotion activities. 5/ “how satisfied are the beneficiaries with the services such as trainings and products such as materials provided by the project?” As it regards to the cost benefit analysis, the evaluation documented the justification of the costs incurred by the project compared to the accrued benefits that were being enjoyed by the beneficiaries. According to survey results as of March 2018, five months prior to project closure, distribution of farm inputs to 6450 beneficiary farmers, construction of 97 shallow wells, distribution of 120 treadle pumps and training of 5250 farmers on IPM could justify that the benefits outweigh the costs. Equally, recruiting and training of 36 community hygiene promoters (13 females and 23 males), installation of 6312 hand washing facilities, engagement of 22 village agents and establishment of 114 VSL groups indicated that the costs of the project are justifiable and do not exceed level of benefits that were enjoyed by the beneficiaries. When the beneficiaries were asked how satisfied they were with the services, such as training and materials provided by the project, almost all respondents to the FGDs in Kalumbi and Fransiku expressed high degrees of satisfaction. They considered that the project had bridged the knowledge gap following the training provided and this had enabled a rise in their household’s economic status. An exception was observed in the GVHs Madani and Chilema in TA Ndamera where floods destroyed some winter crops in the dambos. 23 3.4. IMPACT mpact appraises the positive and negative, primary and secondary, long-term effects on the livelihood of the beneficiaries produced by the NERRB intervention; directly or indirectly; intended or unintended. In order to measure the impact of the project over time, a before and after comparison approach was employed. Two questions were focal and they are: 1. What did the beneficiaries say were the fruits of this project? 2. How has the project changed the lives, families, and community of the beneficiaries? The evaluation results indicated that the project achieved intended and unintended, long- and short-term, effects and made a real difference in the lives of the beneficiaries. Nevertheless, the challenge, for instance in agriculture and food security sector, was poor climatic conditions, i.e. prolonged dry spells, flash floods especially in Madani and Chilema and FAWs infestation that characterized the years of project implementation. While in VSLs, the major setback identified was that men weren’t actively participating in the microfinance activities. This was coupled by increased incidences of loan defaulting and low savings. Overall, 75% of respondents were of the view that the project has had a positive impact on their lives. They also affirmed that the project changed beneficiaries’ life, family, and community as a whole. The evaluation herein provides answers to the above asked questions by sectors. 1. Agriculture and Food Security In general, the evaluation shows that the project has improved the inputs accessibility by the targeted famers, while also broadening the coping strategies of the beneficiary households. Likewise, close to 16% of the beneficiary farmers are now having year round access to a water source for crop cultivation. The findings show that 90% of the respondents to the FGDs presented that beneficiaries, courtesy of the NEERB project, now have increased access to farm inputs which are enabling them to practice conservation agriculture. During a discussion with Kalumbi beneficiary women and men in TA Ndamera it was affirmed that the seed fairs greatly helped on accessibility of farm inputs. Further to that, Kalumbi respondents said that the seed fairs were acting as marketing centres where competition was created amongst the agro-dealers enhancing the effectiveness of the service provided. I 24 Community discussions with VDC members from Fransiku in TA Chimombo and Madani and Chilema from TA Ndamera stated that awareness of the improved farming techniques, enablement to practise both winter and summer cropping and knowledge of the integrated pest management were the key fruits of the project. Nsanje crops officer said that the targeted farmers have their own Decentralized Vine Multipliers (DVMs) and lead farmers have so far sold 10,000 bundles to their fellow farmers. However, the winter cropping in Madani and Chilema was adversely affected by the floods that occurred in July 2018 which destroyed the conservation agriculture plots, so these areas have not felt the same immediate benefits. 2. Water, Sanitation and Hygiene On account of WASH interventions, NERRB has helped nearly 30% of households gain unrestricted access to adequate and available sanitary facilities. 71.9% of households are now transporting drinking water in safe storage containers with 91% storing drinking water in safe storage containers. Equally, 82.2% of the beneficiary are now practising at least three of the main critical moments for hand washing. All the four senior HSAs and two hygiene promoters that were engaged as key informants felt that it was due to NERRB’s interventions on WASH that the project area had experienced reduced incidences of diarrhoea and cholera. These interventions included, but weren’t limited to, safe storage and collection of drinking water; provision of sanitary facilities and proper hand washing behaviours. Most households are now practicing good hand hygiene with soap at critical moments and the drinking water is collected, transported and stored safely. Below is an extract of what the VDC members in Madani said during a discussion in terms of the impact of WASH interventions by the NERRB action. “Courtesy of the NERRB action, our area is now an open defecation free (ODF) community. This is due to unrestricted access to sanitary facilities, i.e. latrines at household level. This is different from how things were before NERRB since we used to have increased incidences of open defecation owing to inadequate pit latrines and lack of knowledge in hygienic practices. Likewise, we’re now consuming hygienic water by virtue of proper storage and treatment of water. ” — An extract from FGD with Madani VDC Members in GVH Madani, TA Ndamera. 3. Economic Recovery and Market Systems 25 As a result of NERRB involvements, nearly 60% of the beneficiaries made known that they own or access livelihoods assets. Bicycle ownership was mentioned by approximately all respondents (99%) who indicated possessing improved livelihood assets. Despite community members of the area facing numerous climatic-related shocks, owning a bicycle is considered a priority. During discussions with VSL Group members it was revealed that NERRB’s VSL integration has improved the beneficiaries’ access to financial credits with 60.2% of households reporting having access if required. This has been accomplished through the establishment of 114 VSL groups that members are able to save and access credit from. Usage of savings varies from food expenses, fees payment and buying of agricultural inputs to procurement of households’ assets. Apart from saving with VSLAs, the evaluation found that only a small proportion of the beneficiaries (5%) were saving with formal saving institutions i.e. commercial banks. The main reasons suggested for this were as follows:  Lack of stable cash flow for consistent depositing  Not receiving any remittance from relatives  Too much paper work with banks especially during the opening of the account  Long distance to the commercial bank. FDH bank is the only commercial bank in Nsanje; located at the boma several kilometres from villages. What beneficiaries narrated was reiterated by the Nsanje Community Development Officer who said that, as a result of the VSL groups, beneficiaries have improved high financial base and consequently, poverty levels have lowered and beneficiaries are owning livelihood assets like livestock and houses. When asked about the differences VSL interventions has made in their life, family and community The Kalumbi VSL Group members said this: “We were not able to pay school fees for the children before we joined the VSL but now, through the shares in the VSL, we’re managing to cater for school expenses including school fees and uniforms. This has reduced absenteeism and school enrolment has improved in the area.” In contrast, three-quarter of the FGDs with non-beneficiaries stated that they neither save with VSLs nor formal institutions. In these areas, members of the community put the money inside their dwellings, i.e. put it under the mattress and/or bury in the ground. The evaluation found that financial illiteracy limits the community members from joining the 26 saving bands as 90% of VSL non-members still find it hard to appreciate the role VSLs can play in transforming their livelihoods. 3.5. SUSTAINABILITY ustainability is concerned with measuring whether the benefits of the interventions by the project are likely to continue after donor funding has been withdrawn (permanence). Projects need to be institutionally, technically and financially sustainable. In this evaluation sustainability was gauged along the following two indicators. 1. What would the beneficiaries say are some benefits created by the project beyond its intended objectives? 2. How the beneficiaries think this project will be continued after funding is withdrawn? In summary, this evaluation uncovered that there’s a possibility of the continuity of the project benefits across all three sectors following the withdrawal of funding. The local structures have been trained and equipped with the relevant knowledge and skills required to carry on the activities in the future. 1. Agriculture and Food Security When asked what they think are some benefits created by NERRB beyond its intended objectives, 75% of the FGDs respondents felt that the improved farming practices the farmers have learnt and adopted were highly useful and had the potential to continue despite lack of funding. The improved farming practices which will be sustained include; shallow well irrigation, use of certified seeds resulting in improved crop yields and manure making for improved soil condition. These results were achieved through collaboration of NERRB with extension workers to relay the necessary extension messages to the beneficiary farmers. During a discussion with lead farmers in Fransiku GVH, a very meaningful discovery was implied. “The money maker treadle pumps will be sustained since we were trained on how to operate and maintain them. Also, we do contribute money for buying spares like valves and pipes. Also we’ve hired the security personnel who looks after the pumps in the fields. In a similar manner, seed multiplication and manure making perhaps will be sustained considering that farmers have been S 27 trained and they’re able to do it on their own.”-An extract from the FGD discussion with Fransiku Lead Farmers. 2. Water, Sanitation and Hygiene In terms of WASH, a discussion with hygiene promoters and senior HSAs divulged that increased numbers of improved toilets in the project area is one of the key sustainable benefits achieved by NERRB. Similarly, the discussion with members of Kalumbi VDC indicated that the open defecation free status which their villages have registered will continue being there. While the beneficiary men and women in Madani said that they’ll continue practising hygienic practises, 100% of the hygiene promoters that were interviewed stated that they’ll also continue raising awareness of hygienic and sanitary behaviours in the households. There is one unique case in which one VSL beneficiary from Fransiku GVH revealed that she has managed to have piped water installed at her dwelling yard courtesy of the VSL shares. 3. Economic Recovery and Market Systems From the analysis of the primary data from FGDs, KIIs, and community observations a comprehensive understanding of the factors that would lead to sustainability of the Economic Recovery and Marketing Systems was formulated. It was found in the FGDs in Kalumbi GVH that members in the VSL groups managed to build iron sheet-roofed houses which are more resilient compared to the grass thatched houses. VSL members from Fransiku GVH also found that through the VSL shares they were paying school fees for their children. Both these developments have long term benefits including stable housing and education of future generations. While FGDs with Chilema VSL members revealed livestock purchase was another sustainable benefit. In the neighbouring Madani village the reduction of economic dependence of women on men was highlighted as a long lasting benefit too. Through the community observations done in Chilema and Madani GVHs, the evaluation revealed that 90% of the VSL members were doing Income Generating Activities (IGAs) at Marka Trading Centre. An interview with the community development officer found that proceeds from these IGAs were enabling the beneficiaries to buy LIFO motorcycles from the neighbouring Mozambique, therefore deepening the sustainable socio￾economic development among beneficiaries. He further added that VSLs have been 28 linked to the government structure at the district council through the establishment of the community development Officer. 3.6. Key Lessons and Challenges actors that were influencing key achievements of the gains registered by the project have been identified. This assessment also underpinned the significant challenges that inhibited, to a certain degree, attainment of the pre-determined targets set by the project. The lines of enquiry under this section were as follows: 1. What was good about this project and needs to be continued? That is to say, what are project success factors? 2. What wasn’t good about this project and needs to be discontinued? i.e., project inhibitors? 3. What was lacking about this project which needs to be introduced? 4. What specific challenges did the beneficiaries face and how were they addressed? 1. “What was good about this project and needs to be continued? That is to say, what are project success factors?” The evaluation unveiled that NERRB action was farmer centred and community centred action and was a pro-smallholder farmer concept using participatory approaches throughout. Both community stakeholders and district partners were involved from the project initiation through implementation to project closure (review). This enhanced project sustainability by cultivating project ownership among the partners. This also significantly reduced direct project costs as evidenced by contribution of sand, stones, and bricks by the beneficiaries for construction of the shallow wells. Further success is displayed by the fact this project worked with local people who were trained to practise improved agricultural technologies alongside sound collaboration with the agriculture staff at both district and community levels. This, in turn, brought about another success factor by enhancing local economy empowerment through engaging local agro dealers. The evaluation has also shown that strengthening of irrigation committees can help in enhancing resilience of households to the effects of climate change related activities. While the AEDOs that were interviewed mentioned seed fairs and training of the beneficiaries on the use of the improved technologies as practices that were good and needed to be continued, VSL intervention as a project success factor was stated by all village agents. Further to that, a discussion with the lead farmers made known that Farmer F 29 Group Approach or cluster establishment and exchange visits were factors worthy scaling up. Recruitment of hygiene promoters was also a highly successful factor or NERRB with 100% of HSAs interviewed attesting to this. 2. What wasn’t good about this project which needs to be discontinued? i.e., project inhibitors. An interview with the crops officer at Nsanje DADO reported that the standard of operations (SoPs) weren’t shared among the stakeholders. The SoPs stipulate the role and responsibility of each stakeholder. Failure by Concern to share the SoPs implies that its stakeholders weren’t aware of their roles (responsibilities) in the implementation of the project. Following the discussion with lead farmer groups in Madani it was uncovered that some seeds for onions and tomatoes which were redeemed during the seed fairs had poor germination rate. In future, only agro traders who supply good quality seeds should be sourced. 3. What was lacking about this project which needs to be introduced? At the time of the evaluation all the agricultural extension workers that were contacted reported that mobility in the catchment area needs to be focused on by giving motorcycles to the extension workers. This will also act as a prominent factor for motivation. This opinion was also shared by the lead farmers who stated that they had to walk long distances to gardens of the secondary farmers to train them in modern farming practices. Whilst AEDOs state the need for motorcycles, the lead farmers are in need of bicycles. The evaluator is of the view that this might also promote the hand-out spirit. However, this supply is not sustainable and it isn’t cost effective at all. Nearly all FGDs with VSL group members lamented the low participation of men in VSL related activities. In all the evaluation discussions there was no single man who participated in the VSL discussions and all respondents in the FGDs targeting VSLs were women. The Project Support Officer also echoed that the 22 village agents (VAs) that were trained for VSL activities were all female. The evaluator is, however, quick to mention that men weren’t barred from joining or creating their own groups, but they showed lack of interest. Since seeing the results they’re now more than willing to create their own groups and/or join the already existing ones. The project M&E advisor confided that people with disabilities and the elderly as crosscutting concepts weren’t robustly represented in the project. It was reported during 30 an FGD with the disabled members of the community that the elderly and disabled weren’t involved in project activities. Instead, lead farmers were encouraged to work with only the energetic people who can demonstrate results on the ground. 4. What specific challenges did the beneficiaries face and how they were addressed? Madani and Chilema FGDs with the project beneficiaries, who were mainly farmers, indicated that the destruction of winter crops by the July 2018 floods was a major setback as farmers were left with no alternative on how to manage this challenge. Through community observation it was proven that some of the conservation agriculture plots were severely damaged by flash floods. Lead farmers also noted that FAWs were still an ongoing pest infesting fields. Farmers were encouraged to use local pest control methods which have, to a certain extent, proved to be ineffective. This challenge was further worsened by the USAID-OFDA’s preference of natural methods of pests control instead of chemical methods. As a method of curbing this challenge, it was divulged that farmers were buying chemicals from local agro dealers for pest control. In terms of WASH, the hygiene promoters and HSAs that were interviewed reported that some households were reluctant to construct latrines and non-defecation sanitary facilities owing to negligence (lack of interest). In order to solve this challenge the evaluation established that such households were reported to the village heads. In terms of VSLs, it was uncovered that the cases of loan defaulters were rampant in the established 114 VSLs. This was forcing members to contribute extra money as a way of covering up the defaulted loans. This issue was not addressed within the project timeframe but will be something that should be considered in future. 3.7. Project Partnership, Collaboration and Integration nvolvement of the stakeholders in the project implementation was reviewed. Likewise, logical contribution of the individual project participants to achieve the overall goal of the project was assessed. This information was attained by examining the following two indicators: 1. “To what extent did the project collaborate with key stakeholders and partners?” I 31 2. “To what extent were different participants of the project working and helping each other to achieve the project goal?” 1. To what extent did the project partner and collaborate in its operation with key stakeholders? An in-depth appraisal at the conclusion of this project found that Concern collaborated effectively with key stakeholders and the partners. An interview with the Crops Officer from Nsanje DADO and the Community Development Officer from Nsanje District Council revealed that the NERRB program also utilised their offices as district-level partners. For instance, the DADO Office was primarily acting as the source of improved climate smart agricultural technologies and providing agricultural extension services to the beneficiary farmers. Meanwhile, the community development office was working closely with the VSLs to ensure effectiveness by properly guiding them on VSL methodology and management. However, the evaluation noted that the 114 groups were too many to be effectively monitored by the single CDO. This could potentially be one of the reasons for the failure by the CDO to properly resolve the loan defaulting problem. “The project was implemented through government partners for easy follow-ups, monitoring and evaluation. This also strengthened partnership among key project players including farming communities, project staff, extension personnel at the grass roots and district authorities.”-An extract from an interview with the NERRB Project Support Officer. At a community level, it was reported that NERRB worked closely with local governance structures, mainly VDCs and ADCs. However, it was reported that there was little involvement of the civil protection committees (CPCs) despite NERRB incorporating a component of climate change related interventions. Other partners that worked closely with the project at a community level were the lead farmers and AEDOs, who were principally engaged in agriculture and food security interventions. In terms of WASH, NERRB partnered closely with community hygiene promoters and HSAs whose role was to relay messages on hygienic practices. These messages included safe water management, hand washing practices and latrine accessibility. The hygiene promoters were also responsible for distribution of chlorine for water treatment to the communities. In terms of economic recovery across the community, NERRB worked with 22 village agents whose responsibility was to train and equip VSL members with knowledge on VSL methodology and management. 32 In summary, the evaluation has established that the project used wholly inclusive approaches throughout its life cycle. Both community stakeholders and district partners were involved from the project initiation phase throughout implementation up to its conclusion. This enhanced sustainability by cultivating the spirit of project ownership among the partners. 2. To what extent were different components of the project working together to achieve the project goal? Based on the survey findings (primary data) and literature review (secondary data), it’s an independent view of the evaluator that the project was excellently integrated. For instance, the food security component was aligned to the WASH and VSL sectors. Because of improved crop productivity farmers were able to sell some of their yields and proceeds from such sales were used to improve hygiene and sanitation or invested as shares. This feeling was echoed by lead farmers who, some of them being vine seeds multipliers, stated the following: “With earnings from the vines bundles that we sold, we’ve managed to buy improved containers for safe water collection, transportation and storage. Apart from that we’re also using a proportion of the farm incomes to save in the VSL banks.” Similarly, WASH interventions were supporting both the food security and VSL sectors. With improved WASH practices the beneficiaries lead healthier lifestyles resulting in an increased ability to perform in agriculture and VSL activities, as demonstrated by one Madani hygiene promoter. Likewise, during a discussion with Kalumbi VSL Group members, it was stated that the VSL activities were strongly linked to the agriculture and WASH operations. VSL activities enabled households to have economic access to farm inputs and those that had financial stability have the capacity to construct eco-friendly latrines for defecation and buy sanitary facilities. 33 4.0. Conclusions and Recommendations oncern Worldwide in Malawi implemented a USAID-OFDA funded resilience building and economic recovery project between July 2016 and August 2018 called Nsanje Early Recovery and Resilience Building (NERRB). The project had its three higher level outcomes segmented along the three sectors described below. Outcome#1: Increased access to agricultural inputs for drought and pest infestation affected households (Food Security and Agriculture Sector). Outcome#2: Improved hygiene practices for the most vulnerable households involved in irrigation activities (Water, Sanitation and Hygiene Sector). Outcome#3: Increased ability of communities to build up resilience to future shocks (Economic Recovery and Market Systems Sector). The baseline survey was conducted for establishing benchmarks of the key performance indicators for comparison at the conclusion of the project. At the end of the project, an evaluation guided by the OECD-DAC evaluation criteria (relevance, effectiveness, efficiency, impact and sustainability) was conducted to measure the overall achievement of the project. Based on the evaluation findings, the following have been documented as key conclusions: i. All three sectors of the project were found to be aligned to the aspirations of the community members. Not only was NERRB relevant to the beneficiaries, it was also consistent with the development priorities of Nsanje district council and national strategy of Concern Worldwide. ii. The most significant improvements have been made in the WASH sector. Whilst most of the indicators under agriculture and food security haven’t shown a compelling level of effectiveness, the indicators for economic recovery and market systems sector were on track. C 34 iii. The project interventions were done in a cost-efficient manner. The resource depletion rate and management was satisfactory and most of the activities were implemented on schedule and within the agreed budget. iv. Despite unavoidable challenges including FAW infestations and poor climatic conditions, the project achieved intended and unintended effects and made a real difference in the lives of the beneficiaries. v. Across all 3 sectors there is the potential for continuity of project benefits in future following successful training and provision of materials. vi. It has been observed that the project coordinated very well with its relevant stakeholders at both district and local level hence the gains achieved. Agreeably to the survey findings and discussion above, NERRB evaluation recommends the following: a) In future projects Concern Worldwide should ensure that it has an improved project design matrix. For this project some indicators, especially those included within the food security sector, were not precise and clearly defined. To avoid this problem in future the design matrix should focus on formulation of outcome level indicators that are specific, measurable, achievable, and time bound. It is of high importance that the indicators are easily and well understood by all stakeholders and partners. b) Despite being a climate change response project, NERRB concentrated on risk mitigation interventions only. In future programming, disaster preparedness and disaster risk reduction (DRR) should be focused on also. The integration of such measures would be an innovative and holistic approach to social protection and DRR in Nsanje district which can bolster local resilience, support livelihood diversification strategies and reinforce coping strategies of the households exposed to exacerbating levels of vulnerability due to the climate change-related shocks and hazards. c) The evaluator’s view is that in order for resilience building and social protection projects to have sustained gains they should always integrate crop and livestock production systems. With this in mind, the project would have increased household safety nets if distribution of livestock (such as goats, pigs, ducks etc) under a livestock pass-on scheme was included within the interventions to strengthen household asset (materials and livestock) ownership. Increasing stock size of 35 households would make them less vulnerable to climate change related hazards and shocks, such as drought and flooding, than those without. d) In future, Concern should take steps to ensure that the agro-dealers are supplying farmers with attested seeds that have approximately 100% germination rate. e) Though the project addressed pest management through IPM measures, the FAWs are still troubling farmers in the project area so there’s a need for further efforts on how to control these pests. For instance, provision of chemically pre-treated seeds/plant materials against the pests during both pre- and post-emergence periods. A consideration should also be made to liaise with the government, United Nations development agencies, and researchers to support the fight against FAWs and identify suitable IPM. f) To continue the advances already made in the WASH sector, it would be beneficial for community members to graduate from the construction of traditional basic latrines to more improved facilities in the future. The basic latrines are not disability friendly and are less resilient to heavy rains so in future, latrines such as eco-friendly pit latrines should be constructed. g) In addition, there should be continued promotion of messages and campaigns on the importance of using hand washing facilities, especially after visiting toilets, and/or general hand washing behavior using soap at five critical times. Alongside this, continued efforts should be made to inform the community that environmental protection is very crucial in the availability of water in their communities. h) If NERRB action is to be replicated elsewhere, the project timeframe should be amended to consider the time taken to build resilience within a community. This project includes key indicators that are interested in behaviours of individuals and time allowances should be made for this change in knowledge and attitudes to take place. In order for the project to be more effective in future, sufficient time should be allocated to allow for implementation and adaptation to the project. i) The evaluation highlights the need for continued capacity training for farmers to begin considering farming as a business. This training would improve the households’ disposable income levels and therefore, increase their purchasing power. This in turn would improve the additional assets accumulation by said households and so improve their resilience to future climatic shocks. 36 j) The established VSLs from this project weren’t linked to any microfinance organizations. Moving forwards Concern Malawi should use its international presence to learn how other Concern International members do incorporate strong microfinance structures into their climate mitigation and disaster risk management projects. The evaluator emphasizes that microfinance can be a major coping strategy if properly integrated into a climate change project. It is recommended that cooperative farming should be promoted in future endeavours to enhance farming business culture among farmers/households. 37 Annex1: The NERRB’s VSLs Case Study he evaluation was very interested to find out more about the NERRB’s VSL approach to economic recovery and enhanced safety nets of the targeted vulnerable households. During a discussion with Kalumbi VSL group members it was revealed that the beneficiaries were well-informed of the VSL methodology. “...VSL is the group of village dwellers that centres primarily on saving and providing financial loans to its members at a manageable interest. The group is regulated by the agreed by-laws stipulated in the groups’ constitution. The savings are made by the members of the group. Likewise, borrowing is done by the members. The monies are kept by the group treasurer who forms part of the executive running the daily affairs of the association. The treasurer is selected through the screening process. The treasurer uses the money box that was given by Concern Worldwide to keep safe the monies.”-Excerpt from FGD discussion with Kalumbi VSL Group Members. Further to the above, an interview with a village agent in Fransiku GVH, TA Chimombo, revealed that the village agents trained by Concern on VSL methodology and management were responsible for providing the necessary training to their respective VSL group members. It was reported by the Project Manager that so far 22 village agents (all females) were taught by Concern on VSL methodology and related concepts. When asked to shed more light on how they run their VSL group, Chilema VSL Group members in TA Ndamera had the following remarks; “A maximum of 25 people are included and do agree the day of meetings. They also decide on the shares i.e., 1 share equal to 200 Mwk with 5 shares as the maximum, thus being equal to 1,000 Mwk. In terms of borrowing, each member is free to borrow times 3 of his/her shares at a 20% interest rate. Thus, the higher the shares, the T 38 higher the power of borrowing. The loan is expected to be settled after 30 days, failure to do so attracts an extra interest.” The evaluation was also keen to learn what benefits the beneficiaries had so far accrued through VSLs. This is what Chilema VSL Group members had to say: “Through the VSL we’re buying school uniforms, paying school fees for both secondary and college students, buying clothes, doing house construction, and purchasing livestock such as goats. We’re accumulating assets such as sewing machines through the money from VSLs too. We’re also using monies from VSLs to finance farming activities like buying seeds, fertilizers and water-canes for small-scale irrigation.” These benefits were also noted in the community observations and was revealed to be a widespread opinion when the Nsanje Community Development Officer was contacted. Despite the positive strides which VSL groups have made in the area, at the time of the evaluation it became apparent that there were challenges which the beneficiaries in the community banks were facing. 6 out of 8 (75%) of the VSL FGDs disclosed that there was increased numbers of loan defaulters and VSLs lacked capacity on how to handle these loan defaulters. Even the police and traditional leaders were failing to deal with this problem once-for-all. Furthermore, it was mentioned by 60% of the VSL FGDs that male participation was very low, with only 26% of male participants in the VSLAs. The gender bias in VSL involvement stemmed from a lack of interest from the men in joining the groups. In conclusion, are the NERRB’s established VSLs sustainable? "We have much optimism that we’ll continue the VSLAs, even though NERRB is phasing out. We believe this is possible because we’ve the required knowledge on VSL methodology and management. Henceforth, we’re capable of standing on our own and continuing with VSL activities. In addition, we’ll sustain VSL groups because we’ve cherished its benefits and we’re more than motivated to continue enjoying the benefits and returns of the VSLs.”-A compendium of the combined voices of VSL Members from Madani, Chilema, Kalumbi and Fransiku GVHs, Nsanje District. 39 Annex 2: Evaluation Tools and ToRs 1. FGD Checklist for Men and Women 2. General Checklist 3. NERRB- Community Observations Checklist 4. NERRB- FGD Checklist for Community Structures 5. NERRB- FGD Checklist for Men & Women 6. NERRB- KII Checklist 7. OFDA 2 Final Evaluation TORs 8. NERRB Evaluation - Respondents Sampling and Booking Form 9. Sampling and Booking Form 10. Youths FGD Guiding Tool NERRB Evaluation Tools &ToRs.zip Annex 3: Inception Report OFDA2 Evaluation Inception Report-240718.docx 40 NERRB Evaluation Report Concern Worldwide-Malawi Plot No. 227, Area 3, Mtunthama Drive Post Office Box 1747, LILONGWE, MALAWI. Copyright © Wednesday, August, 06th 2018. All Rights Reserved.