Sabal Midterm Evaluation Report TANGO International, Inc. Tim Finan, Team Leader David Doledec, Health and Nutrition Hemant Ojha, DRR/CCA Jeff Gray, Livelihoods Opper Maravanyika, Gender Monica Mueller, Report Coordinator/ QA 9/28/2017 i Acknowledgements The success of an evaluation depends on support from many corners and so it was with this one. Our team was warmly received by the Sabal country office whose staff provided us generous amounts of their time and expertise. The Sabal Chief of Party, Robert Cardinalli, was open and forthright and provided many insights, as did his senior management team. We wish to acknowledge specifically the consistent backstopping provided by the Senior MEAL Manager, Sriju Sharma, who attended with patience and grace to our never-ending requests for project information, and Samina Pradhan, who organized schedules, drivers, meetings, and reservations. We are very grateful to both of them and wish to recognize their professionalism and consideration. The team also acknowledges the senior management of the consortium partners who shared their frank assessments of the project with us. The Sabal district management and staff in the three districts we visited invested great effort in assuring that the evaluation team was provided an objective and comprehensive view of Sabal in the field. The Sabal and PNGO staff demonstrated to us their dedication to the project goals and to the well-being of the population they serve. To travel to rural areas in Nepal is a challenge, at times a harrowing one, and we must acknowledge the skill and experience of the three drivers who took us up and down the hills. We are immensely grateful to the four Nepali interpreters who were by our side through all interviews and came to be our assistant researchers. We could not have been better served. Finally, the team wishes to thank the six project communities that embraced us so warmly and shared with us so much of their reality. We marvel at the human warmth and inner resilience of people who face such difficult challenges on a daily basis, and we hope that Sabal will bring them opportunities to define a less-constrained future. ii Table of Contents Acknowledgements.....................................................................................................................i Table of Contents.......................................................................................................................ii Table of Tables ..........................................................................................................................v Acronyms..................................................................................................................................vi Executive Summary.................................................................................................................vii I. Introduction ............................................................................................................................1 II. Evaluation Objectives and Methodology ..............................................................................1 Evaluation Objectives............................................................................................................1 Evaluation Team....................................................................................................................2 Methodology and Evaluation Process....................................................................................2 Theory of Change ..................................................................................................................3 Limitations and Challenges....................................................................................................3 III. Review of Project Design ....................................................................................................4 Appropriateness of Design.....................................................................................................4 Geographic and Overall Beneficiary Targeting.....................................................................5 Design Modification I – Post-earthquake (2015)...................................................................6 Design Modification II – Scope and Coverage (2017)..........................................................6 IV. Results..................................................................................................................................7 Purpose 1: Livelihoods ..........................................................................................................7 Summary of activities........................................................................................................7 Targeting............................................................................................................................8 Findings: effectiveness.......................................................................................................8 Findings: evidence of change...........................................................................................11 Conclusions – Purpose 1..................................................................................................13 Purpose 1 recommendations............................................................................................13 Purpose 2: Health and Nutrition ..........................................................................................14 Brief description of activities...........................................................................................14 Relevance to TOC............................................................................................................14 Targeting..........................................................................................................................15 Findings: effectiveness.....................................................................................................16 Findings: evidence of change...........................................................................................19 Conclusions – Purpose 2..................................................................................................21 Purpose 2 recommendations............................................................................................21 Purpose 3: DRR/CCA..........................................................................................................21 Brief description of activities...........................................................................................21 Relevance to TOC............................................................................................................22 iii Targeting..........................................................................................................................23 Findings: effectiveness.....................................................................................................24 Findings: evidence of change...........................................................................................26 Conclusions – Purpose 3..................................................................................................28 Purpose 3 recommendations............................................................................................29 V. Gender and Social Inclusion ...............................................................................................30 Brief description of activities...........................................................................................30 Relevance to TOC............................................................................................................30 Targeting..........................................................................................................................31 Findings: effectiveness.....................................................................................................32 Findings: evidence of change...........................................................................................34 Conclusions – Gender and Social Inclusion ....................................................................36 Gender and Social Inclusion recommendations...............................................................36 VI. Progress toward Resilience................................................................................................37 VII. Factors Affecting Results.................................................................................................38 Delay in project implementation..........................................................................................38 Partnerships..........................................................................................................................38 Project management.............................................................................................................39 Delivery systems..................................................................................................................40 Technical support and frontline staff...............................................................................40 Training and technical capacity of staff...........................................................................40 Cascade approach as model of technology transfer.........................................................41 M&E system ........................................................................................................................41 Social and behavioral change communication (SBCC).......................................................42 Environmental compliance ..................................................................................................42 VIII. Overall Conclusions........................................................................................................43 IX. Project-wide recommendations .........................................................................................45 Community groups...............................................................................................................45 Management.........................................................................................................................46 M&E ....................................................................................................................................46 Relationships with new administration................................................................................47 Project extension..................................................................................................................47 Annex A: Map of Original + Five Expanded Districts............................................................48 Annex B: Interview Guides .....................................................................................................49 Purpose 1: Livelihoods ........................................................................................................49 Purpose 2: Health and Nutrition ..........................................................................................52 Purpose 3: DRR/CCA..........................................................................................................55 iv Gender Equity and Social Inclusion ....................................................................................58 Annex C: Fieldwork Schedule .................................................................................................62 Annex D: List of Interviews and Focus Groups ......................................................................69 Annex E: Map of Original + Two Expanded Districts............................................................77 Annex F: Livelihoods Implementation Structure ....................................................................78 Annex G: Targeting for Technical Interventions (diagram)....................................................79 v Table of Tables Table 1: Purpose 1 – livelihoods outputs...................................................................................9 Table 2: Purpose 1 – livelihoods outcomes .............................................................................13 Table 3: Purpose 2 – health and nutrition outputs (selected)...................................................17 Table 4: Purpose 2 – health and livelihood outcomes .............................................................19 Table 5: Purpose 3 – Selected DRR/CCA outputs (till Sept 2017) .........................................24 Table 6: Purpose 3 – DRR/CCA outcomes..............................................................................27 vi Acronyms AMS Annual monitoring survey ARR Annual Results Report CCA Climate change adaptation CDMC Community disaster risk management committee CDO Chief district officer DIP Detailed Implementation Plan DPHO District public health officer DPRP Disaster preparedness and response plan DQA Data Quality Assessment DRR Disaster risk reduction EHFP Enhanced homestead food production ENA/ EHA Essential Nutrition Actions / Essential Hygiene Actions FCHV Female community health volunteer FFP Food for Peace GBV Gender-based violence GESI Gender equality and social inclusion GoN Government of Nepal HSS Health System Strengthening IMAM Integrated management of acute malnutrition IPTT Indicator Performance Tracking Table LDMC Local disaster management committee LRP Local resource person MEAL Monitoring Evaluation Accountability and Learning MHI Micro-Health Insurance MoH Ministry of Health MOFALD Ministry of Federal Affairs and Local Development MTE Midterm evaluation NGO Partner non-governmental organization ODF Open-defecation free PLW Pregnant and lactating women PNGO Partner non-governmental organization PVSE Poor, vulnerable, and socially excluded RMS Recurrent monitoring system SBCC Social and behavioral change communication SCUS Save the Children US SIYB Start and Improve Your Business TOC Theory of change VDC Village Development Committee VMF Village model farmer VSLA Village Savings and Loan Association WASH Water, sanitation and hygiene vii Executive Summary The goal of the Sabal project is to increase the resilience and food security of targeted vulnerable populations in Nepal. The project, funded by USAID 2014-19, commenced in 2014 and is scheduled to end 2019. Sabal is currently implemented by a consortium of seven technical partners and seventeen local NGO implementing partners. Save the Children is the overall program leader and primarily responsible for implementation. The project has three primary “purposes”: 1) livelihoods, 2) health and nutrition, and 3) disaster risk reduction/climate change adaptation (DRR/CCA), as well as a cross-cutting component for gender and social inclusion. Under each design element are sub-purposes and accompanying activities that are meant to achieve sub-purpose goals but also to contribute to cross-purpose outcomes, as part of the desired integration strategy. In May 2017, Save the Children US (SCUS) contracted a five-member independent team of evaluators led by TANGO International to conduct a midterm evaluation (MTE) of Sabal. As a process evaluation, it has followed a qualitative methodological approach consisting of project document review, phone and Skype interviews with Food for Peace Washington and SCUS Washington staff, an inception visit (May) to refine the methodology, and a 20-day evaluation mission (July) in country. Mission activities included focus group discussions and key informant interviews at district and VDC levels, emphasizing project stakeholders such as district staff, partner NGO staff, and government counterparts, including district officers and newly elected political leaders. The evaluation also availed itself of monitoring data from the MEAL unit, though due to data backlog, the most recent data available are from the 2016 Annual Results Report (ARR).1 Now that the project is back on track with beneficiary registration data and indicator monitoring, partial reporting on some 2017 indicators is possible, although the overall lack of achievement data has been a limitation to the quantitative analysis of project performance. After its start-up in late 2014, the Sabal project experienced a significant delay in the rollout of its activities due to a number of factors, including the catastrophic earthquake that struck the project area in early 2015. Sabal began in six districts in the intermontane regions of central and eastern Nepal; but after the earthquake, Sabal activities were expanded into five additional districts. This expansion put great pressure on management capacity, and the targeting and registration of beneficiaries required much time and effort. It is also likely that project goals in terms of targeted achievements were too ambitious within the local reality and had to be readjusted. In addition, project management at multiple levels (including Washington) failed to reach a consensus on the monitoring and evaluation system until 2016. As a result, project progress was slow and choppy with most achievements failing to meet targeted goals. In 2017, a fast-track strategy was approved, and targets were adjusted. Accordingly, project implementation is now in full force, and the delivery of project interventions has achieved the necessary implementation pace. The MEAL monitoring and evaluation component has also caught up and is providing appropriate feedback on activities and indicators. The uniqueness of the Sabal project, as emphasized in its theory of change (TOC), lies in its multi-dimensional focus on an integrated approach to build resilience. From the perspective of the MTE team, the core assumptions are that livelihoods interventions, layered with health 1 The MTE team is aware that two Data Quality Assessments (DQA), one carried out in September 2016 and another in January 2017, found significant flaws in some of the reviewed indicators, and USAID did not accept the data in the 2016 ARR. It is further acknowledged that the Sabal project management in response to the DQAs addressed the issues of concern and revised the system. viii and nutrition interventions, and layered with an improved capacity to manage disaster risks and climate change, interact in a synergistic way to enhance food security and community and household resilience. Interwoven through these three dimensions of project intervention (purposes) is the focus on gender and socially excluded groups. One of the Sabal challenges has been to assure that this integration is occurring throughout the delivery systems. Sabal mainly targets the most vulnerable households, and it spent almost a year conducting a targeting and registration protocol to define its beneficiary population. Although the process was time-consuming, the project has successfully targeted the resource-poor and socially￾excluded groups. The Sabal project has demonstrated that its overall design and TOC with some significant adjustments define an appropriate pathway to achieving project goals. The integrated multi￾purpose approach inherent in the project TOC, although as yet not fully realized, holds promise for enhancing local-level resilience and improving the well-being of the most vulnerable populations in the target area. On the other hand, the MTE team finds a significant weakness in the delivery system process that impedes the critical transition from access to new knowledge to behavioral change. The key channels through which behavioral change must pass are the community groups— farmers groups, mothers groups, and community disaster risk management committees. To insure effectiveness and sustainability of Sabal, these local institutions must be strengthened significantly. This is essentially a project based on sustained knowledge/technology transfer under the understanding that access to technical knowledge results in adoption of practices that lead to measurable change. This assumption is flawed in Sabal for two reasons. First, the project has invested heavily in the “supply” side of knowledge: it has marshalled multiple levels of technical expertise both in country and externally, and developed a top-down, cascade system of knowledge transfer to achieve desired scale. Despite, however, baseline studies and assessments, the “demand” side has not been adequately developed, and the community groups are for the most part mere recipients of such knowledge without much say in what trainings would be preferred and in what order. There is little actual ownership of the process by these community groups. The second component of this flawed assumption is that lack of knowledge is the principal constraint to change. The visits to beneficiary communities made it exceedingly clear that this is not the case. In such a vulnerable population, capital constraints are as binding as knowledge constraints. People can attend a training on tunnel hothouse vegetable production or micro-irrigation technologies, but they do not have the financial capacity to invest in the necessary equipment. The provisioning of such assets to a lead farmer (to achieve a demonstration effect) has limited impact upon rates of adoption precisely because of the capital constraint. Similarly, on the demand side, the project presence is simply not adequate in the communities. The Sabal operations manual states that monthly meetings with community groups are the desired routine; however, there are not enough field personnel to achieve this frequency of contact. The Sabal facilitators and local resource persons (LRPs) are overworked and cannot possibly spend the needed amount of time with each farmers group given the constraints of geography and infrastructure. It is also apparent that frontline staff are not adequately trained to develop the institutional maturity of these groups and cannot provide sustained technical assistance. Their tasks are predominantly organizational— scheduling trainings in the action plan and making sure that everyone shows up—when more substantial cultivation of group empowerment is called for. ix The ultimate success of the Sabal project will be judged by the strength of community-level institutions nurtured by the project, particularly the farmer groups, and the post-project environment of technical assistance under the new administrative structure. The MTE team appreciates the value of the other sets of project interventions, especially those that build the capacity of government counterparts and enable a sustainable flow of enhanced public services in agriculture, livestock, health, and local disaster management. Those interventions that diversify household livelihoods (e.g., vocational training, youth business start-ups, veterinary services entrepreneurs) and support increased market linkages and more professional farming are equally important, if somewhat limited in scope and impact. The safety net interventions are both traditional (cash for work) and more innovative (crop and livestock insurance, rural health insurance), and the sustained impacts will reveal themselves more into the future. The key focus of the project, however, should be on the development of community and local institutions (including the Village Savings and Loan Associations) and empowering them to plan and negotiate their own futures. There are important issues in coordination among consortium partners and with partner NGOs (PNGOs) that should be addressed at the midterm. The maximum use of complementary pools of expertise, and productive, creative exchange of ideas among the consortium partners, appear to have fallen victim to management bureaucracy. Partner interactions, even within Sabal, are mostly limited to contractual obligations, developing work plans, and measuring achievements, and there is inadequate space for collaborative problem solving, reflection, and common visioning. The lack of a more productive communication process thus affects the cross-purpose integration upon which the project approach is built. With regard to the PNGOs, the sustainability of Sabal initiatives will eventually rest with local technical assistance groups, which implies the need to assure strong PNGO capacity and an expanded PNGO role in strategic planning within the project. The importance of the new constitutional configuration in Nepal is both a challenge and an opportunity for Sabal over the next two years. The restructuring of politico-administrative units into rural municipalities with an elected leadership will eliminate the geographical units that Sabal targets. Government services, many of them counterparts of Sabal, will similarly be reconfigured in terms of coverage and institutional jurisdictions. Under the new system, rural municipalities control their budgets and provide public services to their citizens; how the current civil service will be reorganized is not yet defined. The challenge for Sabal will be to understand how its coverage areas are adjusted (if at all) and how counterpart relationships will be defined. From the project perspective, this should be seen as a major opportunity; for it is widely thought that the elected leaders of the rural municipalities will work closely with established community groups, such as those being supported by Sabal. It is likely that new system will require strong support from the NGO sector as it is defined and rolled out. In reviewing the many activities of Sabal, the MTE team recognizes the tremendous challenge of managing such a complex project, and it considers Sabal to be on track. The project does require, however, some significant changes in its approach and its coordination in order to establish a sustainable dynamic of local-level change that will continue to build community and household resilience and leave behind viable local institutions. The following recommendations are project-wide and complement the specific purpose recommendations listed above. It is important to note that several recommendations are interrelated. Provide innovative approaches that strengthen community groups. x 1. To achieve sustainable behavioral change, the project must seek to strengthen local community groups, particularly the 5,000 or more farmer groups. The explicit goal should be to build the capacity of these groups as community-level institutions that ultimately enjoy formal status. To further address the importance of community strengthening, the MTE team suggests stronger linkages among the various local groups mobilized by Sabal—farmer groups, mothers groups, and local disaster management committees. They currently operate in an isolated fashion, each attended by a different field staff person, each oblivious to the activities of the others. A more desirable scenario would imply regular and systematic communication among these groups and the staff that support them. 2. To move toward that goal, the team recommends adopting a site-specific, integrated group planning approach in which the individual Sabal groups, in consultation with frontline staff, decide what priorities and course of action they wish to pursue over a project year. In such an approach, instead of presenting beneficiaries with a set of pre￾arranged trainings, the frontline staff work with farmer groups to decide what problems they wish to solve (or begin solving) over the course of a year. Each farm group sets up its plan for change with specific goals and concrete actions. Frontline staff compile these plans, and district-level technical staff develop the annual Sabal work plan as a response to the priorities expressed by the farmer groups. 3. Under this problem-solving approach, the role of the Sabal facilitator and the LRP entails more frequent monitoring of farmer group progress and a greater focus on problem￾solving technical assistance. To assure adequate technical support to the farmer groups, the number of Sabal facilitators and LRPs should be doubled in each focus VDC. Furthermore, the need for greater technical assistance implies that Sabal and PNGO technical staff increase their field interaction with the farmer groups. 4. The capacity building of Sabal facilitators and LRPs should be expanded to include facilitation skills focused on farmer group support and technology change. 5. The individual farmer group plans will require some start-up assets to overcome cash constraints. It is recommended that a combination packages of technology transfer and asset support tailored to farmer group plans be tested in a sample of VDCs. 6. Include trainings that deliver organizational and decision-making skills to the farmer groups. Improve management coordination. The MTE team acknowledges that this set of recommendations will require a significant reallocation of project resources as more frontline staff are hired, their skill sets expanded, and asset packages added in some of the VDCs. The potential impact on the positive outcomes of the entire project merits this shifting of resources, if only on a limited basis. 7. Over the next two years, reinforce the integration of purpose-based interventions in a more explicit, vision-led manner at all levels of management. Most of the integration thinking and reflection has occurred at the central level, while at the district and frontline levels, the integration theme is relegated to the shadows as staff focus on concrete input targets. Integration across purposes and cross-cutting themes requires creative problem￾solving, readjustment of intervention packages, and a project-wide appreciation of how important integration is to the project resilience goal. xi 8. It is recommended that project management define strategies to increase ownership of the project across the consortium partnership. Now partner meetings are quarterly and are mostly perfunctory communications on progress. The MTE team recommends a series of reflection meetings in the fourth year in an effort to expand partner contributions to strategic planning and respective roles, particularly in light of the institutional strengthening approach recommended above. 9. A similar advance toward partner “restart” is needed for the PNGOs as part of a broader exit strategy that maximizes the sustainability of project interventions and impacts. This restart can be initiated with more frank and open discussions of capacity-building needs at the PNGO institutional level and the staff level; it can be further enhanced by more frequent reflection sessions (now once a year) in which frontline staff are given a voice. 10. It is recommended that more opportunities for sharing of lessons learned be generated across districts and across purposes as well as between central and district levels. These exchanges will help disseminate positive outcomes and impacts and enhance the culture of integration that currently is underdeveloped. 11. Under the approach recommended above, it is recommended that the activities of the SBCC unit be intensified to promote behavioral change directly among the community groups, disseminate their community plans, and give the groups an individualized identity as a means of encouraging effective institutionalization of their collective actions. Make M&E more responsive to project needs and less determined by external demands. 12. The MTE team realizes that USAID FFP and FTF require that sets of indicators be tracked globally. Many of the 96 required project indicators, however, are of little use as feedback to the progress of the project. Moreover, many are highly time-consuming. It is thus recommended that Sabal management, at this midterm opportunity, review the tracking table with SCUS project management to reduce the overall number of indicators and cull those that do little to inform project activities and progress. 13. At the same time, should Sabal agree to accept recommendations 1-5, it is recommended that qualitative monitoring be extended to farmer groups to mark the progress of their individual problem-solving plans, according to agreed-upon milestones. This indicator would focus on behavioral change and should involve the participation of the SBCC team. 14. It is also recommended that with the analytical capacity of OPMIS, MEAL begin to track the level of intervention integration and the beneficiary subset that participates in cross￾purpose activities. Pursue articulation with the newly elected bodies in the project area. 15. It is recommended that Sabal management, in coordination with the country director, initiate a dialogue with locally elected leaders at the rural municipality and (new) ward level as a proactive strategy to adapt Sabal interventions to the reorganized system. In these discussions, the role of farmer groups as institutional points of entry for government programs and assistance should be prominent. Disaster management committees and plans will also be impacted by the new leadership and the structure of local government, which requires that the Sabal team discuss adjustment strategies with the local governments. xii Extend support for Sabal for an additional year. 16. The MTE team recommends a one-year extension of the Sabal project. This recommendation is based on three factors: the initial delay in project rollout, the current momentum that the project implementation schedule enjoys, and, most importantly, the opportunity to assist the government of Nepal during its critical transition period to elected local government responsible for providing public services to its citizenry. Sabal is well positioned with strong government counterpart relationships to contribute to this transition period. In sum, the benefits generated by Sabal are now in ascension, and the full sustainable potential of the project could be realized with another year of activity. 1 I. Introduction The goal of the Sabal project is to increase the resilience and food security of targeted vulnerable populations in Nepal. The project commenced in 2015 in six districts in central and eastern Nepal; after the April 2015 earthquakes, Sabal expanded programming to include an additional five of the most severely affected districts (see Annex A). Sabal is currently implemented by a consortium of seven technical partners and seventeen local NGO implementing partners. Save the Children is the overall program leader and primarily responsible for implementation. By the end of FY17 (Sept 2017), Sabal will have phased out active operations in three expansion districts and will continue programming in eight of the eleven districts from FY18 through the life of the agreement. The project is funded by USAID 2014-19. The Sabal project design (and management) is segmented into three primary “purposes” of livelihoods, health and nutrition, and disaster risk reduction/climate change adaptation (DRR/CCA), with a cross-cutting component for gender and social inclusion. Under each design element are sub-purposes and accompanying activities that are meant to achieve sub￾purpose goals but also to contribute to cross-purpose outcomes, as part of the desired integration strategy. In May – July 2017, an independent consultant team conducted a mid-term evaluation (MTE) of Sabal. This is a report of that evaluation. II. Evaluation Objectives and Methodology Evaluation Objectives The MTE is a process evaluation with the following four objectives, with emphasis on the first objective: 1. Assess the quality of Sabal’s program inputs, implementation activities, and outputs in terms of meeting terms agreed with USAID Food for Peace (FFP) and their acceptability and usefulness to the targeted communities. Identify factors that appear to enhance or detract from the efficiency, quality, effectiveness, sustainability, and acceptability of the activities’ implementation and outputs. Examine and highlight any significant differences between districts and clusters. 2. Present evidence of change associated with Sabal’s program activities and outputs. Assess how well the observed changes support the Theory of Change (ToC), and identify factors in the implementation or context that impede or promote the achievement of target results. a. Determine how well outcomes, systems, and services are designed to be sustainable, and the progress towards implementing sustainability strategies. b. Determine how effectively Sabal has mainstreamed and implemented GESI with regard to and control over resources, meaningful participation, increased ability for decision-making, and decreased workload towards social equality. 3. Review and assess the processes for capturing and documenting lessons learned, including feedback from the perspective of stakeholders and beneficiaries. Assess the extent to which these systems are used to refine project design, adjust implementation, and enhance project efficiency or effectiveness. 2 4. Recommend adjustments to the Sabal TOC, activity design, resource allocation, activity management, M&E plan, or implementation that could improve the likelihood of achieving desired results by activity end. Evaluation Team The MTE was conducted by a five-member team of consultants with technical expertise relevant to the project purposes: agriculture and livelihoods, health and nutrition, DRR/CCA, and gender. All team members hail from outside organizations, save for one employed by Save the Children in another region. While this team was assembled by Save the Children US (SCUS), no team member participated in the design, implementation, or management of Sabal, and the evaluation represents an independent assessment of the project’s approach and performance. The team leader was contracted by TANGO International. Methodology and Evaluation Process This evaluation followed a qualitative methodological approach consisting of the review of project documents, phone and Skype interviews with the FFP Washington office and SCUS Washington staff, an inception visit (late May) to refine the methodology, and a 20-day team visit in country (July). The team was provided access to a collection of project documents and other relevant literature through a DropBox site that was regularly revised and expanded at the request of the MTE. The inception phase included document review, defining the evaluation methodology, drafting the evaluation protocol and data collection tools, and drafting and finalizing the evaluation plan and workplan. In June, the team leader and TANGO project manager conducted a four-day inception mission to Kathmandu and met with key staff from Sabal, USAID Food for Peace, and consortium partners, as well as ministry representatives, to come to a common understanding of the evaluation purpose and scope, solicit input on the evaluation approach and workplan, and obtain information relevant to refining all aspects of evaluation planning. The inception visit was highly valuable in presenting an overall orientation to the project and its challenges and in meeting the wide range of Kathmandu-based consortium partners and Government of Nepal (GoN) counterpart agencies. The team arrived in Kathmandu on July 7 for the evaluation mission. After two days of introduction and organizational meetings, the team spent three days in Kathmandu interviewing Sabal senior management and technical staff, consortium partner organizations, and GoN counterparts. Following the central-level interviews, the team began an eight-day visit to project sites in the rural districts, which provided a particularly rich source of insights into project strengths and weaknesses. During the inception visit, it had been decided based on available information that the MTE team would visit three districts of the eight now covered by Sabal—one (Kavre Palanchok) from the expanded districts (see below), and two (Sindhuli, Okhaldhunga) from the original project area. The two original districts were selected because they represented different agro-ecological zones and livelihoods as well as different levels of access (Okhaldhunga being of higher elevation and more remote). In each district, two Village Development Councils (VDCs) were selected to reflect livelihood differences and levels of access. In each district, the team carried out a series of key informant interviews and focus group discussions at the district and VDC levels (see Annex B, Interview Guides and Annex C, Fieldwork Schedule). The interview guides were developed by the JMTR team in the inception stage and shared with Sabal management and Save HQ staff for comment prior to the commencement of fieldwork; the team also prepared and received approval of an 3 inception note2 that outlined the evaluation approach including methodology, sampling strategy, phases/process, and ethical considerations. At each district center, the team as a whole initially met with district staff, PNGO staff, and GoN counterparts, including district officers and newly elected political leaders. After the introductory meetings, the MTE team met individually with district-level technical staff. At the VDC level, introductory meetings were followed by team breakout sessions with the relevant beneficiary group(s) and frontline staff. In all, the MTE team interviewed approximately 180 stakeholders and held 52 focus group discussions (see lists in Annex D) in the sampled districts. Four professional interpreters accompanied the four non-Nepali-speaking team members for all interview sessions. Theory of Change The innovativeness of the Sabal project lies in its integrated strategy to enhance food security in a sustained, resilient way. The theory of change (TOC) builds upon the assumption that households and communities in the Sabal districts face a multidimensional set of challenges that include low agricultural productivity, insufficient income-earning activities, inadequate health and nutrition, and exposure to both natural and economic shocks and stressors. Furthermore, it assumes that food security, income, health, and nutrition and disaster risk reduction are structurally interrelated and mutually reinforcing, so that resilience outcomes are generated when all these components are addressed in a layered fashion. The TOC further recognizes that poverty in the targeted area is significantly related to enrooted forms of social exclusion based on caste, ethnicity, and gender, and its strategy is designed to address these sources of structural inequality. Finally, it is assumed that the sustainability of Sabal interventions, outcomes, and impacts are based on building local capacity—of both government partners and local development actors. Based on these assumptions of the need for integration and social inclusion, the TOC lays out a comprehensive approach that specifies desired outcomes with strategic project interventions in the three purpose areas of the project: livelihoods; health and nutrition; and disaster risk management (DRR) and climate change adaptation (CCA). Social inclusion and gender strategies are also systematically embedded throughout the design of the TOC. The midterm evaluation (MTE) team oriented its field inquiry and based much of its analysis on the TOC framework and design. The TOC reflects the overall complexity of the Sabal project and the synergies expected from its many subcomponents in achieving designed outcomes. A key element of the MTE approach has been to determine the extent to which these strategic components are integrated in field activities, the extent to which Sabal and counterpart staff understand the critical importance of integration, and how these expected synergies are evidenced in the Sabal communities. Limitations and Challenges One of the limitations of the evaluation was the lack of a complete set of project results (e.g., targets vs. achievements) due to a backlog of data to be entered. Most achievements reported here are thus from the 2016 Annual Results Report (ARR). With regard to the fieldwork, the visits to the districts and VDCs proved decidedly fruitful, but the team faced certain challenges due to project site remoteness and difficulty of access. It required a tremendous amount of time to cover relatively short distances because of road quality and elevation. To reach one VDC (in Sindhuli), the team had to cross a fast-flowing river by car, and the river 2 TANGO International. 2017. Sabal Midterm Evaluation Inception Note. 27 June. 4 was rising rapidly due to upland rains, so it was decided to choose a replacement VDC that was accessible. Moreover, in each VDC visit, the MTE team routinely found all the PNGO frontline staff and beneficiary groups concentrated at the site to welcome them, in part for convenience sake. This lent itself to an atmosphere of expectation on the part of the beneficiaries, and it was necessary to clarify the reason for the visit. Individual and group breakout interviews were then distributed around different points around the community to minimize the public formality of the visit. With centralized interviews, however, it was difficult for team members to move to more-private spaces and to observe agricultural fields and other livelihood and DRR features. Despite some limitations on reaching the more isolated communities, the MTE team encountered a certain uniformity in terms of project implementation issues, even in the face of agro-ecological diversity. III. Review of Project Design Appropriateness of Design The principal design characteristics reviewed by the MTE team were its complexity, the number of interventions, the synergy between these interventions, and the effectiveness of the delivery systems that translate these interventions into activities implemented at the field level. The complexity of the project design reflects that of the TOC itself. The commitment to an integrated, layered approach to food security and resilience finds support from the experience of previous projects (e.g., PRIME in Ethiopia, SAREL in West Africa). The MTE considered the project design consistent with the TOC and appropriate to the overall goals of Sabal. The MTE team did identify, however, design flaws that could be corrected. Regarding the complexity of the project design, the mechanisms to assure integration across purposes are not sufficiently explicit. Given the logic of the TOC, the project design needs a more prominent strategy to reduce the tendency to isolate activities under a purpose heading and to maximize the awareness of integrative activities and outcomes. The MTE team recognizes that the management decision to assemble consortium partner staff in the same Sabal office space (at both central and district levels) was an enabling factor favoring integration, but a shared “creed” of integration was not explicitly established in the design. After frank discussions with project management at several levels, the MTE team agrees that the Sabal design had too many partners and too many interventions. It is not clear to the team if during the design phase the large number of consortium partners resulted in the large number of interventions (each partner defining its activity area) or the reverse (i.e., a large number of interventions required the assemblage of a larger team of partners). It is clear, however, the participation of many partners and the wide set of interventions has stressed the management capacity of the project, and a leaner design with fewer interventions (discussed below) would increase its effectiveness. The large number of interventions associated with the different purposes naturally leads to a prioritization process in which some interventions are more “critical” to project success than others. As a result, rollout rates and coverage areas vary for reasons not always explained by the logic of project design. In fact, as a design strategy, some interventions were introduced as innovative pilots and others as minor appendages to a larger sub-purpose. The MTE team questions whether the beneficiary impacts expected from some of these lower-priority interventions merit the management effort required to implement and monitor them. The original project design also lacked a greater focus on the crossover potential of the intervention set. In the livelihoods purpose, for example, the multiple interventions appear to 5 be isolated menu items without adequate regard for the community-level context and the problems needed to be solved. The design of interventions should be crafted with an eye to their interactive impact—e.g., how an agricultural intervention might support a livestock activity or how increased income might result in investments in farming. In the same spirit, the interventions should be assessed in terms of how they contribute to the resilience goals expressed in the TOC. As an example, safety net interventions, such as cash for work, could be directly integrated with the goals of DRR/CCA. The delivery system is also a matter of concern for the MTE team. The principal design mechanism for delivery of project interventions uses a cascade approach: knowledge is transferred through a chain of actors using a training-of-trainers strategy to maximize scale of coverage. Lead trainers are drawn from diverse pools of technical expertise (e.g., post-harvest technologies); they train “local resource persons” (LRPs) and “lead farmers” who then provide the technical messages to beneficiary groups organized by the frontline Sabal facilitators. The LRP is usually a local community member and frontline staff; there are 1-2 LRPs per VDC. The weakness in this delivery design is that the frontline staff (both the facilitator and the LRP) are often not adequately prepared technically to meet the complex needs of beneficiaries; furthermore the frontline is understaffed and the VDC-wide coverage demanded of each LRP and facilitator is unrealistic given the difficulty of access. It is true that the cascade approach can prove effective as a means of delivering concrete information, such as messages on breast-feeding or child nutrition; but its impact diminishes when the activity implies longer-term behavioral change and the monitoring of group organization and progress. For these types of interventions (e.g., adopting irrigation technology), the cascade approach requires more frequent contact than the delivery system currently provides. Geographic and Overall Beneficiary Targeting Sabal was designed to reach the most vulnerable populations of the project districts. To achieve this targeting goal in the original six districts, the Sabal team implemented a targeting strategy that included official government classifications of VDC vulnerability and a community-based participatory well-being technique to identify the most vulnerable families. In the expansion districts, the process was somewhat accelerated by working with VDCs with approximately 80 percent of vulnerable families, and using local VDC leaders to target existing vulnerable groups. In both cases, each household was registered with a unique identifier in the MEAL database. This process was very systematic and comprehensive, thus time-consuming, and one of the reasons for the slow rollout of project activities. For DRR/CCA work, only four of the Sabal VDCs are targeted in each district, and within these VDCs, only the four most vulnerable wards receive intensive DRR/CCA support services; poor, vulnerable and socially excluded households from the selected wards received DRR/CCA services. Despite the time required, the MTE team found this targeting strategy to be highly effective. It is a very positive achievement of the project at the midterm point that it indeed assists the most vulnerable populations, including socially excluded groups, indigenous communities, and poor, resource-scarce households. In the VDCs the team visited, most of the beneficiaries are women, including participants in the farmer groups and savings groups. One inherent challenge, as pointed out by a Sabal partner, is that targeted households are sometimes so vulnerable and land-scarce that they have difficulty in adopting some of the interventions 6 designed to improve productivity, a point further addressed in the findings under the different purposes. Design Modification I – Post-earthquake (2015) Sabal was officially initiated in late 2014 and catastrophic earthquakes hit the central and eastern mid-montane district in mid-2015. Several of the original project districts were severely affected, as were surrounding districts. In negotiating its disaster response with USAID, Sabal expanded into five of the heavily damaged districts (see Annex A) and introduced an adapted Sabal intervention set for the most vulnerable populations there. The management structure for the expanded districts was adjusted to organize the districts into two management clusters, with one PNGO per district to carry out the activities. Several of the interventions, such as the vocational training program, were revised to respond to the need for building trades, particularly masonry skills. Based on concerns over start-up delays and management capacity, USAID later requested that Sabal reduce the number of expanded districts from five to two (Kavrepalanchowk and Sindhupalchowk) (see Annex E). Consequently, phase-out in the other three districts was initiated and project activities will cease by September 2017. It was not clear to the MTE team what indicators were used to identify the districts selected for phase-out, but certainly the decision was a difficult one for Sabal staff, GoN, and beneficiaries. Efforts have been made to shift beneficiary support to other institutions in the region. Design Modification II – Scope and Coverage (2017) As of September 2017, Sabal activities will cover the six original districts and the two remaining expanded districts. The scope of the project has also been adjusted mid-term. Despite the logic of the TOC, not all VDCs receive the same set of project interventions. For example, in the original six districts, Purpose 2 (Health and Nutrition) activities cover all the VDCs in each district, not just in the Sabal targeted districts. This decision was in response to a government mandate and to the fact that many of the interventions involve the training of government health workers. Accordingly, each of the original districts has adopted the rather awkward terminology of “focus” and “non-focus” VDCs, the latter being those that only receive Purpose 2 interventions. Similarly, in the two expanded districts, only the livelihood and DRR/CCA components are implemented. This decision is based on the presence of a similar USAID-funded project, SUAAHARA II, which provides health and nutrition support. During its visit to the expanded district Kavrepalanchowk, however, the MTE team did not find evidence of SUAAHARA II activities in our sampled VDCs. Similarly, WASH activities and DRR/CCA interventions are not carried out in all the focus VDCs, but target those populations where greater need or vulnerability level has been determined. Also at mid-term, several intervention packages have undergone revision or will phase out. The value chain initiative targeted somewhat less vulnerable households and sought to engage a number of public and private sector actors to develop more vibrant input and output markets for selected cash crops. This effort faced a significant number of constraints related to remoteness, difficulty of access, distrust of buyers, and lack of storage and transportation. The Sabal management team has refashioned the value chain component as a more feasible effort to enhance market participation in ways that reflect contextual realities. In addition, two consortium partners will phase out their activities in 2017: NTAG (health training) and ACF (malnutrition surveillance and response). 7 IV. Results Purpose 1: Livelihoods Summary of activities Sabal’s livelihoods, agriculture, and natural resources management component is designed to increase income and build resilience of vulnerable households and communities. Purpose 1 interventions address multiple factors that contribute to stable household incomes. The livelihoods of the majority of the target population depend on crop farming and livestock. As such, Sabal supports the following major livelihood activities: improving farm production and productivity; transferring employable skills (e.g., construction trades); developing microenterprises (Start and Improve Your Business [SIYB]); increasing access to financial services; strengthening market contacts; and building awareness of and access to social safety nets. A major delivery channel of Purpose 1interventions is through small community group organizations of around 20 members: the farmer groups and the Village Savings and Loan Associations (VSLAs). In the VDCs visited by the MTE team, there is a wide overlap in membership between the two groups. The Sabal field staff mobilize these community groups, and they are the central recipients of the technology transfer trainings that are core to the Purpose 1 intervention set. The VSLA members make small monthly contributions to a common fund (the MTE team observed an amount of 20 rupees per member per month). A primary expected outcome of this activity is that local households will acquire an appreciation for group savings and will at some time become associated with larger savings organizations in the district (i.e., credit cooperatives). Relevance to TOC The livelihoods component is consistent with the TOC, especially in the original districts. Sabal interventions actively address the major sub-purposes of strengthening livelihoods and improving economic risk management capacity by targeting the most marginal and vulnerable populations. The range of interventions enhances absorptive, adaptive, and transformative capacities by increasing the value of farm production, expanding market access, and diversifying household incomes outside of agriculture. KIIs confirmed that the Sabal team conducted a field-based needs assessment prior to deciding on the livelihoods interventions. Examples included mapping of farmer groups and natural resources, assessments of labor and value chain markets, and the evaluation of stakeholder interest in the GoN micro-health insurance (MHI) scheme. This advance work provided the foundation upon which to identify training topics and to guide the selection of appropriate public- and private-sector partners engaged in these trainings. The most frequently cited training courses that support the TOC include adoption of better farming practices, especially the use of improved seed, water and soil management practices, drip irrigation, composting techniques, and crop diversification. Specialized training for Village Animal Health Workers (VAHW) and LRPs is also a key component of the agricultural service delivery system. Also relevant to the TOC are income-generating opportunities derived from vocational training and SIYB training for youth. Kavre and Sindhuli districts reported success stories about Sabal-trained masons, plumbers, carpenters, beauticians, and mechanics who find professional work or become self-employed. For example, in Kavre district alone, 2500 8 masons have been trained and earn the national average wage rate for masons. This initiative has also trained women in these skilled trades. Sabal also offers connections to safety net programs directly tied to the TOC. These programs improve the capacity of vulnerable households to manage risks. They include the GoN￾subsidized MHI, and crop and livestock insurance programs. The insurance initiatives are built upon a public-private partnership with GoN-regulated private insurance companies and, according to Sabal staff, over 6,000 farm animals have been insured to date. Targeting Livelihoods targeting has proved effective. Sabal conducted thorough due diligence and accurately identified the poor and most excluded segments of Nepalese society. The targeting strategy used UNDP Disadvantage Group mapping data, which consist of four poverty categories. Sabal also took into account demographic health survey data concerning the number of children under 5 years of age and ethnicity. In addition, the livelihoods team looked at the market and employment potential for on-farm diversification and off-farm occupations that would be most in demand in the target districts. Sabal livelihoods activities target 264 VDCs that cover 164 original districts and 100 expansion districts. Of all families targeted by Sabal, 80 percent fall into most vulnerable category.3 See Annexes F and G for diagrams showing Sabal livelihoods implementation structure and targeting for technical interventions. Sabal mapped the district agro-ecological zones to select the interventions consistent with existing livelihood patterns. The project focus crops are rice (lowlands), maize and wheat (uplands), off-season commercial vegetable gardening, and spice production in some VDCs. Sabal’s central office technical staff also conducted field visits to meet with stakeholders, PNGOs, and GoN technical services to validate what types of assistance would be most suitable to include under Purpose 1. Once Sabal district technical teams were established and PNGO selection completed, Sabal began the training cycles associated with the cascade approach in the targeted VDCs. With regard to financial services, the primary beneficiaries are farmer groups – more specifically, VSLA members within the farmer groups. Sabal expects to reach over 5,000 farmer groups across 11 districts (approximately 119,000 persons) over the life of project. Registration data show that women comprise 60 percent of all farmer groups, well above the initial target. The primary participants in the employment and enterprise activities are in the poor, vulnerable, and socially excluded (PVSE) category, with a target of 50 percent women. The number of individuals trained in each district is proportional to the number of VDCs. Findings: effectiveness Table 1 presents Sabal livelihoods outputs from 2016 and 2017 (through September). The low percentage of attainment through 2016 is consistent with the struggles Sabal experienced with project start-up, especially missing the window for the cereals crop calendar for starting improved seed multiplication. The table does not reflect the most recent three quarters, in which outputs have dramatically increased; these are anticipated in the FY17 ARR. The SC 3 i.e., those households that lack adequate access to food 12 months out of the year; have poor access to healthcare; and have difficulty responding to shocks. 9 Nepal Sabal IPTT March 15th Submission “Revised Targets” does not yet reflect 2017 revised targets. The number of individuals trained under the indictor SABAL-1 (2,578) represented 1,502 who received vocational skills training and 1,076 who received SIYB training. It is important to note that for SABAL-2, of the actual number trained (663), 82 percent passed their accreditation exam, and 85 percent found employment within three months of completing their training. The number of individuals trained under indicator FFP-11 up to 2016 (15,103) included 299 master trainers/retailers; 100 local service providers/VAHW; 8,630 individuals from farmer groups; and 6,074 model farmers/homestead food producers (mothers). For FY17, indicator FFP-11 estimates that out of the 43,171 individuals targeted, 25 persons will come from GoN, 1,263 from the private sector, 180 from civil society (NGO/CBO/university), and 41,703 from farmer groups. The MTE recognizes that the IPTT FY17 “revised targets” do not yet reflect the reduction of the three expansion districts (at USAID/FFP request) no longer included as part of the program. A revised IPTT with adjusted targets will be included in the 2017 ARR. Table 1: Purpose 1 – livelihoods outputs4 Target FY16 Actual Actual vs planned FY16 Target FY17 Actual to Sept 2017 Actual vs planned to Sept 2017 Vocational training and business training SABAL-1: Number of individuals trained in entrepreneurship and job readiness skills 4,448 2,578 58% 6,233 4,035 64% SABAL-2: Number of vocational trainees who are employed within 3 months of training completioni 1,466 663 45% 3,024 1,500a 50% a FFP-25: Number of medium, small, and micro enterprises, including farmers, receiving business development services from USG-assisted sources 2,309 1,187 51% 24,665 22,736 92% Agriculture and food security FFP-11: Number of individuals who have received USG supported short-term agricultural sector productivity or food security training 26,394 15,103 57% 43,171 90,427 216% FFP-12: Number of food security private enterprises (for profit), producers’ organizations, water users’ associations, women's groups, trade and business associations, and community-based organizations (CBOs) receiving USG assistance 1,250 2,276 182% 6,550 7,931 121% Source: Sabal FY16 ARR; Nepal Sabal IPTT March 15th Submission; includes revised FY17 targets. a/ Will be updated on OPMIS by first week of October 2017. 4 Please refer to footnote 1 above, which details USAID concerns about the quality of these data. 10 After a slow start-up, Sabal livelihoods’ activities have achieved momentum, and a solid foundation is in place to scale up activities. The system for delivering training is working well; however, staffing is a challenge. Below is a brief review of major findings. Project management. Based on KIIs, it is possible to assert that Sabal technical leads grasp the complexities involved in managing the livelihoods component, especially within their respective levels (national, district, local). They have received adequate training on Sabal delivery systems and have established a successful working relationship with technical GoN counterparts. One area of concern is the lack of communication and integration across Sabal components. For example, the livelihoods staff do not work as closely with the health/nutrition staff as they could. A second concern is that there are too many livelihoods interventions overall, which creates an endless demand for staff to organize training, often at the expense of promoting improved practices and effective technology transfer. A careful review of the list of livelihoods activities with an eye to cost-benefit could reduce the number to a more manageable level. Staffing. The turnover of frontline field staff needs to be addressed. As the primary interface with project beneficiaries, these staff serve a critical role in the delivery system, and turnover reduces the quality of technical services and decreases beneficiary monitoring. The high turnover rate also impedes the accumulation of experience by staff members, thus limiting their performance. Suggestions to improve quality include: i) Sabal central office technical leads should conduct more frequent staff and partner on-the-job training; ii) incorporate beneficiary feedback reports in monthly meetings with action items; and iii) institute a system of financial incentives for high-performing field staff. Community engagement and participation. Overall, there is good feedback on the relationships Sabal has formed with the community. Sabal facilitators and other service providers (VAHWs and LRPs) are well integrated in their communities and their roles are understood and respected. Most farmer groups, however, do not play an active leadership role in their communities, which would position them to encourage the adoption of the improved farming practices and technologies provided through the Sabal trainings. Possibly, a pilot institutional strengthening training for select farmer groups in each VDC could lead to a greater sense of ownership and a broader spillover effect of their new knowledge. Partnerships and linkages. The Sabal livelihoods component has established a wide range of successful public and private partnerships to help implement the program. Sabal collaborates with GoN technical services and in array of activities. The Nepal Agricultural Research Council provides access to improved seed and support for seed multiplication farmers; the VAHWs receive their training at the national training center in Pokhara; and the Ministry of Health (MoH) works with Sabal to roll out the new MHI policy. Private sector support comes from independent contractors who deliver agricultural services and goods; local input suppliers (“brick and mortar” stores) such as Agrovet; and several crop and livestock insurance companies, who offer policies to protect farmers from crop and livestock losses. Local partner capacity building. Considerable Sabal energy has gone into building the capacity of GoN technical service partners to develop a multi-faceted, private rural service delivery system via paravets, VAHWs, and LRPs. This rural delivery system has been responsive to the needs of the beneficiary farmers and is a positive step toward project sustainability. The livelihoods activities depend on the PNGO partners, who display varying levels of competency. There does not appear to have been sufficient capacity building with the PNGOs. During a MTE debriefing, PNGO performance was discussed at length by 11 consortium partners. The overall feeling is that Sabal does not engage the PNGOs as effectively as it could, and this can cause tension. GESI progress. Based on MTE team visits to beneficiary VDCs, it was clear that the intended targeting of socially excluded groups and women had been successful. Many of the communities are predominantly low-caste or indigenous households, and they constituted the majority of the farmer groups, VSLAs, and vocational training beneficiaries. Purpose 1 interventions provide opportunities for these households to increase their food security and income and, as importantly, to glean the benefits of collective action and increased access to government services and outside resources. Many of the Sabal facilitators are also members of these excluded groups. In a similar vein, the great majority of Sabal Purpose 1 beneficiaries are women. The interviews revealed that these women are mostly married, and their husbands have either migrated or are engaged in paid day labor. Women tend not to own land separate from their husbands, and they are exceedingly cash-poor. Nonetheless, women do manage much of the farmland and provide the majority of the labor, particularly in transporting inputs and outputs to and from distant fields. Although certain structural forms of gender inequality are not directly addressed by Purpose 1 (e.g., control over income from sales), women appear nonetheless to value the opportunity to increase production, engage in sales (of vegetables, eggs, chickens, small animals), and to increase household food security. Findings: evidence of change Change is evident across the visited districts, particularly for poor marginalized women. According to the field FGDs, females represented around 85 percent of the farmer group/VSLA participants. Explanations for this heavy representation of women farmers range from high levels of male out-migration to male activities in off-farm labor. Women farmers are receptive and eager to learn from Sabal. The most impactful livelihoods interventions and key technology changes noted in these discussions are summarized in Table 2. The small plastic reservoirs located upland for small garden irrigation, combined with drip irrigation plastic hoses, are an ideal technical package for commercial off-season vegetable production. These are good examples of Sabal introducing climate-smart agriculture requiring no external renewable energy source. Farmer group interest in building semi￾permanent water ponds is also increasing. Under the current model, the VDC agrees to contribute cement while Sabal provides cash for work and the salary for an engineer and a master mason. The farmer groups provide the necessary labor. The most widely accepted technology introduced by Sabal is the adoption of improved seed varieties. Benefits include increased yields and reduced loss from pests/plant diseases, and it is easy for farmer group/VSLA members to see the benefits firsthand in a demonstration plot. The strategy of using LRPs/lead farmers to manage seed multiplication demonstration sites and to train farmers groups in the use of improved varieties has been successful. Key informants expressed their appreciation to Sabal for introducing them to off-season vegetable gardening coupled with SIYB training for the additional income it brings to the household. Training in cultivation techniques – such as raised seed beds for growing vegetables; types of irrigation methods and timing schedule for irrigation based on seed, soil and weather; compost making and proper use; pruning and staking tomato plants; and use of plastic greenhouses – are having positive effects, changing the cultivation practices in many farmer groups/VSLAs. Farmers did cite, however, constraints to technology change that should be addressed by the project as it moves forward. Most farmers recognize the advantages of micro-irrigation and 12 tunnel vegetable gardening, but they do not have the available resources to invest in these innovations. They are simply too cash-poor to cover this investment and thus the technology remains out of reach. VSLA member savings levels are too low to generate a capital reserve that members could access for productive investment, and the MTE team did not document a significant level of VSLA integration into larger credit cooperatives. At current levels of group savings, capital constraints remain binding and do not enable the technology change promoted by the project. Sabal’s agricultural technical service delivery using LRPs, VAHWs, and paravets complements GoN technical assistance offered through its agricultural services centers. Sindhuli and Okhaldhunga district officials report these GoN technical services can only meet 18 percent and 35 percent, respectively, of local demand for agriculture extension assistance. District officers recognize that Sabal helps fill this gap. There is evidence that some farmer groups/VSLAs are recognizing the value of having this service delivery and willing to pay a small fee for it. This service delivery could be scaled up if Sabal trained more agro-service providers, which is planned for the future. There is evidence of positive change in the agricultural sector resulting from Sabal’s livelihoods interventions. The program is poised to scale up, but there will be implementation challenges ahead. Major hurdles will be recruiting/retaining qualified staff, and in support operations, including: • improving frontline staff incentives that encourage long-term employment in difficult rural areas; • increasing the monitoring of knowledge transfer to confirm that beneficiaries are adopting the new farming practices/techniques; • supplying relevant assets to trained beneficiaries, e.g., providing tool kits for the skilled trade workforce upon graduation, and equipping paravets with refrigerated canisters for improved breed semen, etc.; and • addressing the low number of qualified field technical staff; Sabal does not need any more staff stationed in Kathmandu. (see table next page) 13 Table 2: Purpose 1 – livelihoods outcomes Color key: Little achievement Partial achievement Significant achievement Sub-purpose Immediate outcome Evidence of change in sample districts SP 1.1 Strengthened livelihoods of communities including marginalized and vulnerable women and men IO 1.1.1 Increased employment & SMEs of vulnerable households in target communities Significant numbers (3,500) of unemployed youth and experienced masons trained to date IO 1.1.2 Increased access to inputs and output markets for vulnerable households Better access to inputs (improved seed, organic composting and pesticide making); minimal achievement on improving market access IO 1.1.3 Increased farm production of vulnerable households Households introduced to commercial vegetable production; cereal production increases show promise using improved seed from farmer seed multipliers IO 1.14 Improved post-harvest management Disseminating improved post￾harvest handling techniques SP 1.2 Improved economic risk management capacity of especially vulnerable men and women IO 1.2.1 Enhanced natural resource management by vulnerable households Farmer group use of climate smart improved seed, organic composting, and drip irrigation IO 1.2.2 Increased access of vulnerable households to social safety net facilities MHI awareness trainings; a number of livestock are insured; no crops insured to date IO 1.2.3 Increased adoption of diversify livelihoods by vulnerable households Introduced SIYB business training to support commercial vegetable farming, goat rearing, and other MSME Conclusions – Purpose 1 Sabal livelihoods’ objective to increase income and build resilience of vulnerable households and communities is apparent in the sampled VDCs. Year 4 must devote attention to scaling and improving operational efficiencies. This requires recognizing best practices and models to make sure Sabal benefits from learning that has occurred. Purpose 1 recommendations 1) To sustain momentum, Sabal must accelerate farmer seed multiplication support to respond to demand created by the program. Seed multiplication activities must also be informed by climate change, and linked to attempts to build adaptive capacities. 2) Key informants reveal there is a high turnover of frontline staff. Sabal could explore the causes. In the meantime, Sabal frontline staff (Sabal facilitators, VAHWs, LRPs, agriculture technicians, agriculture officers) could be encouraged in the following ways: more frequent senior technical staff mentoring; motivators such as performance awards (non-monetary), in-house professional training, and perhaps small bonuses. 3) Strengthen capacity of farmer groups/VSLAs to play a larger role in the community in areas such as organizational development, collective problem-solving, and creating better market opportunities (e.g., through produce collection centers). 14 4) Support the evolution of VSLA growth in the direction of rural credit cooperatives as a way forward, with additional training in business plan development (SIYB), in order to expand into MSME diversification. 5) Key informants and FGDs reveal that beneficiaries think there are too few asset transfers by Sabal to facilitate their adoption of new farming and business practices. Sabal could review its budget to see if it is possible to address this issue without creating dependency. Purpose 2: Health and Nutrition Brief description of activities Purpose 2 describes outcomes and activities from the health and nutrition domains. Its largest component seeks to improve the health and nutrition status of vulnerable households through such activities as training women – especially “1000-days mothers” – on health and nutrition best practices (Essential Nutrition Actions / Essential Hygiene Actions [ENA/EHA] packages), providing equipment for female community health volunteers (FCHVs) who act as trainers, and using village model farmers (VMFs) to provide seeds and chickens to 1000-days women to improve their access to nutrient-dense foods.5 In addition, water, sanitation, and hygiene (WASH) activities are conducted to transfer knowledge on best practices to various community groups and members, and to provide knowledge and equipment to vulnerable households and health facilities to improve practices, services and availability of latrines and hand-washing stations. A third component focuses on health facilities. Equipment (e.g., scales, SBCC materials, posters, family planning kits) and training are provided to facilities and FCHVs to conduct outreach activities, including on contraception. Activities to support the integrated management of acute malnutrition (IMAM) are carried out in one district. Finally, Sabal supports the government in coordination functions, provides training to ministry managers and empowers local governments as master trainers. Relevance to TOC The Sabal TOC generally respects recommended pathways to improve the health and nutrition status of women and children. It integrates elements from the three layers of interventions proposed in the 2013 Lancet series on maternal and child nutrition. 6 These include nutrition-specific interventions such as treating severe acute malnutrition and promoting exclusive breastfeeding for children until six months of age; nutrition-sensitive interventions such as, in the Sabal case, WASH and enhanced homestead food production (EHFP) (represented by the seeds and chicken distribution and by training community members on ENA/EHA and farming); and interventions aiming at building an enabling environment, such as Sabal’s health system strengthening component. The TOC offers a comprehensive approach that addresses the main determinants of poor health and nutrition in Nepal, among which are a lack of knowledge and inadequate childcare practices. Thus, it is relevant that a major component of the Sabal health and nutrition purpose focuses on knowledge transfer to mothers of children under two years of age. Improving household 5 It is important to note that Sabal mobilizes and works directly with 1000-days mothers organized in small groups of around 20 mothers. In addition, each community has larger “mothers’ groups” that are not directly mobilized by Sabal and have a long history of existence in rural Nepal. Sabal also supports these groups through the FCHVs in providing project health and nutrition and hygiene messages. These mothers’ groups are considered important conduits to the community as a whole. 6Black, R.E., Victora, C.G., Walker, S.P., Bhutta, Z.A., Christian, P., De Onis, M., Ezzati, M., Grantham￾McGregor, S., Katz, J., Martorell, R., Uauy, R., 2013, Maternal and child undernutrition and overweight in low￾income and middle-income countries, Lancet Vol. 382 pp. 427-451, available from: http://www.thelancet.com/series/maternal-and-child-nutrition 15 capacity to produce nutrient-dense vegetables is also likely to have an impact on dietary diversity for children and pregnant and lactating women (PLW), while WASH efforts may reduce the vulnerability to water-borne diseases of these two vulnerable population groups. The ToC also defines FCHVs and VMFs as the central resource persons at the community level to deliver project activities. Using FCHVs is a relevant strategy: many women interviewed have been FCHVs for 20 years and have a strong influence on other women, particularly younger women. They are supported by the government and are already integrated into the health system, attending monthly meetings in the health facilities and serving as a liaison between health facilities and communities. Likewise, identifying VMFs to show their demonstration gardens and transmit knowledge to other women is an adequate approach. In several instances, the VMF and FCHV is the same person, which may even increase project impact, as transfer of knowledge on farming and health and nutrition can be done more efficiently by a FCHV well known in the community than by someone in a newly appointed VMF role. The TOC adopts a strong gender and social inclusion lens. There is ample evidence, in Nepal and elsewhere, that gender inequities are a significant determinant of poor nutrition and of women’s limited capacity to access essential services. In Nepal, social exclusion was also reported commonly as a recurrent issue. Adopting a GESI cross-cutting approach is therefore an important step in the right direction. It rightfully aimed at integrating men in the discussion with women during 1000-days mother group meetings and promoting inclusion of the most vulnerable community members. The TOC promoted discussion between men and women in most of its components to ensure gender inequities do not hamper the project’s potential impact. The TOC is relevant as long as a full package of interventions is delivered to households. However, while a health and nutrition purpose was implemented in all 326 VDCs in Sabal’s six original districts, the DRR purpose was only implemented in 100 VDCs and livelihood activities in only 264. Moreover, the health and nutrition purpose was not fully implemented in all VDCs: only 65 VDCs implemented all activities, 164 implemented ENA / EHA training and EHFP, and 162 VDCs implemented only the ENA / EHA training. Resilience is defined in the TOC as the result of synergistic effects from all components of the project, therefore the relevance of providing only ENA / EHA knowledge in 162 VDCs without reinforcing household dietary diversity (e.g., via the EHFP activity) is questionable, as mothers may acquire knowledge but still have little capacity to find seeds to grow new vegetables. Targeting The project targets women and children in the period called the 1000 days, extending from conception to the child’s second birthday. This period is critical because it represents a window of opportunity to provide children with the maximum chances for optimal development. However, because of the time needed to effect improvements in knowledge and practices and for households to see increased production of nutrient-dense foods, starting interventions before women become pregnant would be more likely to help them reach an improved health and nutrition status by the time of pregnancy. It is acknowledged that a section of the project targets adolescent girls, reaching them in school to promote best health and nutrition practices and the use of family planning services. The fact that this component is associated with research is a positive aspect as it builds evidence that will help strengthen the project in the future. A stronger connection between the adolescent component and the 16 various women and mothers groups could be beneficial to help young women improve their knowledge and practices and ultimately their nutritional status, conditions that will help their future child develop optimally. Similarly, continuing support until children reach five years of age would also address the vulnerability of these children to diseases and undernutrition, which decreases at 24 months but does not disappear. Beyond the core groups targeted, Sabal targets a variety of groups and individuals with its range of activities: mothers’ groups, farmer groups, health personnel, government officials, etc. Such wide targeting is likely to increase the impact of the project – but also makes it more complex to manage and monitor. Finally, targeting both women and men allows the project to address gender-based barriers and identify solutions that are gender-sensitive. It creates the opportunity to initiate a dialogue between men and women that ensures the benefits of the project can be sustained. Findings: effectiveness Effectiveness was assessed through discussions with all project actors, from the central to the most decentralized level. Data were only available to the evaluation team at a late stage, but could however be discussed with Sabal teams at central and district levels during the fieldwork. First, implementation of activities was slowed by a number of issues, many of them independent of the Sabal team. Specific to the health and nutrition purpose, the registration of thousands of households took several months, as did finalizing and validating training packages at the central level. It is advised that the Sabal team reflect on the delays associated with these preparatory steps to identify areas where greater efficiency could have been reached. As a result of these initial delays, targets were not met during FY 16, which forced adjusted targets for FY 17 All teams mentioned that since the middle of FY16, when field activities could finally start, they were able to increase the pace of service delivery to make up for initial delays, which was confirmed by all beneficiaries interviewed. Sabal teams in the visited districts consistently reported their expectation to meet the targets for FY17, which would place the project back on track in terms of its overall objectives. This projection was supported in the MTE team’s discussions with all categories of project actors and beneficiaries. After an intensive training and implementation period for the second half of FY16 and first half of FY17, all Sabal district teams mentioned being busy with monitoring activities, i.e., entering data for the last two years in the new monitoring system. Along with implementing activities, district and VDC teams have started conducting technical support visits to monitor and provide on-site coaching to beneficiaries. All FCHVs interviewed had received health and nutrition training and had transferred the newly acquired knowledge to 1000-days mothers groups composed of PLW. Training focused on health and nutrition practices for PLW and children under two. All FCHVs interviewed had also received SBCC-produced booklets or posters to support their activities in the communities. A major SBCC initiative to promote behavioral change is a weekly FM radio show that promotes best practices on all topics covered by the Sabal project. Research on how best to reach adolescent girls with health and nutrition promotion activities had also been conducted (report expected October 2017). Finally, all VMFs interviewed had received a five-day training on improved farming and management of poultry, and had been provided 17 10 chickens and three cycles of seeds. All 1000-days mothers had received farming training, seeds, and poultry. In the WASH sector, transfer of knowledge and provision of equipment had also taken place according to all community members interviewed. Sanitation activities are underway, in accordance with the National Sanitation and Hygiene Plan. Sabal supports open-defecation free (ODF) VDCs through training, the provisioning of latrines and hand-washing stations, and supporting resources for the most vulnerable households. In 2016, four VDCs were certified ODF out of a target of 17, and 11 more VDCs have initiated ODF activities this year. Three of the Sabal districts were already declared ODF prior to start-up, and in these districts, Sabal promotes a post-ODF clean and healthy house strategy. The MTE team also noted evidence of support to the health sector, with some of the facilities visited having received training and family planning, SBCC and WASH equipment. Despite issues around supplies of ready-to-use therapeutic foods that are being addressed by Sabal, at the time of the MTE, the IMAM project was getting back on track with its targets and the team was concluding a comprehensive health system strengthening (HHS) exercise that would allow complete transfer of activities to local government by early 2018. Support was also provided to local governments in areas of training and coordination, following the Sabal work plan. However most health personnel in charge of local health posts reported recurrent shortages of drugs. It is understood that the drug supply chain is not part of the Sabal TOC; however, Sabal does support the effectiveness of the health facilities and under a more holistic approach could work with local counterparts to define constraints and solve the problem. The HSS exercise conducted around the IMAM component is a good example of such a comprehensive approach. Table 3: Purpose 2 – health and nutrition outputs (selected) Target FY16 Actual Actual vs planned FY16 Target FY17 Actual to Sept 2017 Actual vs planned to Sept 2017 Results: Health and nutrition activities (cross-cutting) FFP-56: Number of people trained in child health and nutrition through USG-supported programs 26,200 12,639 48% 80,635 86,946 107% Results: Nutrition activities Homestead food production SABAL-10: Number of people trained in homestead food production (a subset of FFP-56) 17,820 6,074 34% 25,320 24,830 98% Nutrition services FFP-57: Number of children under 5 reached by USG-supported nutrition programs (homestead gardens, MIYCN, and IMAM) 30,000 8,457 28% 131,296 63,925a 48%a Multi-sectoral nutrition planning SABAL-15: Number of people trained on multi￾sectoral nutrition planning 2,490 365 15% 2,545 2,203 87% Results: WASH activities SABAL-11: Number of people trained on WASH 1,850 2,194 119% 3,620 2,679 74% a/ Will be updated on OPMIS by first week of October 2017. 18 Issues of coordination were raised at all levels. Teams from Ministry of Health, consortium partners, and Sabal offices all reported that communication and coordination constitute the weak link in the project. Information does not channel efficiently, neither vertically nor horizontally. Such gaps prevent the consortium from making optimal use of the expertise of all partners and actors, most of whom feel like they are not really part of the project. For instance, no in-depth discussion was ever organized between all Sabal actors to address main issues and identify solutions: meetings were organized (but did not always take place) several times a year but just for a few hours. In August 2017, the health and nutrition teams from all districts planned to meet (for the first time) to discuss project achievements with partners and the central team. Such a workshop is a move in the right direction but arrives too late and should be conducted twice a year in the remaining years. Sabal teams at district level and in the field, as well as Sabal partners, identified financial and administrative management as significant barriers to project effectiveness. Technical teams appeared in many cases to be spending an unreasonable proportion of their time managing operations instead of providing technical support. Allowing PNGOs to manage their own budget can be a simple way to reduce operations workload of technical teams. Similarly, most partners and actors expressed complaints about the rigidity and inadequacy of the financial management of the project. They all called for greater flexibility to allow field activities to be conducted smoothly. Finally, although the project teams managed to deliver a large amount of work in a short time and show confidence they will catch up with project targets, some inefficiencies were identified in the structure of the Sabal project that may require rethinking. The project appears to be functioning in top-down fashion, with large teams at central level (six for health and nutrition purpose), then medium-sized teams at district level (four for health and nutrition), and the closer to the field, the smaller the Sabal team for health and nutrition. In most cases, the Sabal facilitator is the only Sabal full-time field staff at the community level and must support all purposes, covering nine wards and between 25 and 40 groups from all project purposes. Despite their title, facilitators appeared to be mainly tasked with controlling and monitoring the activities of groups in communities, aiming at attending all group meetings to fill attendance sheets and manage paperwork that will later be transmitted to Sabal district teams. 19 Findings: evidence of change Table 4: Purpose 2 – health and livelihood outcomes Color key: Little achievement Partial achievement Significant achievement Sub-purpose (SP) Immediate Outcome (IO) SP2.1 Increased adoption of project￾recommended household health, hygiene and nutrition behaviors and practices. IO2.1.1 Increased knowledge of nutrition, hygiene and health. IO2.1.2 Enabling and supportive nutrition environment created. IO2.1.3 Improved knowledge on sexual and reproductive health, nutrition and hygiene among adolescent boys and girls. IO2.1.4 Increased access to and availability of nutrient-dense and bio-fortified food. SP2.2 Household environment sanitation and water management practices improved IO2.2.1 Improved knowledge of recommended water, sanitation and hygiene (WASH) practices by households. IO2.2.2 Increased household access to sanitation, products and services. IO2.2.3 Increased availability of sanitation products in communities. SP2.3 Use of quality health and nutrition services by women and children is increased. IO2.3.1 Improved access to high quality/ community based services. IO2.3.2 Increased demand for health and nutrition services in communities. IO2.3.3 Supply of family planning commodities and services improved. SP2.4 Government mechanisms for combating malnutrition are effective. IO2.4.1 Lessons learned, considered and/ or adopted by MSNP stakeholders. IO2.4.2 MSNP structures at district and VDC level are established and strengthened. IO2.4.3 Improved incorporation of integrated nutrition and WASH activities in DDC and VDC planning and implementation. Increasing knowledge of target beneficiaries is a significant outcome of Purpose 2 (see Table 4). All 1000-days mothers interviewed showed a fair knowledge of adequate practices around feeding children under 2 years old, hygiene and health, which suggests that the knowledge transfer was effective. Similarly, health care providers expressed satisfaction around their gains in knowledge. Family planning activities could not be assessed effectively as among all health personnel and FCHVs interviewed, none had been part of the family planning kit component. They had however received knowledge and transferred it to other women. The cascade training system used for the health and nutrition purpose therefore seems to have been effective and loss of quality throughout the cascade chain was limited. One area where knowledge transfer showed limits was in the SBCC domain. In the visited VDCs, most interviewed mothers, VMFs and FCHVs never listened to the Sabal radio shows. Some of them knew of their existence, but mentioned that the inconvenient timing (8 a.m.) did not allow them to listen to it. They unanimously recommended the show be aired in the evening instead of during morning hours when most women are busy. Regarding cooking demonstrations, it was too early to measure effectiveness. Although all 1000-days mothers attended one or more sessions, none ever tried the recipes at home. It could be interesting to ask FCHVs to regularly investigate during group sessions the barriers that prevent mothers from trying the recipes. Mothers mentioned being willing to try them out, but expressed that 20 one demonstration may not be enough to make them confident enough to try the recipe later at home. As the project continues and more demonstrations are provided, it is expected that more mothers will start cooking the new recipes at home. All groups were asked about men’s participation and if gender was an issue when it came to women’s decision around health and nutrition, or if men would support women during farming or take care of children at home or when women would be busy. Mothers reported receiving a fair amount of support from their husbands, although it was not clear whether they really understood what a minimum level of support should be. Men’s participation in women’s groups was rare. In most cases, they were not invited to participate in 1000-days mothers or other women’s groups monthly sessions, and it looked like women had not been sensitized sufficiently on the importance of gender and the benefits of including men in their discussions. Most FCHVs and FMVs had been sensitized on the question, but for such a complex issue, a few hours sensitization may not be sufficient. In terms of social exclusion, all groups indicated that the project was inclusive enough not to leave any vulnerable member excluded. In terms of access to nutrient-dense food, all 1000-days mothers enrolled in the project benefited from a larger variety of vegetables than they usually do; okra, lettuce, parsley, broccoli, cauliflower, tomatoes, and pumpkin are some of the various vegetable seeds provided. Vegetables were of bigger size in most cases, and increased food availability at household level. In discussions with mothers, it was highlighted that fruits are only available in small quantity and in specific seasons. Increasing their availability might increase project impact, as fruits contain high quantities of micronutrients essential for child development. Increased food availability through chicken distribution was less successful, as many 1000- days mothers mentioned high mortality among the chicks they received. Some mothers lost all their chicks almost immediately after the distribution. One recurrent complaint was that the eggs would not hatch, but VMFs managed to sustain their flocks by breeding the project chickens with local ones. In a number of VDCs, there was a high mortality rate of chickens after the February 2017 distributions, and Sabal staff confirmed that transportation and handling stress were the cause. Further incidents of widespread poultry mortality have not been reported to date. Some mothers however mentioned being able to sell more eggs than before Sabal (in addition to increasing the availability of eggs in the household diet), that this has increased household income. Overall, the seeds and poultry component is considered to have positive effects. Sustainability proved to be a major issue identified by the MTE for both mothers already enrolled in the project and for the ones who became pregnant during the life of the project. After an initial enrolment, training, and distribution of seeds and poultry, 1000-days mothers are supposed to continue having access to improved seeds and should increase the number of their chickens at home. However, seed multiplication had not started in any of the VDCs visited, and poultry breeding activities had not been organized. Mothers were enrolled in FY16 and received the three rounds of seeds before mid-FY17. They were expecting to have used all seeds by end of FY 17, after which the only remaining benefit of the project would be the knowledge they acquired through training. Even if seed multiplication was to start immediately, a gap would exist between the moment they finish the seeds they currently have and the moment new ones are available. All mothers and farmers groups interviewed also complained that new 1000-days mothers did not receive any support. Since the initial enrolment period in early FY16, new women 21 became pregnant – and were therefore eligible – but were not enrolled, as no more seeds or chicken were available. VMFs had not yet received guidance on working with the FCHVs to organize distribution to new women, and only a few VMFs had been sensitized on the need to organize the distributions to new women, but they lacked capacity to do this. Purpose 2 also encompasses a WASH component that successfully equipped households and health posts with sanitation kits, and more communities have been declared ODF. However, the relatively small number of VDCs targeted limits the contribution of the WASH component toward the overall building of resilience. It was also noticed consistently when visiting communities that waste management remains a major issue in most areas and should be considered by local WASH committees. Conclusions – Purpose 2 The Sabal project design is meant to build resilience of vulnerable households. However, the fact that not all VDCs receive the whole package of interventions significantly weakens the project’s potential impact: only 65 VDCs received all health and nutrition components. The MTE team contends that if more households are reached with all services defined in Sabal, greater impact will be achieved on resilience. Implementation suffered from major delays but overall the project seems to be back on track and able to meet its targets. Nevertheless, some important issues exist regarding sustainability of EHFP activities, and targeting should be reviewed for the health and nutrition purpose. Purpose 2 recommendations 1) Multiplication of seeds and poultry should be urgently initiated and support given to VMF and FCHV to organize the support of new pregnant women. 2) Extend the WASH component to more VDCs, and consider promoting waste management activities in WASH communities with local WASH committees and governments. 3) Adjust targeting for the health and nutrition purpose to integrate women of reproductive age who have not yet had any child. This would allow better integration with family planning activities. Integrate households with children under five to further strengthen overall maternal and child health and nutrition status. 4) The district health systems should be strengthened beyond support to coordination only, for instance by integrating activities to strengthen the supply chain to avoid drug shortages. 5) FCHVs should be further sensitized on the importance of inviting men into women’s groups and stimulating dialogue between men and women on issues of health, nutrition, and agriculture. They should be provided more in-depth training so they can sensitize women more efficiently on the importance of men supporting them and on what they should be able to expect. Purpose 3: DRR/CCA Brief description of activities Sabal Purpose 3 states: “The ability of households and communities to mitigate, adapt to, and recover from shocks and stresses is strengthened.” This is broken into three sub-purposes: 3.1 improved adoption of climate and disaster resilience practices by the poor, vulnerable, socially excluded (PVSE) households, 3.2 improved community-based disaster risk management, and 3.3 improved capacity of the government in disaster risk reduction (DRR) 22 and climate change. The Sabal TOC outlines that Purpose 3 can be achieved by a range of activities that strengthen “absorptive, adaptive, and transformative capacities at household, community, and government levels.” In 2016, 33 activities were implemented under the three sub-purposes and geographically focused on four demonstration VDCs in each of the 11 Sabal districts, thus directly benefitting target communities in 44 VDCs. Despite slow start-up, activities have been in full swing from Year 3 onward. Interventions draw from the experience of CARE and other Sabal partners as well as other programs supported by USAID and other international agencies. Sabal’s action under Purpose 3 emphasizes building institutional capacity for DRR and climate change adaptation (CCA). Sabal has assisted in the development of integrated DRR and CCA plans at various levels: community disaster risk management plans, local disaster risk management plans (VDC level), and disaster preparedness and response plans (district level). Sabal has also facilitated two new types of institutions to implement these plans – community disaster risk management committees (CDMC) (community level) and local disaster management committees (LDMC) (VDC level). A large part of Sabal’s investment has gone into creating awareness and capacity building on a range of DRR/CCA themes, including early warning systems, disaster response, and school safety management plans. In the six original Sabal districts, specific activities included workshop and information￾sharing sessions for district-level stakeholders, review and analytical exercises to understand social dynamics and gather disaster risks and adaptation challenges, awareness raising and training events (including REFLECT sessions) for local community groups focusing on women, formation of CDMCs and LDMCs with the representation of both men and women, providing DRR support to schools, and establishing local weather monitoring stations (including training male and female students). Three of the five task forces under LDMCs – early warning, search and rescue, and first aid – have been trained by Sabal, and these include women members, too. In the expansion districts, Sabal has also conducted masonry training to local men and women, and safe housing design workshops for government engineers. In FY 2016, the Sabal DRR team oriented 15,500 individuals in earthquake-affected VDCs, and trained 101 engineers on safe building construction. Sabal has also spearheaded the development of a national-level integrated DRR/CCA planning framework for VDCs and districts. The plan is in the process of approval by the Ministry of Federal Affairs and Local Development (MOFALD). With Sabal support, this plan has been piloted in 24 demonstration VDCs, who are preparing integrated local disaster risk management and CCA plans. Sabal has trained 3,872 people on identifying and ameliorating climate change risks, which is six percent higher than the target for 2016. With regard to working with schools to prepare safety plans, Sabal achieved 67 percent of target and plans to cover the shortfall in 2017. Relevance to TOC We find that activities implemented under Purpose 3 broadly reflect the interventions identified in the theory of change. Sabal’s simultaneous actions at multiple levels (household, community, local government, district agencies, and the national government) are consistent with the TOC to catalyze transformational adaptation. Efforts to link CDMCs with VDCs and district-level agencies can create new possibilities for resilience building. Facilitating the integration of DRR and CCA at policy level and piloting the idea at the local level supports the strategy to build institutional capacity in DRR and CCA. 23 However, there are three key deficiencies in Sabal theory and action in relation to Purpose 3. First, the Purpose is guided by the assumption that access to information and knowledge (much of the work is confined to awareness raising and human capital building) will automatically lead to asset building needed to cope with risks and shocks in a transformational way. Of the 11 intermediate outcomes under Purpose 3, only one (3.1.3) goes beyond human capital and stipulates improving access to resources and technologies – and this has only a sub-intermediate outcome focusing on improving the ‘network’ to access resources. Second, in practice, the logical framework and TOC thinking have been adopted mechanistically to identify unique activities for each of the indicators of the sub-intermediate outcomes. This has not only created an extra burden on program resources when activities are implemented without real demand on the ground, but led to the fragmentation of the program into numerous and disjointed micro-activities, at the expense of more strategic interventions that could help tackle context-specific DRR and CCA problems in an integrated way. The integrative value of the TOC has been lost in practice when CDMCs and LDMCs wait for resources and technologies for action. Third, due to its human capital focus and fragmented micro-activities, Sabal is yet to demonstrate problem-solving cases or likely pathways of transformational resilience in relation to disaster risks, shocks, and adaptation challenges. Despite clearly identifying risks such as landslides and floods, the solution implemented to date is primarily institutional- and planning-focused. The program theory offers little guidance to design interventions that can trigger bottom-up, solution-based institutional learning while also stimulating resilience and enhancing asset building in the context of Nepal’s current political economy. Sabal believes that access to information on weather, climate, risk, and vulnerability will lead to behavior changes of vulnerable communities toward disaster- and climate-resilient practices. Information, institutions, and planning are important steps towards reducing vulnerability, but without capital and technological assistance, outcomes remain limited. MTE fieldwork reveals that communities are waiting for capital and problem-solving interventions, such as bioengineering inputs to control landslides in Madhavpur VDC in Okhaldhunga and flood control in Marin in Sindhuli. Targeting In general, Sabal targeting is happening as planned: the same sets of households and groups receive multiple services. For selecting Sabal VDCs within each district, the seven-step VDC selection process7 and the eight criteria8 used are reasonable. Targeting under Purpose 3 has benefitted from CARE’s important experience on participatory vulnerability assessment and targeting. For DRR/CCA work, only four of the Sabal VDCs are targeted in each district, and within these VDCs, only the four most vulnerable wards receive intensive DRR/CCA support services. These wards are identified using an Integrated Vulnerability Assessment (IVA) tool. Targeting in the expansion districts is also reasonable – for example, in Kavre, 25 VDCs were initially selected, and from the pool, the four most vulnerable VDCs were chosen. In each VDC, the four most vulnerable wards were targeted. Household data were collected and 7 The Sabal document, “Sabal VDC Selection Process,” states the following steps: 1) development of selection criteria; 2) data collection and compilation; 3) VDC ranking; 4) mapping; 5) VDC clustering; 6) consultation with district stakeholders; and 7) final selection by District Food and Nutrition Committee. 8 The eight criteria used are: 1) number of households; 2) number of PVSE households; 3) geographical remoteness (road access); 4) distance from market point; 5) VDC with high risk of natural disaster; 6) presence of other similar development program; 7) illiteracy; and 8) DASG VDC as per DAG mapping. 24 tallied against the list of beneficiaries under Livelihoods and Health and Nutrition Groups, and if overlapping, selected for the DRR and CCA component. For Purpose 3, a total of 22,400 households have been targeted. Most of those we visited in our sample were the poor, women, Dalit, and marginalized, thus adequately reflecting the Sabal criteria for PVSE households. The MTE team heard no reports from any stakeholder of dissatisfaction over the selection of target VDCs and communities. There is, however, a lack of adherence to the strategy of targeting the same households for all the services under the three purposes. At least one interviewed Sabal facilitator stated that not all households are receiving all the Sabal component services. Likewise, targeting relies upon the process of identifying households that are poor, vulnerable, and socially excluded. This largely works, but also misses an important dimension of targeting for DRR/CCA in the context of disaster risks such as floods and landslides that expose all households – not just the poor – in a specific locality (though the sensitivity to risks and adaptive capacity varies across households). Mitigating these risks is beyond the capacity of PVSE households, and the experience in the 2015 Nepal earthquakes showed that local elites often take important leadership roles in disaster response action. This means that Sabal targeting of households needs to be complemented by a group-based risk management approach. For example, a number of ephemeral streams create flash floods in Nipani, a project VDC of Sindhuli. To minimize the flood risks, beyond identifying and targeting households and providing training on search and rescue or providing gabion wire at the bottom of the river, it is important to stabilize the fragile watershed of the stream. Within the watershed and group-based risk management approach, Sabal can keep targeting PVSE households within the settlements for more focused and equitable resilience-building services. Such settlement and watershed scale activities can also create a platform for addressing gender and social inclusion aspects of DRR/CCA. Findings: effectiveness Table 5 summarizes the progress of Purpose 3 to date using selected indicators. It shows that most DDR/CCA activities are close to or have exceeded target. In addition, DRR interventions have been well received by local communities and district government agencies. Particularly noteworthy is Sabal’s direct working relationship with chief district officers (CDO) on disaster response. CDOs appreciate Sabal’s work but expect more equipment support to prepare personnel for rapid disaster response. Sabal has also successfully mobilized co-funding from VDCs to implement DRR/CCA plans it supported. However, Sabal has yet to prepare a strategy on how it should adjust its collaboration strategy with new local governments, which become active soon. Several of the community-level awareness raising workshops, most notably REFLECT sessions, have worked well. More specialized training such as search and rescue and first aid have been found useful by the local communities. Table 5: Purpose 3 – Selected DRR/CCA outputs (till Sept 2017) Target FY16 Actua l % of Target Target FY17 Actual as of 21 Sep FY 17 % of Target Disaster Risk Reduction FFP-31: Number of people trained in disaster preparedness as a result of USG assistance 3,089 5,011 162% 8,625 4,491 53% Sabal-22: Number of school safety plans are in place 30 20 67% 68 80 117% Sabal-23: Number of school curricula with DRR 30 20 67% 68 80 117% 25 and CCA Sabal-24: Number of individuals oriented on earthquake resistance and safer building construction 16,250 15,500 95% 49,500 23,728 48% Sabal-25: Number of GoN personnel trained on earthquake resistance and safer building construction technology 101 101 100% 0 N/A N/A Climate Change Adaptation Mission-1: Number of people receiving training in global climate change as a result of USG assistance 3,656 3,872 106% 6,624 8,443 127% Mission-2: Number of institutions with improved capacity to address climate change issues as a result of USG assistance 44 47 107% 98 98 100% Mission-3: Number of stakeholders with increased capacity to adapt the impacts of climate vulnerability and change as a result of USG assistance 44 44 100% N/A N/A N/A Mission-4: Number of climate vulnerability assessments conducted as a result of USG assistance 44 44 100% 191 204 107% Note: N/A means dropped this indicator from IPTT Despite an integrative approach taken to tackle DRR and CCA, Sabal work on CCA is less effective than its work on DRR. The Sabal team faces challenges in communicating adaptation options to communities for slow-onset risks related to climate change. Although people recognize risks related to drought and erratic rainfall patterns, a systematic approach to CCA is needed. For this to happen, CCA activities need to be better integrated with livelihoods (Purpose 1) and health and nutrition (Purpose 2). The problem is not so much about targeting but about developing a strategy for working through the multi-faceted problems that defy simple thematic categories. For example, community groups, especially women, in Singhadevi and Madhavpur VDCs of Okhaldhunga report water management as a major challenge related to livelihoods and health and sanitation, and the water system is likely to be impacted by climate change, too. Instead of different Sabal components undertaking discrete activities, an opportunity exists for Sabal to improve overall water management systems, which contribute simultaneously to livelihoods, health and nutrition, and CCA outcomes. This work could also help reduce the workload of women. The current tendency to design and implement micro-activities under each purpose, sub-purpose and immediate outcomes, undermines the potential of effective problem solving and building transformational resilience. Sabal’s overarching focus on training, planning, and awareness raising leaves an important gap in enhancing access to resources and technologies (or asset building), without which the reduction of disaster risks and adaptation to climate change remain extremely limited. A Sabal field technician remarked, “We have not been able to offer concrete support except words.” A PNGO coordinator appreciates the community demand for material support but he finds it difficult to respond due to the program’s restriction on providing communities any physical asset or financial support. Field staff said that even cash for work is not enough, as people also need material support, skilled labor wages and asset subsidies. REFLECT sessions have mobilized women to identify risks and opportunities, but lack of substantive support prevents them from taking actions. A REFLECT facilitator said, “I am facing a new challenge now. Participants complain that we are only talking about the problems and not moving further to act on them.” 26 Several activities could be strengthened. Agro-advisory services are not yet fully functional. District Preparedness and Response Plan (DPRP) could be improved by preparing geo-maps of hazards based on historic data and on hazard frequency or intensity of impact. Such maps could also be overlaid with settlements mapping, showing the locations of the poor, socially excluded households. While the current book version of DPRP is useful for organizations, a two-page flyer could be distributed widely among community members. This could be made less wordy and more pictorial for the benefit of the less educated or neo-literate, to achieve greater social inclusion impact. School-based interventions are weakly integrated into the school management system, as demonstrated by the fact that in two of the school-based weather monitoring stations we visited, the weather data recording register was not well maintained. Despite this weakness, schools are very receptive to the program, so there is an opportunity to improve. A more integrated risk management support package is needed beyond training and some assistance in developing safety plans. Sabal has also rightly recognized the opportunity of using upstream weather data for early warning in downstream areas. Staffing level and capacity in the field do not match the complexity of DRR/CCA problems, and there are risks to sustainability. PNGOs are not adequately engaged in managing the program, which creates sustainability risks beyond the project. “We have been driven by the project and there is no identity of our own in the program,” said one senior PNGO staff. Some even said that the district Sabal team is an unnecessary parallel structure overlapping the PNGO role. Despite significant flow of information from local to higher levels, limited district-level processing happens, which means that district-level learning through program implementation is limited. Field staff working conditions are not conducive to effective fieldwork: they have too large an area to cover on foot, and some of the field staff we spoke to also mentioned that they do not have enough local travel and accommodation support from Sabal. In every district, we met PNGO staff who started their jobs 1-3 months ago, after the previous ones left their jobs, suggesting high turnover. Sabal facilitators and DRR LRPs feel they have not received enough guidance, and want district technical teams to visit the field more frequently. The heavy workload of Sabal facilitators was mentioned not only facilitators themselves, but by many other staff. In the expansion districts such as Kavre, the focus has been on training community members and government officials on safe housing. We did not have enough evidence to establish the outcome of the training, but as in other districts, the need for building linkages for rehabilitation is important. Giving training on resilient housing is one step ahead in the process, but as the community members we spoke to in Kavre District mentioned, they are in need of material and financial support to rebuild their damaged houses. Findings: evidence of change The MTE finds evidence that Sabal’s DRR/CCA interventions are leading to important changes in terms of increased knowledge and capacity of key local actors, wider appreciation of the need to act proactively to mitigate shocks, and improved collaboration among stakeholders with a unique ability to contribute. The activities have benefitted both men and women of the poor households. Through trainings, planning and capacity building, men and women have been able to better understand the nature of risks, and to better assess and respond to disaster risks. In Okhaldhunga, we found that at least two individuals were saved from life-threatening disasters: one was saved from drowning in the river; the other was 27 nearly buried under a landslide at a road construction site. 9 In both cases, Sabal-trained people were able to rescue the persons in danger. The disaster management plans at community and VDC levels have provided an important process and instruments for collective action among community members, including the integration of men’s and women’s voices and knowledge in disaster management. There is also evidence of policy change, with two notable examples: Sabal’s contributions to the integrated DRR/CCA guidelines of MOFALD and to more recent DRR and CCA learning center guidelines. Table 6: Purpose 3 – DRR/CCA outcomes Color key: Little achievement Partial achievement Significant achievement Evidence of change in sample districts PVSE HH adoption of climate & disaster resilience practices improved (3.1) Households have improved access to weather and climate information (3.1.1) Slight improvement in access to weather and climate information. PVSE HH ability to assess risk and vulnerabilities improved (3.1.2) PVSE households affiliated with CDMCs have increased awareness and capacity to recognize risks and vulnerabilities. PVSE HHs have improved access to resources and technologies (3.1.3) Slight increase in access to resources and technology. Community based disaster risk management systems improved (3.2) Community disaster risk management capacity strengthened (3.2.4) Significant improvement in community disaster management capacity. Government capacity in DRR and climate change improved Government staff knowledge on DRR and CCA improved (3.3.1) Significant improvement. Government disaster preparedness, response, mitigation and adaptation plan improved (3.3.2) Significant improvement. Integrated DRR/CCA planning guidelines improved (3.3.3) Drafted, agreed with MOFALD and waiting for final approval. Institutional change on DRR is happening, but CCA-related changes are less obvious. Most of the community groups and district stakeholders in the three districts (CDOs, LDOs, and others) commented that Sabal has too-limited “hardware” support provisions for the communities. A district government officer mentioned that Sabal is doing too much training and not provisioning enough material and financial support to enhance community disaster management. One of the CDOs we met requested the Sabal district DRR officer for more equipment support in front of us, when the Sabal staff was assisting our interview with the CDO. Even the local beneficiary groups want less “talk” and more action-based support to tackle the problems of floods, landslides, and other challenges. One woman Sabal beneficiary confessed that she is increasingly less interested in Sabal meetings, as she did not get concrete assistance beyond the training. Even well-received community activities like REFLECT sessions have slowly become a platform to express frustrations, as these platforms raise expectations without a concurrent strategy to either provide asset-building inputs or link local groups with other service providers in the district. There is an opportunity to link REFLECT sessions outcomes with livelihoods and health-related activities under two other 9 The person saved from the construction site was reported by LDMC members at Madhavpur VDC of Okhaldhunga. The person saved from drowning in the Sunkoshi River was reported by a Sabal staff member in Okhaldhunga. 28 purposes targeting women. The sustainability of Sabal outcomes depends on how well it adjusts its relationship and collaboration with newly elected local governments. Since every new local government body has a female leader, there is an opportunity for catalyzing more gender-inclusive change. The new local government will particularly affect the adaptation side of the work under Purpose 3; the DRR side is less affected, as the lead government agency CDOs remain in the district even in the new governance structure. Two other important factors affecting sustainability are the quality of partnership with district-based PNGOs and the level of institutional capacity of CDMCs and LDMCs at the end of the program. At least one of the six PNGO coordinator we visited was a woman and at least one NGO was led by a female chair, indicating the emerging trend of gender sensitivity in local NGO management. Conclusions – Purpose 3 Despite the slow progress in the first two years and the addition of five expansion districts following the 2015 earthquakes, Sabal is progressing well on its DRR and CCA activities under Purpose 3. Particularly impressive is the work on building local institutions (with significant participation of women and Dalit members) and supporting DRR/CCA planning, together with an active engagement with key district and national stakeholders toward creating an enabling environment at the local level. Sabal’s work has also created significant awareness among communities and stakeholders about a more systematic approach to managing disaster risks and contributed to institutional capacity building for risk planning and management. While Sabal has rightly theorized CCA as a continuum of – rather than distinct from – disaster risk, the adaptation side of the work is weak, partly because of the limited integration of the components and the poor guidance of the program theory and design. Adaptation to climate change has to happen in relation to concrete livelihoods, natural resource management, and infrastructure systems. This should also be inclusive in terms of gender, caste and other measures of social difference. To catalyze transformative resilience, it is not enough to have plans to mitigate risks; one must also enhance incomes and livelihood opportunities considering likely shocks. Current adaptation and DRR plans should be integrated with such development plans. The evidence from the fieldwork shows that the components are not well integrated, and a tendency to design and implement micro-activities that are reportable against multiple outcomes and indicators. In each targeted community, identifiable disaster risks, adaptation challenges and livelihoods opportunities can be mapped out and then more integrative, problem solving interventions planned, implemented and demonstrated (such as integrated water management contributing to livelihoods, adaptation, and health and nutrition; and integrated landslide rehabilitation in Madhavpur contributing to DRR, adaptation, livelihoods). Sabal needs to strengthen component-wide learning and experience-sharing among staff and PNGOs on an annual basis. This could enhance Sabal staff learning on the principles and practical skills on DRR/CCA, and hence improve the quality of implementation in terms of social inclusion in the process as well as generate concrete, targeted outputs. The sustainability of Sabal outcomes depends on linkages and coordination with new locally￾elected governments and the endorsement/registration of Sabal DRR/CCCA groups (mainly CDMCs), strengthened institutional capacity of PNGOs to continue acting beyond Sabal, and effective and case-specific collaboration with other agencies who can offer complementary services (e.g., Soil Conservation Office, Water-Induced Disaster Mitigation Center, and 29 Chure Conservation Program). Purpose 3 recommendations 1) Strengthen CCA interventions by focusing on important natural resources systems. The latter include water management, agriculture and livestock, community forestry, and agroforestry. Although people recognize risks related to drought and erratic rainfall patterns, a systematic approach to CCA is needed. For this to happen, CCA activities need to be better integrated with livelihoods (Purpose 1) and health and nutrition (Purpose 2). Integrated water management solutions deliver outcomes on all three program purposes. Sabal should support CDMCs and LDMCs to coordinate with farmer and women’s groups to plan and implement these activities and improve the integration of livelihoods and DRR/CCA activities at the community/ VDC level. 2) Create 5-7 flagship problem-solving demonstration cases for tackling key DRR/CCA challenges in the program area across different ecological regions. In each targeted community, identifiable disaster risks, adaptation challenges, and livelihoods opportunities can be mapped out; then more integrative, problem-solving interventions can be planned, implemented and demonstrated (such as integrated water management in Singhadevi, Okhaldhunga, contributing to livelihoods, adaptation, and health and nutrition; integrated landslide rehabilitation in Madhavpur, contributing to DRR, adaptation, livelihoods; and flash flood control at Nipani VDC (Sindhuli). These can trigger bottom-up learning for communities, the Sabal program, stakeholders and policy makers. 3) Improve the implementation of various ongoing activities: • provide better equipment and guidance for community rainfall stations; • create effective school-based institutional arrangements for data collection, storage and dissemination; • improve the content of various plans – DPRPs, community disaster risk management, and local disaster risk management – by adding spatial mapping of hazards and producing reader-friendly fliers for wider circulation • enhance the institutional capacity of CDMCs and LDMCs by holding district-level annual planning and review workshops for key CDMC and LDMC leaders; • create an annual program-wide DRR/CCA learning and experience-sharing forum for Sabal and PNGO staff; • strengthen PNGO participation in planning and implementing Sabal. 4) To stimulate community asset building for DRR and CCA, develop more targeted, activity-based and site-specific collaboration with other non-Sabal government and non￾government stakeholders. Revise the Sabal TOC to recognize the need for providing basic start-up physical inputs to stimulate the creation of vital community assets vital to building resilience (e.g., bioengineering support, skilled labor wages, and Gabion wire). 5) Enhance technical expertise and skills of staff working at the field level. Increase the direct fieldwork time of district-based technical staff in the implementation of Sabal activities. 30 V. Gender and Social Inclusion Brief description of activities Sabal implements gender equality and social inclusion (GESI) as a crosscutting component across the three program purposes. It uses a two-pronged approach in the implementation of GESI activities by addressing the social exclusion deeply entrenched in Nepal’s socio￾economic fabric. Nepal is not only a multi-ethnic and multi-cultural country; it also has a caste system with marginalized ethnic groups such as Dalits and Janajatis. According to the TOC, activities to promote social inclusion in Sabal include targeting poor, vulnerable, socially excluded and marginalized groups; ensuring equitable and meaningful participation of different groups such as by age, sex, religion/ethnicity, race and social status, thereby promoting access to resources and services; and raising awareness on progressive government policies such as the Caste-Based Discrimination and Untouchability Act. The Sabal TOC further outlines specific gender equality and gender-transformative activities that contribute to the achievement of the three purpose outcomes. Under Purpose 1 (agriculture and livelihoods), GESI activities aim to increase gender-equitable decision￾making control of financial resources for men and women; Purpose 2 (health and nutrition) supports women’s empowerment activities, targeting women of reproductive age through mothers groups and promotion of household decision making; and Purpose 3 GESI activities focus on the need to conduct orientations on gender mainstreaming in DPRPs and ensure the development of gender-sensitive DRM plans. Relevance to TOC Sabal’s TOC adopted inclusion and gender-mainstreaming approaches intended to stimulate and enhance transformative capacity at the household and community levels. These approaches are reflected at all levels within all three purposes. KIIs with staff members demonstrated their understanding of GESI as being at the center of the Sabal program. Inclusion of poor and marginalized groups is facilitated by the program’s ability to target the poor and vulnerable, including different ethnic groups, through various strategies, such as participatory capacity and vulnerability assessments. To ensure effective inclusion of vulnerable groups in program activities, the Sabal M&E system, IPTT and OPMIS capture sex-disaggregated data, and track registration of beneficiaries by gender of household head and by ethnic group. Documents reviewed and interviews with Sabal staff confirm that Sabal conducted extensive assessments at the inception of the program and these helped to inform GESI interventions. The Gender Equity and Social Inclusion Analysis10 reveals that discrimination against women, caste and ethnic groups is prevalent in the program areas. The data, also confirmed through community FGDs, show that women, Dalit, and Janajati, among others, face barriers in accessing economic resources, including land. This challenge is exacerbated by their limited participation in community decisions, restricted mobility, and violence against women. The Sabal gender equality and social inclusion mainstreaming strategy and the GESI Common Understanding Paper11 were designed to operationalize GESI-specific activities in the TOC using the CARE gender equity and inclusion framework. Key GESI strategy issues include vulnerability targeting through community social mobilization; awareness raising on 10 Sabal. 2016. Gender Equity and Social Inclusion Analysis (GESI) Report. April 6. 11 Save the Children. 2016. SABAL Gender Equality and Social Inclusion Mainstreaming Strategy (2016-2019). August. 31 inclusion and exclusion issues; and addressing violence against women and engagement of men and boys. While the TOC has captured gender- and inclusion-sensitive and specific activities for each purpose, the Detailed Implementation Plan (DIP) does not show GESI-specific activities under each purpose area. Rather, GESI activities in the DIP are stand-alone activities captured under one section, which can create an implementation challenge for staff. For instance, in the FY17 DIP, Sabal planned to establish 11 gender-based violence (GBV) task forces at district level. While this is a very good initiative, there is an opportunity for the DIP to plan for GBV sessions targeting the different Sabal groups – farmer groups, mothers groups, and DRM – under each purpose area. Further, Sabal is implementing time diary sessions through REFLECT classes, and these GESI transformative sessions could also target the other Sabal groups under P1 and P2. A DIP is a key project plan that should demonstrate how GESI activities are integrated across the purposes. Since staff members implement activities through purposes, it is advisable to have GESI-specific activities for each purpose area in the DIP. Staff members then become responsible and accountable to implement GESI under their purpose area. While the TOC has GESI-sensitive and GESI-specific activities, the Sabal M&E framework is limited on GESI qualitative monitoring and reporting. The IPTT monitors inclusion through sex- and ethnically-disaggregated data and proxy indicators that measure access to livelihood activities by different groups. The program needs to consider strengthening qualitative monitoring for gender and inclusion by including GESI questions as part of routine monitoring processes. Guided by USAID ADS 205,12 the program should qualitatively monitor changes in the following gender domains of change: gender roles and responsibilities, control of assets, resources and income, decision-making, and violence against women. Targeting GESI being a crosscutting component, its activities are targeted at all Sabal groups: farmer groups, VMFs, VSL groups, FCHVs, mothers’ groups and DRM committees. Sabal program participants are drawn from poor, vulnerable and socially excluded groups. For instance, in Dhulikhel, the Janjati constitute 53 percent of the registered households; Dalit 8 percent; Newa 9 percent; Brahmin/Chhetri 28 percent and other 2 percent13. Selection of participants is done through a four-staged process that yields positive program results. This process involves 1) deliberate targeting of poor and socially excluded groups using selection criteria such as group mapping, poverty levels and geographic remoteness; 2) ensuring participation of the poor, vulnerable and marginalized in the different Sabal groups 3) ensuring that representatives of poor, socially excluded and marginalized groups take up leadership positions in Sabal groups 3) awareness raising on policies and laws related to social exclusion. KIIs with the Sabal team indicate that at least 60 percent of program participants are women, attributable to deliberate targeting aimed at increasing women’s access and benefits to resources. In all districts visited, more women than men participated in program activities; e.g., in Kavre District, women constituted 70 percent of farmer group leader training. 12 USAID ADS 205 on Integrating Gender Equality and Female Empowerment in USAID Programme Cycle. 13 SABAL. 2017. Dhulikhel Overall Progress Status Report: 14 July. 32 Sabal facilitators and LRPs have the responsibility of rolling out GESI trainings to program participants. In terms of design, a GESI topic should be included in all community-level livelihoods, health and nutrition, and DRR/CCA activities. In KIIs, frontline staff highlighted having limited skills and confidence to deliver GESI dialogues, especially those related to gender transformation such as violence against women, household decision making and control of resources and income. Staff also indicated having competing priorities at times, resulting in their not conducting GESI trainings. However, through the Sabal social mobilization strategy, frontline staff have been able to address dimensions of social exclusion through vulnerability targeting. Thus, women and vulnerable ethnic groups have increased access to nutrition and livelihood resources through their participation in program activities such as trainings in vegetable production, livestock management, and improved farming systems. The program has also devised inclusion-sensitive strategies like establishing community-based childcare centers, whose childcare services create time for mothers to participate in Sabal activities. Findings: effectiveness Based on interviews with the Sabal team at all levels and Sabal groups at community level, the program’s gender equality and social inclusion component has been effective in promoting mostly inclusion issues, which has enhanced access of the socially marginalized groups to resources and services. One of the community women from the Dalit group in Okhaldhunga indicated that through her work as a Sabal VMF, she has gained good agricultural skills to produce tomatoes, chilies, cucumbers, and beans; moreover, she could sell the produce at a local market thereby increasing her household income and access to food for her family. However, this level of empowerment for her did not translate to improving household relationships with her spouse. She indicated that her husband does not support her travelling to distant markets to sell her produce. This example is indicative of discussions with many community women, which show that the program needs to urgently step up GESI￾transformative activities. Below are some of the specific findings on effectiveness. Social Exclusion. Based on FGDs with participants, the program approach to address social exclusion is working well, as socially excluded, poor and vulnerable groups are being targeted, included those who are in hard to reach geographical areas. The program makes an effort to prioritize the poor and vulnerable groups making sure they access agricultural resources, finances, health services, and water. However, the program needs to explore and create strategies to address other forms of exclusion including illiteracy and language barriers. GESI training was conducted at different levels for Sabal and PNGO staff. Training components included engagement of men and boys, GESI mainstreaming, strategy development, and gender budgeting. Training modules in Nepali were produced and shared with staff members, which facilitates easy dissemination of information. However, based on some KII, some field staff had not received copies of the module with full training on GESI. Sabal needs to provide its staff members with communication and promotional materials on GESI such as posters, pamphlets, and illustrated/graphic materials for low-literacy populations, to facilitate participatory engagement during GESI dialogues. Staff members seem to be more comfortable implementing social-inclusion related activities. These include ensuring that more than 60 percent of program participants are women, provisioning childcare during trainings, redressing inequalities through participation of 33 different castes, classes, and ethnicities, and ensuring access to resources through program activities. These are all “GESI-accommodative measures,” which acknowledge existing norms and differences between men and women but do not actively seek to examine relations. One staff member indicated that, “GESI is easy to talk about, but difficult to apply, especially when developing plans.” This comment highlights that even though they have a checklist, staff members need more guidance and support on how to operationalize GESI activities. There is a need to develop purpose-wise action plans that articulate how each GESI activity contributes to the achievement of purpose outcomes. Community engagement and increased women’s empowerment. Men and women in community level FGDs provided positive feedback on how they are benefitting from the Sabal program. For women, it has built their knowledge and skills on different fronts related to agriculture production, health and nutrition, and disaster response. Sabal farmer groups and village savings and loans are a vehicle for women’s empowerment, with women taking up leadership positions in these groups. Women – including lead farmers and VMFs – testified they have benefitted from Sabal’s good agriculture practices trainings on composting, using improved seed, vegetable production and goat production. For example, one woman farmer in Kavre District completed a mini-tiller operation training organized by Sabal, and is now earning 450 rupees an hour for providing tilling services with her tiller. From the earnings made possible through Sabal trainings, she bought one more tiller subsidized by the District Agriculture Development Office. It is, however, important to note that these important women’s empowerment activities are GESI-accommodative measures, which are mainly achieved through targeting and inclusion initiatives: by having access to Sabal training, women can achieve access to increased income (accommodative). It is equally important to address issues of control and decision making over use of the income (transformative). Sabal realizes the need to take a step further and implement transformative activities geared toward addressing underlying social norms and practices that impede empowerment, and has made initiatives to implement activities of this nature. Examples include plans to conduct the time diary exercise that addresses “time poverty,” and discussions of women’s work burden in REFLECT classes. While such initiatives are a step in the right direction, GESI transformative work at the community level is weak. FGDs with farmer groups and mothers’ groups across the three districts showed limited engagement and knowledge around leadership skills for women leaders, violence against women, and control of income and resources. Sabal should systematically identify a set of GESI-transformative tools for each purpose area and roll them out across the Sabal groups. Institutionalization of GESI. Sabal has made efforts to institutionalize GESI and ensure it is mainstreamed across the program. To improve accountability, currently the program is working on including GESI roles and responsibilities in key staff’s job descriptions and terms of reference. Senior District Coordinators from the PNGOs have been appointed as gender focal persons with the role of providing GESI trainings to district staff and providing ongoing technical support and mentorship to frontline workers. They are also expected to ensure the establishment of district-level GESI working groups. From KIIs with district coordinators in the three districts visited, they indicated time constraints to deliver on this assignment since they have other competing program responsibilities. This has had negative repercussions on effective delivery of GESI work by Sabal facilitators and LRPs. One district coordinator had to delegate this responsibility to their DRR/CCA officer. While feedback from the district 34 team was that this arrangement was working well, this example illustrates that the program should identify people who have the time and commitment to do the work. Sabal should appoint gender focal persons who are dedicated and available to implement GESI. Collaboration with government structures. Sabal has an opportunity to strengthen its collaboration with the Ministry of Women, Children and Social Welfare. A KII with the Ministry in Okhaldhunga District highlighted that the Ministry received Sabal’s support to participate in international commemorative campaigns and events such as Sixteen Days of Activism Against Gender-Based Violence and International Women’s Day. The official indicated keen interest to partner with Sabal in implementing community programs. As part of its gender and women’s empowerment activities, the Ministry conducts awareness raising and trainings on early marriages, leadership, violence against women, and skills training for women. The Ministry has established a community protection group trained on violence against women. Of the three districts visited, Sabal’s collaboration with the Ministry is strong in Okhaldhunga District. However, there is need to engage the Ministry at national level, including in all Sabal districts, with the view to support program-wide GESI activities. Findings: evidence of change Change due to Sabal interventions is evident across the districts, mainly in terms of increased and equitable participation of poor and marginalised women in Sabal activities, which has resulted in increased access to resources. Examples include: • Women have benefitted from using Sabal-promoted agricultural technologies like corn shellers and post-harvest management, which have contributed in reducing the labor burden. • Women constitute about 60 percent of most Sabal groups, including CDMC, VSL and farmer groups, and occupy positions of leadership. 14 For instance, in Devitar VDC (Kavre District), the CDMC membership is five women and four men. • In Kavre District, a group of 407 women farmers formed a cooperative producing eggplant, chilies, bitter gourd and tomatoes. 15 Upon selling the produce, they have bought livestock, send their children to school, and buy food for the family. • In Nepani VDC (Sindhuli), three adolescent girls are members of a VSL and have been able to use part of the VSL money to pay for school fees. • FGD participants noted that there is remarkable improvement in women’s confidence to speak in public. • Sabal has included women in vocational training and enterprise development. In Sabal, at least 4016 percent of women have received training in carpentry, masonry, electrical work, and tailoring. Male involvement in Sabal. According to FGDs with farmer groups, women represent more than 60 percent of farmer group participants. Reasons for this given by informants are that husbands migrate in search of employment opportunities as far as Kathmandu, Malaysia and Saudi Arabia. Women also indicated that some men are at home but engaged in other activities. Women felt that in cases where men are around, they are not being effectively mobilized to participate in Sabal activities. They indicated the importance of mobilizing men to participate in Sabal to allow both spouses to benefit from the program. Women noted that they were overburdened with agriculture work, including household work, and would need 14 Sabal Progress Status Report, 2017. 15 Ibid. 16 Ibid. 35 support to access markets; hence, they require the support of their spouses. Engagement of men in Sabal is critical if the program is to achieve GESI- equitable program outcomes. Men should be targeted and mobilized, and the program can support the establishment of men’s groups, which can be scheduled at a time convenient for them to participate in Sabal activities, including receiving GESI trainings. Sabal would still need to devise mechanisms to reach out to migrated husbands upon their return so that they are knowledgeable and supportive of GESI behavior changes. DRR/CCA- led “REFLECT” sessions provide an opportunity to address gender and social inclusion, as they follow a participatory and empowerment-learning process at community level. A REFLECT class constitutes an average of 25 women participants from CDMCs, and discussions are mainly on DRR and CCA planning. KIIs with LRPs who facilitate REFLECT sessions indicated that GESI topics are discussed as part of REFLECT, and relate mainly to birth registration, alcoholism, polygamy and marriage rights. There is however opportunity to expand the scope of GESI topics for REFLECT to include those identified in the GESI strategy, including violence against women and women’s leadership skills in CDMCs. Men are not permanent members of REFLECT classes; they participate upon invitation by the REFLECT group. Sabal should rethink how to more effectively engage men for them to regularly attend REFLECT classes, such that men are motivated to contribute and support women and achieve GESI behavior changes. The man’s role is critical in changing the gender dynamic, as pointed out by one woman, a CDMC member from Kavre District, who indicated some of the challenges women face at home after receiving training in REFLECT classes: “It depends on the husband. If he is understanding, then he will be supportive and involved in changing.” Health and Nutrition. KIIs with health and nutrition officers, including FCHVs, show that Sabal is working to improve household decision-making as it relates to health and nutrition. Through mothers groups, the health and nutrition program is building women’s agency on family planning, maternal and child health, backyard poultry, and child feeding. The program targets not only 1000-days mothers but other household members, mainly husbands and mothers-in-law. In promoting male engagement, men should be invited to participate in mothers group meetings and encouraged to do homestead gardening as a way of helping pregnant and lactating women. Some FGDs with mothers groups in the three districts highlighted the need for Sabal to invite men to mothers group meetings; currently very few men are engaged or aware of this activity. Further, the Sabal nutrition team highlighted that as part of sharing health and nutrition messages, FCHVs use a counselling card that promotes equitable division of labor between men and women, specifically encouraging men to assist mothers with household chores. Since men are currently not fully engaged, they are not receiving these important messages. The program should therefore step up efforts on male involvement in nutrition and based on a request from nutrition officers, the GESI team should avail tools on household decision-making and control of income, since these have a direct bearing on access to nutritious food at household level. Desired Changes for Women. In one VSL group, upon being asked about their hopes and aspirations and the changes they want to see, one widow highlighted that all she needed was enough food for her family. Women from mothers groups stated they would need to influence discussions with their husbands and have men and women consult each other. They also hope to influence decisions related to household expenditures, have their husbands’ help with household chores, and stop violence against women. Sabal is expected to further follow up on 36 these desired changes, analyze how these can promote or deter empowerment and program outcomes, and further plan on how these could be addressed by the program. Sabal therefore could conduct a process called GESI outcome mapping. This is an actor￾centered approach that focuses on behavior changes of an actor with whom a project works, that is, men, women, and community leaders. This community consultative process helps the program to develop gender qualitative behavior changes called progress markers, and these are both actionable and observable. Progress markers are developed per each gender domain of change including GBV, decision-making, gender division of labor, and control of resources and income. The benefit of outcome mapping is that it helps both staff and project participants to reflect and understand GESI changes, and understand and articulate the complexity of the GESI change process. Monitoring and supporting adoption of progress markers among beneficiaries helps the program better report on GESI outcomes, especially on progress related to gender relations and socio-cultural norms. For instance, if men report that they can assist their wives with household chores or are able to report cases of gender based violence, the progress marker process provides a qualitative enquiry of the change process. To make this process feasible, Sabal could select a team of staff to lead the process and develop draft progress markers for the program. These could be shared with staff at all levels, and frontline staff would validate the markers during their gender dialogue sessions. Conclusions – Gender and Social Inclusion The Sabal program has made remarkable progress around creating frameworks and GESI mainstreaming parameters and guidelines. The program is founded on a TOC that articulates the importance of GESI mainstreaming as a prerequisite to achieve program outcomes in livelihoods, health and nutrition and disaster risk reduction. The program has made strides to reach the very poor, marginalized and socially excluded groups. Based on some interactions with community members, the program needs to demonstrate GESI qualitative behavior changes at community level and their impacts on program outcomes. Supporting GESI transformation requires a strong grounding in how people experience change in relation to their own aspirations and relationships: as highlighted above, the program should establish the desired changes for men and women and based on this, develop mechanisms on how these changes could be achieved. The success of GESI interventions is dependent on Sabal management making strategic decisions to strengthen systems for GESI mainstreaming, ensuring accountability by all staff members at all levels. Gender and Social Inclusion recommendations 1) Develop clear and concrete messages on social inclusion that can be shared at all levels. Messages should address other factors leading to exclusion beyond caste, ethnicity, and gender discrimination, such as illiteracy and language barriers – hence the need to develop user-friendly materials for all groups. 2) Strengthen qualitative monitoring and reporting by conducting gender and social inclusion outcome mapping. This is a community based consultative process/model of developing a graduated set of specific, achievable progressive statements of behavior change for community men and women. This process helps the program to identify and devise mechanisms of supporting men’s and women’s aspirations and needs. 3) Strengthen technical expertise for Sabal frontline staff and GESI officers by conducting facilitation skills training. Support GESI cross-learning platforms at VDC level that allow sharing of GESI best practices 37 4) Identify and prioritize purpose-wise domains of change for GESI transformation. Organize participatory tool guides for each purpose area, clearly demonstrating the linkages of GESI activities to purpose outcomes. Based on the domains of change, develop specific activities for each purpose area linked to the TOC outcomes. 5) Operationalize the male engagement strategy by targeting and motivating men to participate in Sabal programs. Specifically, mobilize men to end violence against women, develop vegetable homestead gardens, and become role model fathers who support childcare responsibilities. In cases where men have out-migrated, SABAL could work with women to develop strategies to engage men upon their return to facilitate gender behavior change for both men and women. VI. Progress toward Resilience At this point in the Sabal project and taking into consideration the delays in start-up, desired resilience outcomes are yet to appear in any clear and consistent manner. As each of the individual purpose achievements suggest above, including progress toward GESI goals, project activities have generated significant success in disseminating a stock of messages to the beneficiary populations. Farmer groups see the positive results from using improved seeds, diversifying into more high-value vegetable and fruit crops that increase (scarce) land productivity, improving post-harvest storage, introducing small-scale irrigation, and improved animal care. The savings groups (many overlapping with the farmer groups) appreciate the value of savings, even if the quantities are small. The mothers groups have assimilated important messages on early childcare and nutrition-dense diets, and embraced the opportunity to diversify their household gardens. There is active dialogue and collaboration among government counterparts, Sabal staff, and community committees in reducing disaster risk from floods, landslides, and earthquakes (through earthquake-resistant building designs). In the pattern of project implementation, there are mothers groups, farmer groups, and savings groups in each ward of the 164 VDCs in the original six districts. By the end of 2017, CDMCs will be active in 100 of the original VDCs (and another 100 in the expanded district VDCs). On the other hand, MEAL estimates that only about half the registered households are participating in all of these activities. Moreover, within individual purpose activities, there is little overlap across interventions, such as WASH and health/nutrition interventions. This means that among approximately half of the original district beneficiaries, there are groups of registered households without WASH activities, households without nutrition/health activities, households without livelihoods participation, and households without disaster management activities. Possibly, Sabal assumes a strong community crossover impact of the messages provided to individual groups, so that information on improved seeds, for example, will spread beyond the farmer group to the wider community. It might also be determined that not every VDC needs WASH messages (especially the ones that are certified open defecation free). Nonetheless, the TOC indicates that enhanced resilience is an impact of all project components being available to all beneficiaries in one combination or another: “The three program purposes complement one another to increase the absorptive, adaptive, and transformative capacities of communities…” (italics ours). Such complementarity, from the perspective of the MTE team, would require all communities to have access to the full range of purposes; however, such is currently not the case. Interviews at central and district offices also suggest that such integration is not an explicit priority in the implementation process and Sabal facilitators are generally unaware of the importance of complementarity. The MTE 38 team believes that the question of enhanced integration and intervention layering should constitute a major focus of Sabal management as it moves into the next two years. Through its recurrent monitoring system, MEAL sought to register progress in the indicators used to measure absorptive, adaptive, and transformational capacities. There is, however, no specific indicator to capture the complementarity factor critical to building resilience. Some households will experience more stable incomes; some PLWs and CU5 will improve their nutritional and health status; and some households will improve their ability to mitigate and adapt to shocks and stresses. There is, however, the lack of a key indicator that measures resilience among those households with access to the full intervention set. (See Factors section, M&E). VII. Factors Affecting Results Delay in project implementation Sabal experienced significant delay in finalizing partnerships and agreements, preparing necessary documentation, setting up a functional M&E system, and rolling out activities. In such a complex project with so many dimensions, initial management challenges were to be expected; however, several factors, some unanticipated, united to further impede the desired project progress. The 2015 earthquake diverted attention toward the expansion districts, and this taxed the management capacity of the project as it was going through the initial stage of organization and preparation. Within the design of Sabal itself, the strategy of targeting the most vulnerable households using participatory methods proved time-consuming, as was the registration process. Similarly, the preparation of operational documents depended in part on participation of GoN counterpart agencies, which slowed rollout. In addition, a border embargo with India delayed the arrival of inputs needed for project activities. A further factor in delay lay with the multi-tiered decision-making structure in which SCUS, the FFP office in Washington, and the USAID mission managed key elements of the start-up. The multiple management voices and the need to move through layers of approvals were not conducive to a smooth take-off of project activities. Particularly with the M&E system, decisions made in Washington only served to slow the process of getting off the ground. At the time of the MTE, the first complete set of results reporting was available only for 2016; however, the backlog of unreported achievements is scheduled to be entered into the monitoring system by the time this report is finalized. The “fast-track” strategy adopted for 2017 has intensified activities and increased targets for many of the inputs that were slow in being implemented. The project has achieved full rhythm, and it should meet overall achievement goals. Partnerships The complexity of the project has required a network of partnerships with varying levels of function and interaction. Within the GoN, horizontal relationships at the national level were initially important to establish the institutional framework for interaction and a plan of action, but in day-to-day management are mostly informational. On the other hand, the district relationships with the government and its multiple line agencies are vital and dynamic, and represent a strength of project implementation. The MTE team met with local district officers, agriculture and livestock extension staff, health officials and staff, and DRR staff, and heard their working relationships with Sabal are highly effective, participatory, and very much a collaborative effort. As discussed below, the constitutional realignments currently 39 being implemented with countrywide elections will require a reconfiguration of government services structures, and Sabal will have to adjust its relationships accordingly. However, with regard to consortium partners, there is a feeling of dissatisfaction with the nature of the partner relationships. From the perspective of several partners, the terms of partnership were specific and perhaps too narrow in scope. In effect, consortium partners hired technical staff at the national and district levels who were seconded to the Sabal project. It was felt that this minimalist definition of partner collaboration underutilized the institutional capacity and expertise embedded in the partner organization. The nature of partner communication was limited to periodic meetings in which project updates were presented; some partners felt there was insufficient opportunity for more engaged communication, group problem-solving, and strategic decision-making. Another source of frustration mentioned by some partners was the inflexibility and lack of autonomy in the use of funds to support some project-related activities (e.g., organize a visit to the districts). A similar concern involves the partnership with PNGOs. The team met with five such partners in three districts, and, again, project funds have allowed them to staff technical support and frontline positions that carry the major load of field activities. It is true that there is variability in PNGO capacity, and some rather than others have more experience. Despite good working relationships with the Sabal district staff, some PNGOs feel they are simple purveyors of skilled labor with little say in the actual direction or implementation of the project. From their perspective, they are given tasks to perform but do not enjoy a fully effective and participatory partner role. In discussions with the MTE team, there was a sense that inadequate effort had been made to integrate local PNGO experience more effectively. Some also felt that there was not enough emphasis on building the institutional capacity of the PNGO through training (on resilience programming, for example). This latter point is relevant because the PNGOs will remain in place long after Sabal shutters are closed, and they represent an in situ opportunity to maximize the sustainability of the Sabal approach. Project management The MTE team views project management as the ability to create effective systems for project implementation that meet the bureaucratic chain of demands, coordinate among various actors, and address the needs of beneficiary populations. Furthermore, when information systems or other forms of feedback identify constraints to project implementation or weakness in project design, effective management displays the creativity and innovativeness to adjust. Finally, it is also critical that management articulate and maintain a consistent vision of project goals shared by all team members. Under these criteria, Sabal management—from Washington to Kathmandu—has a varied scorecard. The extended delay in the rollout of activities was partially due to over-ambitious targets but primarily to a lack of a clear and consistent decision-making process between the project office in Kathmandu, the SCUS office in Washington, FFP in Washington, and the USAID mission. Central management in country felt itself tied to too many masters, and roles and messages became unclear, system approvals slow in coming. In the USAID mission, there was a frustration with SCUS due to a perceived micro-management of the country office and the indecision regarding an M&E system. Nonetheless, despite these initial shortcomings at multiple levels, project management has demonstrated an ability to revise the design and adjust activities. The systems in place now appear to be up and running well and the project has gained an appropriate rhythm at a critical time in the project life. In the district offices, project management systems are in place and functioning fully and effectively. 40 Nevertheless, project management has not paid enough attention to partner communication and articulation, and the project lacks a more determined process of reflective participation. As a result, the shared vision has been weak, especially around the issue of integration and coordination. Delivery systems Technical support and frontline staff The consensus position of the MTE team is that there is a staffing imbalance in Sabal between technical support and the frontline staff. The success of a knowledge-transfer project such as this depends upon a strong field presence and regular communication with the beneficiary populations. Under the Sabal cascade approach, technical expertise is front￾loaded at central and district levels, not at field level. Part of this concern is the number of field staff relative to the heavy demands at the community level. To the beneficiaries, the face of Sabal in the community consists of the Sabal facilitator—one per VDC—and one or two local resource persons per VDC. In some VDCs, there is also a DRR/CCA local resource person who works with Community Disaster Management Committees. The VDC is a large territory divided into nine wards, each of which consists of one or more communities. Distance, terrain, and lack of transportation limit access between any two wards, particularly in the rainy season. Sabal representatives do not have a means of transportation (e.g., motorcycle), and it often requires a day of walking to reach a community destination. In each ward, the Sabal facilitator is responsible for two farmer groups, two mothers groups, and the savings groups. This workload is overburdensome, and it means that the Sabal facilitator (and LRP) can only cycle through the nine wards every six weeks or so. This level of contact is not sufficient to achieve ambitious goals of effective knowledge transfer, technological adoption, and behavioral change. Technical staff support are located in the district Sabal office and in the PNGO, and few technical specialists actually live in the VDC. Thus with some exceptions, the availability of technical expertise at the beneficiary level is sporadic. Consequently, Sabal facilitators organize a training event with a beneficiary group in livelihoods, health and nutrition, or DRR/CCA; an LRP or perhaps a PNGO technical staff person then provides the training; but there is little or no follow-up for several weeks. Training and technical capacity of staff A second concern is the capacity of frontline staff. The imbalance between technical and frontline staff is that the level of technical knowledge diminishes significantly from the central to the district to the community level. At the central office and in the consortium partner offices, there is a generous stock of technical expertise in the three purpose areas and in GESI. There are technical support staff at the district level (in both the PNGO and Sabal), but their responsibilities appear to be more directed toward the preparation of training materials, articulation with GoN technical counterparts and district- level resources, and training of the local resource persons. In addition, since the Sabal facilitator is in effect a community organizer (and not a technical specialist) and the LRP a local farmer, there is insufficient access to technical expertise at the community level. Sabal facilitators tend to be young females, many with a 10th grade education; LRPs are typically young, knowledgeable about the community, and have practical experience as farmers. In the view of the MTE team, these frontline workers, besides their excessive 41 workload, are undertrained and insufficiently prepared for the important responsibilities that they have. These frontline staff receive an initial job orientation of several days, then content￾specific trainings that last one or two days. Particularly for the LRPs, the technical trainings are typically short, and there does not appear to be much follow-up to verify the level of message assimilation and effective transmission to farmers. It is common in large food and nutrition security projects to suffer from a lack of investment at the frontline level, and so is the case of Sabal. Burdened with their multiple responsibilities, frontline staff have little understanding of the TOC, the vision of the project, and the importance of integration. As a result, they tend to be highly task-focused and concerned with meeting annual and monthly work plan goals. They are also poorly paid, especially the LRPs, who receive a modest fee for each training. The rate of turnover is alarmingly high at the frontline for reasons related to workload and pay, and most facilitators interviewed by the MTE team had only been on the job for several months. The low level of technical knowledge and short-lived experience of these staff are problematic because behavioral change occurs precisely at the level of households and communities: frontline staff should be able to maintain regular contact with beneficiaries and to easily access the technical knowledge necessary for successful outcomes. Cascade approach as model of technology transfer As discussed above, the cascade approach is most effective when the quality of the information and the integrity of the message are consistently transferred through the different nodal training points. It appears that in the case of health and nutrition, where the messages are clear and concrete, the cascade approach functions adequately. To promote technology change and behavioral change, however, the cascade approach appears less appropriate because of the brevity of the training sessions at the LRP and farmer group levels and because of the lack of adequate follow-up. It is unrealistic to assume that the technology transfer objectives can be achieved from a half-day training session provided by someone with limited technical expertise. Without a more regular technical presence in the community, the cascade approach is limited in its potential to effect change. M&E system Monitoring Evaluation Accountability and Learning (MEAL) is the overall monitoring and evaluation system that tracks project output progress and assesses project outcomes and impacts. Project goals are summarized in the IPTT, and Sabal collects information on 96 input, outcome, and impact indicators. At the beginning of the project, a baseline survey gathered information on 34 indicators and established their base values for the population in the project area. An endline survey will establish if the project met its targets for these outcome and impact indicators. In addition, an annual monitoring system monitors and reports on another 62 indicators. This system uses a variety of data sources to track these indicators, including an annual monitoring survey (AMS) of nine key indicators considered key to project progress. The AMS consults a sample of 900 beneficiary households to estimate the impacts of the project on this set of indicators; the first was conducted in 2016 and the 2017 results are expected in September. Furthermore, a recurrent monitoring system (RMS) focuses on a set of variables that define the absorptive, adaptive, and transformational capacities which contribute to a resilient household or community. The system has a mixed quantitative/qualitative methodology, and the initial baseline values for the RMS were reported in May 2017 based on a survey of 578 households (300 control, 278 treatment). The RMS activity, however, has now been discontinued at the request of FFP/USAID. 42 The principal MEAL data collection and analysis tool is OPMIS, management information software that monitors annual project achievements against targets laid out in the DIP. In this system, frontline staff (Sabal facilitators and PNGO technical specialists) record all field activities in situ. A team of MEAL district staff (Data Management and Documentation Officers) then enters the beneficiary information into the OPMIS system at district level. The system (at the central office) then compares these output data against DIP targets and combines different outputs to measure progress with regard to selected indicators in the IPTT. To date the most recent ARR is 2016; the 2017 report is under preparation. Now that the OPMIS system is functional (after the aforementioned delays), data entry is occurring at a rapid pace, and the documentation of project achievements is expected to be current by September. The field level effort for this highly time-consuming. The MTE team feels the project is over-reported, and it is not clear how some of the indicators (e.g., gross margin estimates) actually inform project success either at the outcome or impact level. On the other hand, other indicators of direct project interest, such as the level of intervention integration (e.g., percentage of families receiving the full complement of project interventions across purposes) are not yet reported. Furthermore, although annual reviews and reflection sessions are designed to gather input from all levels of staff, a more regular flow of qualitative feedback on project outcomes and evidence of change from the beneficiary communities would provide relevant input for project management. Social and behavioral change communication (SBCC) Any large development project seeks to assure that intervention outputs become project outcomes leading to positive impacts on the well-being of the targeted population. The SBCC is a cross-cutting component of Sabal designed to reinforce the widespread dissemination of Sabal messages in the effort to effect behavioral change. There are three major thrusts to the SBCC theme—mass media, community mobilization, and interpersonal communication. At the time of the MTE field visit, Sabal messages were broadly transmitted via FM radio on caller participation programs, and “edutainment” fairs (e.g., street theater) were being initiated. An activity that focused on training journalists regarding Sabal and its messaging was also planned. The MTE team observed printed materials promoting Sabal messages particularly in health and nutrition. The interpersonal communication component of SBCC applies a peer education model that intends a multiplicative impact of message distribution through youth leaders. The theme also focuses on building life skills and leadership capacity among youth and marginalized and indigenous communities. This is an important element of Sabal, but SBCC has not achieved its potential. There are SBCC activities listed in the 2017 DIP, but no results are mentioned in the 2016 ARR. The MTE team feels that this theme has been under-utilized and has under-achieved. It notes that recently, a senior level manager was hired to assume the leadership of this group and it is expected that the SBCC activities will intensify and increase its impact. Environmental compliance Sabal interventions must comply with USAID environmental regulations to assure that project activities have no negative impact upon the environment. To address this compliance requirement, Sabal has created district-level environmental working groups whose task is to monitor purpose-level activities that could leave an environmental footprint, such as pesticide use, latrine and tubewell construction, CFW activities, and livestock activities. Regular reporting is intended to identify and environmental or health problems that then are addressed 43 at the central level. The MTE team did not identify any compliance issues in its field visits and assumes this system is adequately designed to meet its purpose. VIII. Overall Conclusions The Sabal project has demonstrated to the satisfaction of the MTE team that its overall design and TOC define an appropriate pathway to achieving project goals. The integrated multi￾purpose approach inherent in the project TOC, although as yet not fully realized, holds promise for enhancing local-level resilience and improving the well-being of the most vulnerable populations in the target area. Despite a halting and uneven start-up, project activities are in full force, and the delivery of project interventions has achieved the necessary implementation pace. The MEAL system has also caught up and is providing appropriate feedback on project activities and indicators. On the other hand, MTE team finds a significant weakness in the delivery system process that impedes the critical transition from access to new knowledge to behavioral change. The key channels through which behavioral change must pass are the community groups— farmers groups, mothers groups, and CDMCs. To insure effectiveness and sustainability of Sabal, these local institutions must be strengthened significantly. This is essentially a project based on knowledge/technology transfer under the understanding that access to technical knowledge results in adoption of practices that lead to measurable change. This assumption is flawed in Sabal for two reasons. First, the project has invested heavily in the “supply” side of knowledge: it has marshalled multiple levels of technical expertise both in country and externally, and developed a top-down, cascade system of knowledge transfer to achieve desired scale. Despite, however, baseline studies and assessments, the “demand” side has not been adequately developed, and the community groups are for the most part mere recipients of such knowledge without much say in what trainings would be preferred and in what order. There is little actual ownership of the process by these community groups. The second component of this flawed assumption is that knowledge is the principal constraint to change. The visits to beneficiary communities made it exceedingly clear that this is not the case. In such a vulnerable population, capital constraints are as binding as knowledge constraints. People can attend a training on tunnel hothouse vegetable production or on micro-irrigation technologies, but they do not have the financial capacity to invest in the necessary equipment. The provisioning of such assets to a lead farmer (to achieve a demonstration effect) has limited impact upon rates of adoption precisely because of the capital constraint. Similarly on the demand side, the project presence is simply not adequate in the communities. The Sabal operations manual states that monthly meetings with community groups are the desired routine; however, there are not enough field personnel to achieve this frequency of contact. The Sabal facilitators and LRPs are overworked and cannot possibly spend the needed amount of time with each farmers group given the constraints of geography and infrastructure. It is also apparent that the frontline staff are not adequately trained to develop the institutional maturity of these groups and cannot provide sustained technical assistance. Their tasks are predominantly organizational—scheduling trainings in the action plan and making sure that everyone shows up—when more substantial cultivation of group empowerment is called for. The ultimate success of the Sabal project will be judged by the sustained development of community institutions, particularly farmer groups, in the post-project reality of technical assistance delivery under an altered political and administrative structure. The MTE team appreciates the value of the other sets of project interventions, especially those that build the 44 capacity of government counterparts and enable a sustainable flow of enhanced public services in agriculture, livestock, health, and local disaster management. Those interventions that diversify household livelihoods (e.g., vocational training, youth business start-ups, veterinary services entrepreneurs) and support increased market linkages and more professional farming are equally important, if somewhat limited in scope. The safety net interventions are both traditional (cash for work) and more innovative (crop and livestock insurance, rural health insurance), and the sustained impacts will reveal themselves more into the future. The key focus of the project, however, should be on the development of community institutions (including the VSLAs) with the ability to plan and negotiate their own futures. To further address the importance of community strengthening, the MTE team suggests stronger linkages among the various local groups mobilized by Sabal—farmer groups, mothers groups, and local disaster management committees. At the current time, they operate in an isolated fashion, each attended by a different field staff person, each oblivious to the activities of the others. A more desirable scenario would imply regular and systematic communication among these groups and the staff that support them. There are important issues in coordination among consortium partners and with PNGOs that should be addressed at the midterm. The maximum use of complementary pools of expertise, and productive, creative exchange of ideas among the consortium partners, appear to have fallen victim to management bureaucracy. Partner interactions, even within Sabal, are mostly limited to contractual obligations, developing work plans, and measuring achievements, and there is inadequate space for collaborative problem solving, reflection, and common visioning. The lack of a more productive communication process thus affects the cross￾purpose integration upon which the project approach is built. With regard to the PNGOs, the sustainability of Sabal initiatives will eventually rest with local technical assistance groups, which implies the need to assure strong PNGO capacity and an expanded PNGO role in strategic planning within the project. The importance of the new constitution configuration in Nepal is both a challenge and an opportunity for Sabal over the next two years. The restructuring of politico-administrative units into rural municipalities with an elected leadership will eliminate the geographical units that Sabal targets. Government services, many of them counterparts of Sabal, will similarly be reconfigured in terms of coverage and institutional jurisdictions. Under the new system, rural municipalities control their budgets and provide public services to their citizens; how the current civil service will be reorganized is not yet defined. The challenge for Sabal will be to understand how its coverage areas are adjusted (if at all) and how counterpart relationships will be defined. As an opportunity, it is widely thought that the elected leaders of the rural municipalities will work closely with established community groups, such as those being supported by Sabal. It is likely that new system will require strong support from the NGO sector as it is defined and rolled out. In reviewing the many activities of Sabal, the MTE team recognizes the tremendous challenge of managing such a complex project, and it considers Sabal to be on track. The project does require, however, some significant changes in its approach and its coordination in order to establish a sustainable dynamic of local-level change that will continue to build community and household resilience and leave behind viable local institutions. 45 IX. Project-wide recommendations The following recommendations are project-wide and complement the specific purpose recommendations listed above. It is important to note that several recommendations are interrelated. Community groups Provide innovative approaches that strengthen community groups. 1. To achieve sustainable behavioral change, the project must seek to strengthen local community groups, particularly the 5,000 or more farmer groups. The explicit goal should be to build the capacity of these groups as community-level institutions that ultimately enjoy formal status. To further address the importance of community strengthening, the MTE team suggests stronger linkages among the various local groups mobilized by Sabal—farmer groups, mothers groups, and local disaster management committees. They currently operate in an isolated fashion, each attended by a different field staff person, each oblivious to the activities of the others. A more desirable scenario would imply regular and systematic communication among these groups and the staff that support them. 2. To move toward that goal, the team recommends adopting a site-specific, integrated group planning approach in which the individual Sabal groups, in consultation with frontline staff, decide what priorities and course of action they wish to pursue over a project year. In such an approach, instead of presenting beneficiaries with a set of pre-arranged trainings, the frontline staff work with farmer groups to decide what problems they wish to solve (or begin solving) over the course of a year. Each farm group sets up its plan for change with specific goals and concrete actions. Frontline staff compile these plans, and district-level technical staff develop the annual Sabal work plan as a response to the priorities expressed by the farmer groups. 3. Under this problem-solving approach, the role of the Sabal facilitator and the LRP entails more frequent monitoring of farmer group progress and a greater focus on problem-solving technical assistance. To assure adequate technical support to the farmer groups, the number of Sabal facilitators and LRPs should be doubled in each focus VDC. Furthermore, the need for greater technical assistance implies that Sabal and PNGO technical staff increase their field interaction with the farmer groups. 4. The capacity building of Sabal facilitators and LRPs should be expanded to include facilitation skills focused on farmer group support and technology change. 5. The individual farmer group plans will require some start-up assets to overcome cash constraints. It is recommended that a combination packages of technology transfer and asset support tailored to farmer group plans be tested in a sample of VDCs. 6. Include trainings that deliver organizational and decision-making skills to the farmer groups. 46 Management Improve management coordination. The MTE team acknowledges that this set of recommendations will require a significant reallocation of project resources as more frontline staff are hired, their skill sets expanded, and asset packages added in some of the VDCs. The potential impact on the positive outcomes of the entire project merits this shifting of resources, if only on a limited basis. 7. Over the next two years, reinforce the integration of purpose-based interventions in a more explicit, vision-led manner at all levels of management. Most of the integration thinking and reflection has occurred at the central level, while at the district and frontline levels, the integration theme is relegated to the shadows as staff focus on concrete input targets. Integration across purposes and cross-cutting themes requires creative problem-solving, readjustment of intervention packages, and a project-wide appreciation of how important integration is to the project resilience goal. 8. It is recommended that project management define strategies to increase ownership of the project across the consortium partnership. Now partner meetings are quarterly and are mostly perfunctory communications on progress. The MTE team recommends a series of reflection meetings in the fourth year in an effort to expand partner contributions to strategic planning and respective roles, particularly in light of the institutional strengthening approach recommended above. 9. A similar advance toward partner “restart” is needed for the PNGOs as part of a broader exit strategy that maximizes the sustainability of project interventions and impacts. This restart can be initiated with more frank and open discussions of capacity-building needs at the PNGO institutional level and the staff level; it can be further enhanced by more frequent reflection sessions (now once a year) in which frontline staff are given a voice. 10. It is recommended that more opportunities for sharing of lessons learned be generated across districts and across purposes as well as between central and district levels. These exchanges will help disseminate positive outcomes and impacts and enhance the culture of integration that currently is underdeveloped. 11. Under the approach recommended above, it is recommended that the SBCC unit be recruited to develop messages directly for the farmers groups, disseminate their community plans, and give the groups an individualized identity as a means of promoting effective institutionalization of their collective actions. M&E Make M&E more responsive to project needs and less determined by external demands. 12. The MTE team realizes that USAID FFP and FTF require that sets of indicators be tracked globally. Many of the 96 required project indicators, however, are of little use as feedback to the progress of the project. Moreover, many are highly time￾consuming. It is thus recommended that Sabal management, at this midterm opportunity, review the tracking table with SCUS project management to reduce the overall number of indicators and cull out those that do little to inform project activities and progress. 47 13. At the same time, should Sabal agree to accept recommendations 1-5, it is recommended that qualitative monitoring be extended to farmer groups to mark the progress of their individual problem-solving plans, according to agreed-upon milestones. This indicator would focus on behavioral change and should involve the participation of the SBCC team. 14. It is also recommended that with the analytical capacity of OPMIS, MEAL begin to track the level of intervention integration and the beneficiary subset that participates in cross-purpose activities. Relationships with new administration Pursue articulation with the newly elected bodies in the project area. 15. It is recommended that Sabal management, in coordination with the country director, initiate a dialogue with locally elected leaders at the rural municipality and (new) ward level as a proactive strategy to adapt Sabal interventions to the reorganized system. In these discussions, the role of farmer groups as institutional points of entry for government programs and assistance should be prominent. Disaster management committees and plans will also be impacted by the new leadership and the structure of local government, which requires that the Sabal team discuss adjustment strategies with the local governments. Project extension Extend support for Sabal for an additional year. 16. The MTE team recommends a one-year extension of the Sabal project. This recommendation is based on three factors: the initial delay in project rollout, the current momentum that the project implementation schedule enjoys, and, most importantly, the opportunity to assist the government of Nepal during its critical transition period to elected local government responsible for providing public services to its citizenry. Sabal is well positioned with strong government counterpart relationships to contribute to this transition period. In sum, the benefits generated by Sabal are now in ascension, and the full sustainable potential of the project could be realized with another year of activity. 48 Annex A: Map of Original + Five Expanded Districts Implementation Period: Oct 2014 to Oct 2019 Districts: 1) Makawanpur 2) Okhaldhunga 3) Ramechhap 4) Khotang 5) Udayapur 6) Sindhuli 7) Dolakha 8) Kavrepalanchowk 9) Rasuwa 10) Sindhupalchowk 11) Nuwakot Livelihood VDCs: 64 in 11 districts H&N VDCs: 326 in 6 districts DRR/CCA VDCs: 224 in 11 districts 49 Annex B: Interview Guides Purpose 1: Livelihoods Agricultural production and diversification using improved NRM practices • What are the most critical constraints faced by FG member? • Do farmers feel that they have been addressed by the SABAL project? • What main agriculture and natural resources constraints were not addressed by the SABAL project? • What are the main technical areas addressed by SABAL e.g., increasing volume of production, increasing volume of sales, improving the quality of produce, building business expertise, etc.? • Have there been any improved agricultural technologies promoted by SABAL to address those constraints or that have considered those constraints (e.g., NRM, climate resilient technologies, seeds, etc.)? Have they been widely adopted by SABAL participants? What technology has worked really well, improved their income directly and should be continued for target HH? What technologies have not improved incomes and should be discontinued? What technologies require adaptation to improve adoption and outcome? • Will farmers be able to continue project promoted technologies after their participation with SABAL ends? • If not what could improve their practice adoption and technology uptake? • What agro-input service providers is your FG/SME doing business with? How reliable are they? Financial Services and Safety Nets • What are the most critical financial constraints faced by FG/members? • Do farmers feel that they have been addressed by the SABAL project? • What financial service constraints are not being addressed by the Sabal project? • Has the financial services training received been useful? How so? Where do you go when you have questions? • What other financial services training would be helpful to you / your group? • Are you currently benefiting from any GON safety net programs [including SABAL CFW]? Where did you learn about these programs? Which ones are most useful to you? Why? Have you received training about the GON safety net programs? • Are you currently participating in the GON Micro-Health Insurance program? Where did you learn about this program? Can you describe the MHI program and the benefits it provides/will provide for you and the family? • Has the SABAL program introduced you to any other insurance programs (e.g., crop or livestock)? Repeat MHI questions Vocational Skills for Youth and Women (Nonagricultural opportunities) • What training have you provided/received that is helping to improve your income? 50 • Has there been an increase in non-agricultural opportunities in the community over the past few years that did not come from the SABAL project? • Has SABAL promoted any nonagricultural employment activities [including CFW)? Has there been a demand for and uptake of those activities? • Have the promoted nonagricultural employment activities [including CFW] been appropriate for the geographic area and vulnerability of households (both youth and employers)? • What other support might improve the employability of SABAL poor, vulnerable and socially excluded youth and women? Entrepreneurship, Market Access and Linkages • What training have you provided/received that is helping to improve your SME? • Has your SME received training in Start and Improve Your Business (SIYB)? If yes, how have you used SIYB training? • What other needs does your SME have that is not currently being met by the SABAL program? • Is there a market for the products or services produced by SABAL SMEs? Is it easily accessible ? • Did SABAL support improving market access? In what way? How successful have these activities been (in terms of increasing income/profits)? How likely are they to remain after the end of SABAL? What suggestions would you give to improve it further? • What forms of market linkages were established? • Have the Farmer Groups established been able to develop relationships with the private sector (producer, buyers, suppliers, retailers etc.)? What kind of relationship? • How can the FG and their relationships with market actors be made more sustainable? Gender Sensitive Agriculture • Have Sabal project interventions helped to promote women’s social and economic empowerment? Which activities have had the most success? Why? • Are women taking on more leadership roles [in their community] as a result of Sabal interventions? What are some examples of roles women are currently undertaking? • What more can be done to facilitate changing behaviors and improving equity among men/women in SABAL? Livelihoods risk mitigation - Institutions and households prepared to respond effectively to shocks • Has this community been affected by any rapid on-set natural disaster in the last 7 years? • If so, what was the impact of this disaster on PVSE households? • Has anything been done by the government or any other institution since this disaster to reduce the impact of future disasters? If so, could you describe? 51 • Is the FG/community affected by recurring disasters? If so, what are the major recurring disasters that community members face? • Are informants aware whether SABAL has implemented any project activities to improve disaster preparedness and risk reduction in the community? If so, what are they? • Have any of these activities improved knowledge of disaster preparedness and response? If so, in what way (who has become more knowledgeable; what skills have been learned; what are participants doing with this knowledge and skill)? • Is there any evidence to indicate increased HH/community capacity to cope with shocks as a result of SABAL activities? • Has the community adopted any specific measures that further reduce their exposure to on-farm risks? • How are plans, training, activities, being undertaken under SABAL in support of disaster preparedness and risk reduction, linked to any government activities in Disaster Management? 52 Purpose 2: Health and Nutrition Domain Topic Questions Review of progress on implementation of the SABAL program? Process for designing the project - Who was involved, at what level, where local partners involved? Community participation? How have activities been defined? Was formative research considered? - What were the criteria for selecting partners? For selecting trainers and trainees? For selecting the beneficiaries? Process for implementing the project - Compare work plan with real timing of various activities - Identify roles and practices around monitoring of implementation - Identify variations between districts and partners Identification of problems and challenges - How did M&E system flow between field and national? Where the roles defined? - How where challenges identified, communicated and addressed? What are the strengths and weaknesses of the project? Project design and workplan - Is the project design relevant to achieving the outcomes indicators? - How feasible is the workplan? - Are all actors aware of the whole project, what are their perceptions on the workplan (feasible? clear?) - Has the workplan been adjusted and why and how? Management of the project - Who is in charge? Are roles well defined? Is management participative? - How does communication flow between central management and field actors? - What are perceptions of various actors at various levels around quality and inclusiveness of management and leadership? 53 Personnel - What type of training have each type of personnel received? What type of training have beneficiaries received? What method employed - Are trainers in the project trained on adult learning? - What are the perceptions around quality of training and transfer into practice both internal and for beneficiaries? - Are numbers of personnel deemed sufficient to conduct the activities? - Are personnel satisfied with their role? What would they suggest to increase their satisfaction? Do they have enough support and resources? - What mechanisms are in place for support and supervision of personnel? Differences per districts and partners? Community - Were communities engaged in the project design (reality and perceptions)? - Do communities feel that the project answers their needs? - What would communities suggest to improve participation in design and implementation? - What are the community perceptions around the design of the project: does it respect community culture, structures, are the outputs relevant and acceptable to the communities? Communication - What is the communication system between actors from all levels? - Are communication requirements respected? - What is the communication out of the project? - What is the communication with communities and beneficiaries? - What are the topics shared? What are the perceptions of all actors and beneficiaries around communication? Are beneficiaries and actors aware of other components of the project? - Do all actors have knowledge of project objectives? Of the vision of the project 54 Changes brought by the project in people’s lives? - How do intermediary beneficiaries perceive the changes brought by the project: knowledge, technical competencies? - Are intermediary beneficiaries able to implement the activities they are supposed to? - Are beneficiaries being reached by the project? How do they perceive the activities being brought to them? Do they see a change in their knowledge or capacity? Are they able to apply the new knowledge they received? - How do intermediary and end beneficiaries perceive the connections between the various components of the project in their community? Do they see a link between improvement of WASH, EHFP and SBCC and do they understand how they are linked? Is the project addressing gender and social inclusion issues? - To be addressed by gender specialist? What is the level of sustainability of the project? - What are the mechanisms in place to ensure sustainability of the project - Do actors consider the project sustainable? What are their perceptions and what improvements do they suggest? - What are beneficiaries and intermediary beneficiaries’ perceptions on the sustainability of the project? Do they believe the effects will last beyond the life of the project? What would they suggest to increase sustainability? - What are government perceptions around sustainability of the project What lessons can be learnt from the project? - What are the lessons learnt from the project from actors and beneficiaries? - What are the lessons to be learnt from the MTE? What changes should be brought to the project? - Ask actors and beneficiaries about the changes they would bring to the project - Analyze MTE to extract recommendations for improvement 55 Purpose 3: DRR/CCA MTE Objectives MTE questions and focus Purpose 3 specific questions /topics O1: Assess the quality of Sabal’s program inputs, implementation activities and outputs Q1: Are interventions on track (schedules, beneficiary numbers and outputs)? How do beneficiaries view the usefulness of the program? Which targets met, which ones not met; Challenges in meeting the planned timeline; Strategies used to overcome the challenges; Adjustments made; Implementation success stories; Beneficiary responses (vulnerable households and communities). Q2: Strengths of Sabal: - design? - implementation - management? - communication and collaboration? Factors affecting management, implementation, collaboration? How was the program designed? How has it been updated? Working in transitional political system? Working with multiple government agencies? Working in the earthquake affected districts? Operationalising the theory of change for resilience building and development outcomes? Linking DRR and CCA at different levels? Harnessing complementary strengths of consortium partners in resilience and development? Balancing GON and NGO relationships? Evidence and MEAL based project management? Q3: Strengths and challenges to the efficiency of interventions? How well implementation adhere to principles and protocols? What factors in implementation and context affect efficiency and quality? Which interventions more or less acceptable to target communities and why? Adequacy of Sabal procedures and protocols – DRR, CCA, integration and mainstreaming? Working with local governments without elected officials? Working with limited and lack of downscaled climate data? How does giving training to staff lead to institutional capacity development of the agency? To what extent climate and disaster information help reduce vulnerability? How is adaptation integrated with local development? How well the interventions across DRR, health, food and agriculture and GESI interact to shape outcomes? O2: Present evidence of change and assess how Q4: What changes (positive and negative) do community Changes could be in the areas of: disaster management, climate change adaptation, collective action and institutional 56 well the observed changes support Sabal ToC members and other stakeholders associate with Sabal? Do these changes match Sabal’s ToC hypothesis? Which interventions influence knowledge, practices, attitudes and behaviours? And which influence wider social and natural environment? development, new knowledge and technology, access to public services and enabling policy arrangement. How has agro-advisory services worked? Is community based early warning system working? How does school based DRR intervention connect to the wider DRR management? To what extent observed changes could lead to or reflect transformational adaptation? Any cases of short-term positive change with potential ‘maladaptation’ in the long run? Changes in institutional practices and collective action? Q5: To what extent Sabal has been effective in operationalizing GESI? Are the most vulnerable and poorest benefitting from Sabal activities? Transaction cost of participating in the long-term planning and disaster preparedness work for the poor? Q6: Has Sabal developed and implemented sustainability strategies? - outcomes to be sustained? - organizations/services relationships needed identified? - service provider motivations? - beneficiary motivation? - support and resources for service providers identified? - sustaining services and motivation after the project ends? - Government engagement? Is there a clear plan for sustainability for the outcomes under this purpose? How are various known / likely attributes of sustainability addressed in practice? Local PNGO capacity? Quality of engagement and collaboration with district government agencies? Institutionalization of DRR and CCA within local governments? O3: Assess MEAL Q7: What are Sabal’s success and lessons learned for MEAL? - capturing and documenting stakeholder feedback? - learning and documentation leading to program change? - improving the use of data in program Key success stories? How are stakeholder feedback collected, analysed and acted upon? Data management, analysis and sharing practices? How are lessons identified, recorded, communicated, and acted upon to enhance the results under purpose 3? Is there a system to monitor risk environment? How often? 57 How did the program adapt to respond to the earthquakes? Knowledge exchange between filed practitioners (mostly using Nepali language) and senior experts and mangers (mostly using English language) – knowledge interface between two language worlds in relation to disaster and resilience? O4: Recommendation: What changes are needed in the following: - TOC - Activity design - Resource allocation - Activity management - M&E Plan - Implementation Q7: How should Sabal’s activities be modified to improve: - the program’s acceptability to targeted communities - for efficiency and effectiveness of implementation? How should Sabal’s ToC be refined or modified? How should Sabal adjust its strategies to fit new local government system? How should Sabal better integrate interventions across different thematic areas to achieve the goal (resilience, health, income generation, and food security)? What are the gaps from target audience perspectives? Areas where efficiency could be improved? Opportunities to improve the quality of implementation? Creating enabling policy response – integration, mainstreaming, decentralisation and others? What changes in M&E practice could enhance DRR/CCA interventions effectiveness? 58 Gender Equity and Social Inclusion Domain Questions Project Design • Is the project design feasible to achieve project outcomes and GESI outcomes? What is working well? What can be improved? • To what extend do staff understand the fundamentals of the TOC? • How is gender equity and social inclusion addressed in the TOC and project design? • To what extend does the Gender Equity and Social Inclusion Analysis inform the TOC? • Were communities engaged in the project design • Do community men and women feel that the GESI approach answers their gender needs and aspirations? What is working well; what can be improved? • What would communities suggest to improve participation in GESI design and implementation? Management and Leadership • Who is responsible for gender equity and social inclusion implementation? • What is the role of consortia management on GESI implementation? • To what extend does each partner organization prioritize gender equity and social inclusion? What is each organization’s role on GESI? • How does GESI communication flow from management to filed staff and beneficiaries? Gender Equality and Social Inclusion Strategy • What are the key gender mainstreaming strategies for Sabal? • What has informed the adoption of these strategies? • What steps did the program follow to operationalize the strategies at consortium level and at field/community level? • What are SABAL key social inclusion issues and strategies? What has informed the adoption of these strategies? • How are social inclusion issues operationalized across the consortium and at community level? What is working well? What could be improved? • To what extend is Sabal implementing gender transformative approaches? • How has been the application of GESI strategies across the consortium/ Are there standard operating procedures on application of GESI strategies? What is working well? What can be improved? 59 Stakeholder Engagement • Who are the program’s key stakeholders and GESI key partners? • Are they aware of program design and GESI strategy? • To what extend were partners involved in the development of GESI strategy? Who was involved and at what level were they involved? • Did they receive training on GESI approaches? • What are their roles and responsibilities on GESI programming? • How can GESI be modified to improve effectiveness of its implementation? What aspects of the GESI approach need to improve and what changes can be brought to program? • What are the lessons learnt from the project on GESI programming? Training • What type of GESI trainings does SABAL provide to staff/PNGOs? • What tools and methods are used? • How does the program ensure quality assurance? • What mechanisms are in place to ensure dissemination of GESI trainings and messages? Who is involved in rolling out the trainings? • Does the programme provide enough resources to support GESI trainings? Implementation of Work plans • What has been the process of developing GESI work plans? • Who is involved in the development of work plans? • How have activities been defined? • Was the gender analysis findings considered in the development of work plans? How? • How have you addressed gender equity issues in the work plans? • How have you addressed social inclusion issues in the work plans? • How are roles and responsibilities defined on implementation of GESI work plans? • What is the role of PNGOs and staff at all levels in the implementation of plans? • How do you monitor implementation of work plans? • Are there differences in implementation of GESI work plans across partners and by districts? 60 Monitoring, Evaluation and Learning; and Reporting • How is the project designed to measure gender equity and social inclusion outputs and outcome indicators? • What kind of tools is the program using to measure gender indicators? • How useful are these tools in capturing gender including gender qualitative changes? • How is GESI information analyzed, reported, shared and disseminated across the consortia and beneficiaries? • What are the gender specific indicators under each Purpose Area? What kind of activities contribute to the indicator and what are the data sources? • Does the project have a gender learning agenda￾what are the key issues on the learning agenda? Changes to people’s lives: Gender and Livelihoods • What gender equity and social inclusion interventions are conducted to promote Purpose 1 outcomes on strengthening livelihoods and improve food security? Who are the target groups? • Are there any positive changes that have been observed as a result of GESI interventions both at household and community level in relation to the following: social and cultural norm change; meaningful participation and decision making; leadership; access and control of resources/assets; income etc. • How do beneficiaries perceive these changes brought by the project? • Do they see any change in knowledge, attitudes, perceptions, and behaviors? • Are there any negative or unintended results that have been observed because of GESI interventions eg gender based violence? How did you address? • How has Sabal ensured integration of women, youth and socially excluded groups in agricultural value chain activities? • How has GESI interventions promoted achievement of the following key components for women and socially excluded groups: -market linkages; access to inputs and agricultural services; life skills training; ag improved technologies; access to savings and credit; vocational and entrepreneurship training. • What changes would you bring to the projects? What changes would you bring to the project? 61 Changes to people’s lives: Gender and Health and Nutrition • What gender equity and social inclusion activities are conducted to promote Purpose 2 outcomes on health and nutrition? • What gender changes have been observed as a result of GESI interventions addressing negative social norms; decision making; control of resources/assets/finance; gender based violence etc • How has GESI interventions promoted achievement of the following key components for women and socially excluded groups: adoption of hygiene and nutrition behaviors; knowledge on health and WASH practices • How do community members perceive these changes? • Are there any negative or unintended results that have been observed because of GESI interventions e.g., gender-based violence? How did you address? • What is working well and what can be improved? What changes would you bring to the project? Changes to people’s lives: Gender and DRR/Resilience • What gender equity and social inclusion activities are conducted to promote Purpose 3 outcomes on disaster management and climate adaptation? • What key strategies did the project use to address gendered vulnerabilities to hazards and shocks • Has the adaptive capacity of the vulnerable and marginalized groups increased? If yes, how? and what gender equity and social inclusion strategies were used? • Are there any positive changes that have been observed as a result of GESI interventions both at household and community level? • How do beneficiaries perceive these gender changes and what is the effect on communities’ ability to mitigate, adapt, and recover from shocks and stresses? • What is working well and what is not? What changes can be brought to the project? Sustainability and Exit Strategy • How does the program ensure sustainability of gender equity and social inclusion interventions? • What factors have been put in place to enhance sustained GESI benefits? • How do you keep the different groups motivated? 62 Annex C: Fieldwork Schedule Date Team member Organization Stakeholder/ Focal person Venue Activity 5-Jul Hemant Raj Ojha and Jeffrey L. Gray Arrival 7-Jul Tim Finan, Opper M. Maravanyika, David Doledec NA Hotel Radisson, Lazimpat Arrival 8-Jul All Hotel Radisson, Lazimpat MtE team meeting 9-Jul All Sabal/Save the Children Sabal COP, Sr. DCOP, DCOPs and Sriju Sharma Interpreters Hotel Radisson, Lazimpat Team meeting, visit by Sabal COP, Sr. DCoP, DCoPs, Sriju Sharma, Sarita and Interpreters 10- Jul All Save the Children SC Security Coordinator Security Department Security briefing All Sabal/Save the Children Senior Management team, MEAL Sabal conf. hall General meeting with Senior Management Team Tim Finan and Jeffrey Gray Li- Bird Dr. Balaram Thapa, Executive Director, Li- Bird Li-Bird office Sanepa/ Sabal conf. hall Individual meeting with Li-Bird 63 David Doledec HKI Dale Davis, Country Director and Asha Basnyat, Deputy Country Director HKI office, Chakupat Individual meeting/KII Opper Maravandyika and Hemant Ojha CARE Lora Wuennenberg, Country Director and Dr. Popular Gentle, Program Director CARE Office, Dhobighat Individual meeting/KII Tim Finan and Jeffrey Gray Sabal/Save the Children Lumanath Adhikari, Technical Director - Livelihood and NRM Sabal conf. hall Individual meeting/KII David Doledec Sabal Dr. Jaganath Sharma, Sr. Technical Manager-H&N Save the Children Siraha hall, 2nd floor Individual meeting/KII Jeffrey Gray Sabal Dr. Amar Shah, Agriculture Specialist Sabal conf. hall Individual meeting/KII Opper Maravandyika Sabal Sarita Karki, GESI Specialist Save the Children Siraha hall, 2nd floo Individual meeting/KII Hemant Ojha Sabal Man B. BK, Sr. Technical Manager-DRR/CCA Save the Children meeting hall 3 Individual meeting/KII Tim Finan USAID/Nepal Bijaya Bajracharya US Embassy, Maharajgunj Individual meeting/KII Jeffrey Gray Sabal Ghuran Thakur, Sr. Advisor Financial Service Sabal conf. hall Individual meeting/KII 11- Jul Jeffrey Gray Tim Finan DEPROSC Ganesh KC, Program Advisor Save the Children Program hall 1st floor Individual meeting/KII David Doledec NEWAH Umesh Pandey, Director, NEWAH Sabal conf. hall Individual meeting/KII 64 Tim Finan and Jeffrey Gray Sabal Gyaneshwar Mahato, VT and Enterprise Development Advisor Save the Children Program hall 1st floor Individual meeting/KII David Doledec NTAG Deepak Thapa, Program Director Sabal conf. hall Individual meeting/KII David Doledec ACF Sudipta Badapanda, Country Director Sabal conf. hall Individual meeting/KII Opper Maravandyika Hemant Ojha Jeffrey Gray Tim Finan Sabal Santosh Ghimire, Sr Program Integration and Social Mobilization Manager Save the Children Program hall 1st floor Individual meeting/KII All Save the Children Delailah (Lala) Borja, CD CD Office, Save the Children Individual meeting/KII Tim Finan Sabal/Save the Children Bhola Shrestha, DCOP-Finance & Operations Save the Children Program hall 1st floor Individual meeting/KII 12- Jul Tim Finan Sabal Bharat Devkota, Sr. SCoP Sabal conf. hall Individual meeting/KII David Doledec Child Health Division (CHD) GoN partner staff Teku Individual meeting/KII David Doledec Nutrition Section, CHD GoN partner staff Teku Individual meeting/KII Jeffrey Gray Department of Agriculture GoN partner staff Harihar Bhawan Individual meeting/KII 65 Jeffrey Gray Department of Livestock GoN partner staff Harihar Bhawan Individual meeting/KII Hemant Raj Ojha Fire Prevention and Disaster Management Section of Ministry of Federal Affairs and Local Development GoN partner staff Singha Durbar Individual meeting/KII Tim Finan Ministry of Home Affiars GoN partner staff Singha Durbar Tim Finan National Planning commission GoN partner staff Singha Durbar Individual meeting/KII Tim Finan and Jeffrey Gray DUDBC, Ministry of Urban Development GoN partner staff Individual meeting/KII Hemant Raj Ojha Ministry of Population and Environment GoN partner staff Singha Durbar Individual meeting/KII All SC Office SC support staff/interpreters/drivers Sabal conf. hall Field orientation/ preparations for the field visits 66 13- Jul All Kavre Sabal district staff, GoN counterparts Meeting with district Sabal team, GoN counterparts 14- Jul Team 1 Baluwapati Deupur PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) Team 1 Devitar PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) 15- Jul All Travel to Sindhuli All SC District Office Sabal district staff, GoN counterparts Meeting with district Sabal team, GoN counterparts 16- Jul Team 1 Hatpate PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) Team 2 Kapilakot PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) 17- Jul Team 1 Hatpate PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) Team 2 Kapilakot PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) 18- Jul All Travel to Okhaldhunga 67 All SC District Office Sabal district staff, GoN counterparts Meeting with district Sabal team, GoN counterparts 19- Jul Team 1 Madhavpur PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) Team 2 Singhadevi PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) 20- Jul Team 1 Madhavpur PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) Team 2 Singhadevi PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) 21- Jul Team 1 Madhavpur PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) Team 2 Singhadevi PNGO staff, GoN countparts, beneficiaries, etc. Field interviews (KIIs and FGDs) 22- Jul All Return to Kathmandu 23- Jul All Kathmandu (hotel) Debriefing preparations 24- Jul All Sabal/ Save the Children Sabal conf. hall Debriefing with Senior Management Team All Hotel Presentation revisions 68 25- Jul All Sabal/Save the Children Presentation of preliminary results to stakeholders All Leave Nepal 69 Annex D: List of Interviews and Focus Groups Key Informant Interviews Name Sex Position Office/Organization Location Purpose 1 Livelihoods Dr. Balaram Thapa Dr. Amar Shah M M Executive Director Sr. Agriculture and Livestock Advisor Local Initiatives for Biodiversity, Research and Development (LI￾BIRD) Kathmandu Lumanath Adhikari M Technical Director SCI Kathmandu Livelihoods Team Ghuran Thakur M Senior Advisor for Financial Services Development Program Service Center (DEPRSC) Kathmandu Livelihoods Team Ganesh Kumar M Program Advisor DEPRSC Kathmandu Livelihoods Team Dr. Amar Shah M Senior Agriculture and Livestock Advisor LI-BIRD Kathmandu Livelihoods Team Minakshi Chhetri F Environment Compliance Officer SCI Kathmandu Moon Pradhan F Senior Knowledge Management and Outreach Communications Manager SCI Kathmandu Sangita Budhathoki F SBCC Manager SCI Kathmandu Grishma Shrestha F SBCC Manager CARE Kathmandu Santosh Ghimire M Senior Program Integration and Social Mobilization Manager SCI Kathmandu Gyaneshwar Mahato M Senior Technical Advisor for Employment and Enterprises SCI Kathmandu Livelihoods Team Surya Prasad Paudel Dal Prasad Pudasainy M F Deputy Director Senior Agricultural Economist Ministry of Agriculture, Department of Agriculture (DoA) Kathmandu Dr. Chandra Dhakal M Senior Livestock Development Officer Department of Livestock Services (DLSO) Kathmandu Devraj Adhikari M Plant Protection Officer District Agriculture Dev. Office (DADO) Kavrepalachok District Shyam Narayan Mahato M Veterinarian Officer District Livestock Services Office (DLSO) Kavrepalachok District Local Resource Person (LRP) - Service Provider F Crop Nursery Owner Devitar VDC - Private Kavrepalachok District VAHW - Service Provider M Village Animal Health Worker (VAHW) Devitar VDC - Private Kavrepalachok District Dinesh Bahadur Basnet M Agriculture and Livestock Coordinator Dhulikhel Cluster Office Kavrepalachok District Village Savings and Loan Facilitator F VSLA Facilitator and Farmer Group (FG) member Baluwati Deupur VDC Village of Gargaon Kavrepalachok District Farmer Leader F Farmer Leader and FG Member Baluwati Deupur VDC Village of Gargaon Kavrepalachok District Hom Raj Bista M Director, Senior Agriculture Development Officer District Agriculture Development Office (DADO) Sindhuli District Karki Ram Kumer M Director, Senior Veterinary Officer District Livestock Services Office (DLSO) Sindhuli District 70 Bhav Narayan Jha M District Agriculture and Livestock Coordinator LI-BIRD Sindhuli District Shiva M Micro-health Insurance Coordinator includes Crop/Livestock Ins. Sindhuli Integrated Development Services (SIDS) PNGO Sindhuli District Kamala Paudel F Start and Improve Your Business (SIYB) Self-employed; FG member Sindhuli District Yadu Ghimire M VAHW - Service Provider Self-Employed Sindhuli District Devika Karki F Manager, Surymukhi Women’s Savings and Credit Cooperative Janakpur, Dhanusa District Sindhuli District Padam Sapkota M Agriculture and Livestock Officer (ALO) Hamro Prayash Nepal (HPN) PNGO Sindhuli District Sabitra BK F Start Your Improved Business (SIYB) Bagawati Farmer Group Member Sindhuli District Surya Bahadur Gole M Lead Farmer and President, Bagawati Farmer Group Bagawati Farmer Group Sindhuli District Binda Khapange F Drip Irrigation and Treasurer, Bagawati Farmers Group Bagawati Farmer Group Sindhuli District Sachina Sunuwar F Agriculture and Livestock Officer LI-BIRD Okhaldhunga District Jhalaknath Kandel M Sr. Agricultural Development Officer DADO Okhaldhunga District Dr. Bidur Prasad Gautam M Senior Veterinary Officer DLSO Okhaldhunga District Indira Shhestha, F Financial Services Officer DEPRSC Okhaldhunga District Agricultural LRP – Service Provider M Agricultural local resource person Madhapur VDC Okhaldhunga District Project Coordinator Punyawati Shrestha F Program Coordinator Kossis PNGO Okhaldhunga District Ram Baniya M Program Coordinator FORD PNGO Okhaldhunga District Purpose 2 Nutrition Dr. Jaganath Sharma M Sr. Technical Manager-H&N Sabal Kathmandu Umesh Pandey M Director, Nepal Water for Health (NEWAH) Kathmandu Deepak Thapa M Program Director Nepali technical assistance group (NTAG) Kathmandu Dale Davis Asha Basnyat, F F Country Director Deputy Country Director Helen Keller International (HKI) Kathmandu Sudipta Badapanda Not noted M M Country Director National program coordinator Action Against Hunger (AAH) Kathmandu Dr. Bikash Lamichhane M Director Child Health Division (CHD) Kathmandu Sangita Budhathoki F SBCC Manager Sabal Kathmandu Raj Kumar Pokharel M Nutrition Section chief Child Health Division (CHD) Kathmandu Not noted F Nurse in charge Health post Baluwapati deupur VDC Dinesh Bhandari M District Head of Health and nutrition Sabal Singhuli district Gujeshwor Lal Shrestha M Nutrition Focal Person District health office Sindhuli district 71 Village model farmer (VMF) F VMF VMF VDC ? Female Community Health Volunteer) (FCHV) F FCHV FCHV VDC ? Not noted F Nurse in charge Health post VDC ? Not noted F Nurse in charge Health post Kapilakot VDC VMF F VMF VMF Kapilakot VDC FCHV F FCHV FCHV Kapilakot VDC FCHV F FCHV FCHV Kapilakot VDC Sabal Facilitator F Facilitator Sabal Kapilakot VDC Not noted M DPHO District health office Okhaldhunga district WASH facilitator M WASH Sabal Madavpur VDC FCHV F FCHV FCHV Madavpur VDC FCHV F FCHV FCHV Madavpur VDC FCHV F FCHV FCHV Madavpur VDC Not noted F Nurse in charge Health post Madavpur VDC VMF F VMF VMF Madavpur VDC FCHV F FCHV FCHV Singadevi VDC VMF F VMF VMF Singadevi VDC Not noted F Nurse in charge Health post Singadevi VDC Purpose 3 DDR/CCA Dolma Tamang F Chair Baluwapati Women Savings and Credit Group Baluwapati, Kavre Saraswati Dhakal F Secretary Mandan-Deupur Municipality Ward 1- 2(new) Baluwapati, Kavre Krishna B Tamang M Village elderly Community Dandagau, Baluwapati, Kavre Sahnha Maya Tamang F Local Resource Person Sabal, Shanti Janaastha Sewa Kendra/Kavre PNGO Dandagaun, Baluwapati, Kavre Ashok Byanjyu M Mayor Dhulikhel Municipality Dhulikhel, Kavre Bhim K Sharma Assistant Chief District Officer District Administration Office Dhulikhel, Kavre Hari Krishna Bhattarai M DRR Coordinator (district level) Sabal Dhulikhel, Kavre Komal P Dhamala M Local Development Officer District Development Committee Dhulikhel, Kavre MR Baral M Admin officer Sabal, Shanti Janaastha Sewa Kendra/Kavre PNGO Dhulikhel, Kavre Not noted M Programme Coordinator Sabal, Shanti Janaastha Sewa Kendra/Kavre PNGO Dhulikhel, Kavre Shakti Lama M DRR/CCA Coordinator Sabal Dhulikhel, Kavre Sunil Shakya M DRR/ CCA trainer Consultant Sabal Dhulikhel, Kavre Bharat Devkota M DCOP Sabal Kathmandu Grishma Shrestha Social and Behavioural Change Manager Sabal Kathmandu 72 Kirti Raya M DRR Coordinator CARE Nepal Kathmandu Lumanath Adhikari M Livelihoods Coordinator Sabal Kathmandu Man B BK M Coordinator of DRR program and member of Senior Management Team Sabal Kathmandu Mardip Sunuwar M Section Officer Ministry of Federal Affairs and Local Development Kathmandu Naresh Sharma M Officer Ministry of Population and Environment Kathmandu Popular Gentle M Program Director CARE Nepal Kathmandu Purushottam Subedi M Under Secretary Ministry of Federal Affairs and Local Development Kathmandu Ram P Lamsal M Joint Secretary Ministry of Population and Environment Kathmandu Robert Cardinalli M COP Sabal Kathmandu Sandesh Bista M Coordinator Network of Community Disaster Management Committee Kathmandu Sangita Budhathoki F Social and Behavioural Change Manager Sabal Kathmandu Sriju Sharma MEAL Coordinator Sabal Kathmandu Shankar Acharya M Under Secretary National Emergency Operation Center Kathmnandu Dinesh Dhamala M Field Supervisor PNGO Koshish Nepal Madhabpur, Okhaldhunga Kalpana Basnet F Ward Secretary Madhavpur VDC Madhabpur, Okhaldhunga Nanibabu Magar M Newly Elected Ward Chair, Madhavpur Madhavpur VDC/New Rural Municipality Madhabpur, Okhaldhunga Bal B Bhujel M Sabal facilitator PNGO Hamro Prayash Marin, Sindhuli Bir Bikram Dhakal M Field technician (agriculture) Haro Prayash Marin, Sindhuli Buddhi Maya Bhujel F DRR/CCA LRP Haro Prayash Marin, Sindhuli Budhi L Majhi M Chair of School Management Committee Kalika Higher Secondary School Marin, Sindhuli Dasharath Thapa M DRR/CCA field supervisor Haro Prayash Marin, Sindhuli Bikaram KC M PNGO Coordinator Haro Prayash Nipani, Sindhuli Chandrawati BK F Sabal facilitator Sabal PNGO Hamro Prayash Nipani, Sindhuli Gomati Uprety F DRR/CCA Coordinator (district level) Sabal Okhaldhunga Okhaldhunga Indra Tamang M Sabal district coordinator Sabal Okhaldhunga Punyawati Shrestha PNGO Coordinator – Koshish Nepal Okhaldhunga Ram B Baniya PNGO Coordinator – FORD Nepal Okhaldhunga Ram P Gyawali Assistant CDO District Administration Office Okhaldhunga Dudarahan Kharwar DRR/CCA Coordinator (district level) Sabal Sindhuli Narahari Khatiwada M Assistant Chief District Officer District Administration Office Sindhuli Tara B Karki M Chief District Officer District Administration Office Sindhuli Hita Raj Karki M PNGO staff FORD Advisor Singhadevi, Okhaldhunga 73 Januka Baral F DRR LRP FORD Singhadevi, Okhaldhunga Ramhari Budhathoki M Newly Elected Ward Chair, Singhadevi (Likhu Rural Council) Local government Singhadevi, Okhaldhunga Nishan Shrestha M Field supervisor FORD Singhadevi, Okhaldhunga Gender Sarita Karki F GESI Specialist CARE Kathmandu Popular Gentle M Programme Director CARE Kathmandu Indu Pant Ghimire F GESI Advisor CARE Kathmandu Lora Wuennenberg F Country Director CARE Kathmandu Sangita Budhathoki F Social Behaviour Change and Communication Manager SAVE Office Kathmandu Moon Pradhan F Snr Knowledge Management and Outreach Communication Manager SAVE Office Kathmandu Lumanath Adhikari M Technical Director￾Livelihoods and NRM SAVE Office Kathmandu Amar Shah M Snr. Agriculture and Livestock Advisor SAVE Office Kathmandu Man Bahadur M DRR and CCA Sr. Technical Manager CARE Kathmandu Gyaneshwar Mahato M Snr Technical Advisor￾Employment and Enterprise Development SAVE Office Kathmandu Jaganath Sharma M Snr Technical Manager￾Health and Nutrition HKI Kathmandu Tri Bikram Dhungana M Snr MEAL Coordinator SAVE Office Kathmandu Tara Nath Paudel M MEAL Coordinator SAVE Office Kathmandu Bharat Devkota M Snr DCOP SAVE Office Kathmandu Not noted F LRP Kavre District Davitar VDC Not noted F LRP Kavre District Baluwapeti VDC Not noted M Micro Health Insurance Coordinator; Employment and Enterprise Coordinator; Micro Finance Service Officer Sabal Office Sindhuli district Bikram KC M Field Coordinator PNGO Sindhuli District Nipane VDC Bal Krisha Lakhe M LRP Sindhuli District Kapilakot Binita Syantang F Sabal Facilitator Sinduli District Kapilakot Prabala Shrestha F Women Development Officer￾Department of Women and Children Okhaldhunga District Gomati Joshi F DRR/CCA Officer/GESI Focal person Sabal Office Okhaldhunga District Not noted M&F 2 Programme Coordinators PNGO Okhaldhunga District Not noted 4 Health and Nutrition team members Sabal Office Okhaldhunga District Not noted F Village Model Farmer Okhadhunga Madavpur VDC Not noted M LRP-DRR/CCA Okhadhunga Madavpur VDC Not noted M LRP- Agriculture Okhadhunga Madavpur VDC Team Leader Interviews Robert Cardinelli M Sabal COP Country Office Kathmandu 74 Sriju Sharmu F MEAL coordinator Country Office Kathmandu Balaran Thopa M Director Li-Bird Kathmandu Lumanath Kari M Livelihoods senior officer Sabal Country Office Kathmandu Sangita Budhathoki F SBCC media staff Sabal Country Office Kathmandu Ms. Moon F SBCC coordinator Sabal Country Office Kathmandu Mr. Ganesh M Financial services coordinator DEPROSC Kathmandu Santosh Ghimire M Social Mobilization Sabal Country Office Kathmandu Tri Bikram, Om Krishnayrestha, Tara Nath Paudel M Meal staff Sabal Country Office Kathmandu Mr. Bhola M Financial Management Officer Sabal Country Office Kathmandu Parikshit Kadaria M Focal Point DUDBC Kathmandu Naresh Sharma M Focal Point Ministry of Environment Kathmandu Shankar Harl Acharya M Focal Point MOHC (disasters) Kathmandu Shakti Lama M Director Sabal District Office Kavrepalanchouwk Not noted F Sabal facilitator Pokhari Narayansthan Kavrepalanchouwk Not noted M Local resource person Pokhari Narayansthan Kavrepalanchouwk Not noted M Local resource person Pokhari Narayansthan Kavrepalanchouwk Mr. Sukra M Director Sindhuli District Office Sindhuli Bhav Nayran M Livelihoods officer Sindhuli District Office Sindhuli Chandrawati F Sabal facilitator Sindhuli Nipani Kamal Katwal M Ag technician PNGO Sindhuli Nipani Ms. Binita F Sabal facilitator Sindhuli Kapilakot Ms. Kumari Bhal F Sabal lead farmer Sindhuli Kapilakot Mr. Suresh Khatiwada M PNGO program coordinator Sindhuli Kapilakot Gadhu Lama M Mason beneficiary Sindhuli Kapilakot Bhim Thapa M Local resource person Okhaldhunga Madhapur Deepak Maugor M Local resource persons Okhaldhunga Madhapur Rama Bhuzel F Sabal facilitator Okhaldhunga Madhapur Urmila Gyimire F Sabal facilitator Okhaldhunga Singhadevi Parbati Katwan Sabal facilitator Okhaldhunga Singhadevi Min Khatri F Local resource person Okhaldhunga Singhadevi Umita BK F Local resource person Okhaldhunga Singhadevi Rebecca Goldman F USAID Mission Kathmandu 75 Focus Groups Focus Group Type # M #F Region District Village Purpose 1 Livelihoods Farmer Group / VSLA 11 14 Kavre Devitar VDC Gorawara Farmer Group / VSLA 3 18 Sindhuli Nipane #9 VDC Zunga Ekata Farmer Group / VSLA 0 24 Sindhuli Nipane #9 VDC Zunga Bagawati Farmers Group 3 12 Sindhuli Kapilakot Ward 6 Nayabasti Farmer’s Group 6 10 Sindhuli Kapilakot Ward 8 Barari Farmers Group / VSLA 0 15 Okhaldhunga Madhapur VDC Rachana Farmers Group / VSLA 2 10 Okhaldhunga Madhapur Mihjubi Krisak Samuha FG/VSLA 2 15 Okhaldhunga Singhadevi Bagkhor Krisak Samuha FG/VSLA 3 14 Okhaldhunga Singhadevi Pragatoshel Krisak Samuha FG 2 12 Okhaldhunga Singhadevi Purpose 2 Nutrition Farmer Group 0 14 Kavre Devitar VDC Farmer group 0 7 Kavre Baluwapati deupur VDC Sabal Health & Nutrition team 4 0 Sindhuli Sindhuli 1,000 days mothers group 0 15 Sindhuli Sindhuli 1,000 days mothers group 0 22 Sindhuli Sindhuli Kapilakot VDC VMF group 0 6 Sindhuli Sindhuli Kapilakot VDC 1,000 days mothers group 0 10 Sindhuli Sindhuli Kapilakot VDC Sabal Health & Nutrition team 3 1 Okhaldhunga Okhaldhunga 1,000 days mothers group 0 4 Okhaldhunga Okhaldhunga Madavpur VDC 1,000 days mothers group 0 13 Okhaldhunga Okhaldhunga Madavpur VDC 1,000 days mothers group 0 20 Okhaldhunga Okhaldhunga Singadevi VDC FCHV 0 5 Okhaldhunga Okhaldhunga Singadevi VDC VMF 0 4 Okhaldhunga Okhaldhunga Singadevi VDC Purpose 3 DDR/CCA Sabal SMT members Kathmandu Sabal head office Sabal district staff 5 3 Sindhuli Sindhuli Sindhuli Community Disaster Management Committee (CDMC) / REFLECT Group 4 5 Kavre Baluwapati CDMC / REFLECT Group 5 9 Kavre Devitar District level government officers 1 9 Okhaldhunga Okhaldhunga Okhaldhunga Sabal and PNGO staff 3 2 Okhaldhunga Okhaldhunga Okhaldhunga Local Disaster Management Committee 3 4 Okhaldhunga Okhaldhunga Madhavpur, Champadevi CDMC 7 6 Okhaldhunga Okhaldhunga Madhavpur, 76 Farmers groups 10 6 Okhaldhunga Okhaldhunga Madhavpur, School Management Committee 2 3 Okhaldhunga Okhaldhunga Madhavpur, Local government experts 5 0 Kathmandu Old Baneshwor School children parents 2 3 Okhaldhunga Okhaldhunga Singhadevi, Kopila and Makhamali women’s groups 2 8 Sindhuli Sindhuli Nipani-5 Jorsimal CDMC members and REFLECT group 6 5 Sindhuli Sindhuli Marin Gender Community Disaster Management Committee (CMDC) 3 9 Kavre Devitar VDC Farmer/VSLA group 0 20 Kavre Baluwapeti VDC Community Disaster Management Committee (CDMC) 2 7 Kavre Baluwapeti VSLA Group 3 21 Sindhuli Sindhuli Nepane Kopila Farmer Group 5 15 Sindhuli Sindhuli Nepane Farmer Group 1 36 Sindhuli Kapilakot WASH Committee 1 4 Okhaldhunga Okhadhunga Madavpur 1000 days mothers group 0 13 Okhaldhunga Okhandunga Madavpur Female Community Health Volunteers (FCHV) 0 6 Okhaldhunga Okhandunga Singadevi 1000 days mothers group 0 24 Okhaldhunga Okhandunga Singadevi Additional FGs by Team Leader PNGO 15 15 Kavrepalanchouwk Dhulikhel Masons/vocational training 6 0 Kavrepalanchouwk Pokhari Narayanstha PNGO Kosis-Nepal 6 9 Sindhuli Sindhuli Sindhuli PNGO Ford-Nepal 5 11 Sindhuli Sindhuli Sindhuli District Commission Office 14 5 Okhaldhunga Okhaldhunga Okhaldhunga 77 Annex E: Map of Original + Two Expanded Districts Implementation Period: Oct 2014 to Oct 2019 Districts from Year 4: 1) Makawanpur, 2) Okhaldhunga 3) Ramechhap 4) Khotang 5) Udayapur, 6) Sindhuli 7) Kavrepalanchowk 8) Sindhupalchowk 9) Dolakha REMOVED 10) Rasuwa REMOVED 11) Nuwakot REMOVED Livelihood VDCs: 214 in 8 districts H&N VDCs: 326 in 6 districts DRR/CCA VDCs: 174 in 8 districts 78 Annex F: Livelihoods Implementation Structure SABAL DISTRICT OFFICE • Senior District Coordinator (6) • Consortium partner reps: LI-BIRD, DEPROSC, HKI, CARE, etc. o Agriculture and livestock coordinators (4), multi-district support (3 districts each in six original districts), agriculture officers (6) o Financial service coordinators (2), financial officers (6) PARTNER NGOS (2 per district) • Project coordinators (17 PNGOs) • Agriculture technicians (in all 11 districts) • Agriculture officers (in 5 expansion districts) • Sabal facilitators (aka social mobilizers) • VSLA facilitators MOU with six ministries SABAL Central Office Consortium Partners District Sector Working Group Technical staff @ district level Social facilitator -Farmer groups -Mother groups -Credit groups -DRR/CCA committees Communities and households National Steering Committee 79 Annex G: Targeting for Technical Interventions (diagram) Note: The Government of Nepal is in process of changing the political administrative structure whereby the “Village Development Council” (current terminology) will be called “Municipal Council” and/or “Village Council.” Original 6 districts: H&N = 326 VDCs Sabal full package: Livelihoods = 164 VDCs + H&N + DRR/CCA + cross-cutting 5/3 expanded Districts: Livelihoods = 100 VDCs DRR/CCA = 100 VDCs Food for Peace Nepal Baseline Survey Indicator Guidance and Resilience Module July 24, 2015 TANGO International is a global leader in food security, livelihoods and resilience. TANGO provides technical assistance in assessment, design, monitoring and evaluation for development and recovery programs. We are a preferred partner for government and United Nations agencies, international non￾governmental organizations, local consultants and community organizations. With TANGO, we bring experience, expertise and commitment to our partnerships. Save the Children invests in childhood – every day, in times of crisis and for our future. In the United States and around the world, we give children a healthy start, the opportunity to learn and protection from harm. By transforming children’s lives now, we change the course of their future and ours. The Technical and Operational Performance Support (TOPS) Program is the USAID/Food for Peace￾funded learning initiative, bringing the highest quality information, knowledge, and promising practices in food assistance programming to implementers and donors around the world to ensure more communities and households benefit from the U.S. Government’s investments to fight global hunger. Disclaimer: This guidance document was made possible by the generous support of the American people through the United States Agency for International Development (USAID), under the terms of Cooperative Agreement No. AID-OAA-A-10-00006, August 13, 2010, to August 12, 2015, through The Technical and Operational Performance Support (TOPS) Program, managed by Save the Children. The contents are the responsibility of TANGO International and do not necessarily reflect the views of USAID or the U.S. Government. Recommended citation: TANGO International. 2015 (draft). Food for Peace Nepal Baseline Survey Indicator Guidance and Resilience Module. Washington, DC: The TOPS Program and Save the Children US. Contacts: Tim Frankenberger TANGO International 376 South Stone Ave. Tucson, AZ 85701 tim@tangointernational.com www.tangointernational.com Mark Fritzler c/o Save the Children USA 2000 L Street, NW, Suite 500 Washington, DC 20036 mfritzler@savechildren.org www.thetopsprogram.org Contents 1. Measuring Resilience ............................................................................................................................ 1 2. Resilience Indicators for PAHAL and SABAL in Nepal ........................................................................... 1 3. Corresponding questions for indicators, and indicators to measure annually ..................................... 1 4. Resilience module ................................................................................................................................. 2 5. Calculation of shock exposure and measures of resilience ................................................................16 5.1. Index of shock exposure .............................................................................................................16 5.2. Index of cumulative impact of shock exposure ..........................................................................16 5.3. Absorptive capacity index ...........................................................................................................17 5.4. Adaptive capacity index ..............................................................................................................18 5.5. Transformative capacity index ....................................................................................................20 5.6. Index of household resilience capacity .......................................................................................22 6. Responses to Shocks and Stresses ......................................................................................................22 7. Suggested research questions for qualitative component ................................................................. 24 1 1. Measuring Resilience Resilience is viewed as a set of capacities that enable households and communities to effectively function in the face of shocks and stresses and still meet a set of well-being outcomes. The ability to measure resilience involves measuring the relationship between shocks, capacities, responses, and future states of well-being. Thus there is no single indicator that measures resilience. There is a need for a number of indicators to be used as part of a measurement framework. There are four key factors to consider in measuring resilience:  Identify the well-being outcomes to be achieved and measure resilience in relation to these outcomes.  Identify the shocks and stresses that individuals, households, communities and systems are exposed to and the severity and duration of these shocks and stresses.  Measure the absorptive, adaptive and transformative capacities in relation to these shocks and stresses at different levels.  Identify the responses of individuals, households, communities and systems to these shocks and stresses and trajectory of well-being outcomes. The key questions to be explored through measurement of resilience are:  Does shock exposure have a negative impact on food security and child nutritional status?  Does greater resilience capacity have a positive impact on these outcomes? 2. Resilience Indicators for PAHAL and SABAL in Nepal Wellbeing Outcomes A number of outcome indicators will be used for measuring wellbeing in PAHAL and SABAL: 1. Depth of Poverty: The mean percent shortfall relative to the $1.25 poverty line 2. Prevalence of households with moderate or severe hunger (Household Hunger Scale ‐ HHS) 3. Prevalence of wasted children under five years of age 4. Average Household Dietary Diversity Score (HDDS) 5. Prevalence of households with moderate or severe hunger (Household Hunger Scale ‐ HHS) 6. Prevalence of stunted children under five years of age Shocks and Stresses The following indices will measure the overall degree of shock exposure for each household. 7. Shock exposure index 8. Cumulative impact of shock exposure index The following two indicators will be used two construct the index. The shocks that will be looked at include: flooding and landslides; drought or unpredictable or insufficient rain; earthquake; land/ forest 2 degradation due to cutting for grazing or fuelwood without adequate replanting; pests or disease outbreak (crop or livestock); hailstorms; severe illness of household member/ death in the HH; market price fluctuation; and theft/ conflict.  Number of shocks exposed in the past 12 months  Perceived severity of the shocks Resilience capacities Resilience capacities are being measured as a set of indices, one for each of the three dimensions of resilience capacity—absorptive capacity, adaptive capacity, and transformative capacity—and one overall index combining these three indexes. 9. Absorptive capacity index Absorptive capacity: The ability to minimize exposure to shocks and stresses through preventative measures and appropriate coping strategies to avoid permanent, negative impacts. The absorptive capacity index will be constructed from eight indicators, some of which are themselves indices. The indicators to be used include:  Perceived ability to recover  Access to informal safety nets  Bonding social capital  Whether any household member holds savings  Access to remittances  Asset ownership  Shock preparedness and mitigation  Household has agricultural hazard insurance 10. Adaptive capacity index Adaptive capacity: Making proactive and informed choices about alternative livelihood strategies based on an understanding of changing conditions. The Adaptive Capacity index will be constructed from the following seven indicators, again some of which are themselves indices. The indicators to be used include:  Bridging social capital  Linking social capital  Human capital  Livelihood diversification  Exposure to information  Adoption of improved practices  Asset ownership  Access to financial resources 3 11. Transformative capacity index Transformative capacity: The governance mechanisms, policies/regulations, infrastructure, community networks, and formal and informal social protection mechanisms that constitute the enabling environment for systemic change. The Transformative Capacity index will be constructed from nine indicators, some of which are themselves indices. The indicators to be used include:  Access to formal safety nets  Access to markets  Access to basic services  Access to agricultural services  Access to infrastructure  Bridging social capital  Linking social capital  Gender equitable decision-making index  Active participation in local decision-making bodies Responses Fundamentally, resilience interventions are about strengthening the ability of households (or society) to choose – from a whole 'portfolio' of options – what they perceive at that time as the “right” response(s). An appropriate response (e.g., using social capital, accessing savings) increases the chances of positive wellbeing outcomes, while an inappropriate or ill-chosen one often leads to vulnerability. Resilience analysis should measure the effect of different resilience responses at multiple levels (i.e., households, communities, local, provincial and national authorities). Improved resilience capacity, however, is not simply about avoiding detrimental short-term response strategies that undermine absorptive capacity in particular. It is also about nurturing or fostering the ability of actors to engage in positive and sustainable responses that improve all three resilience capacities, i.e., absorptive, adaptive, and transformative capacity. The types of responses we will be measuring are listed below. Absorptive Responses  Coping strategy index  Use of savings to absorb shocks  Use of remittances to absorb shocks  Use of hazard insurance  Use of bonding social capital to absorb shock Adaptive Responses  Application of information 4  Adoption of Improved Agricultural Practices  Use of savings for adaptation  Use of remittances for adaptation  Use of bridging social capital Transformative Responses  Active participation in decision making  Participation in collective action  Gender equitable decision making index  Use of formal safety nets Recurrent Monitoring Many of the indicators that are collected more regularly are captured in recurrent monitoring. These are collected after a shock to see how households and communities are responding to the shock. Some of these indicators that are recommended to be collected in recurrent monitoring are in section 3. All of the indicators that are marked yes should be collected at the beginning and end of the monitoring exercise with some indicators that change more quickly being collected every two months over a whole year. This will enable the partner to see what responses are taken to the shock over different seasons during the year. The whole shock cycle is captured this way. Projects will also being carrying out annual monitoring of certain outcome indicators related to their project interventions. Which of these indicators are collected will be determined by the specific project. 1 3. Corresponding questions for indicators, and indicators to measure annually Indicator Questions Indicators to measure annually Shock exposure and cumulative impact Exposure: Number of shocks experienced in the past 12 months R101 YES Shock severity: Impact of shock on income security Impact of shock food consumption R105 R106 YES Cumulative impact R102 NO Absorptive capacity index Ability to recover R107, R108 NO Access to informal safety nets R602 YES Bonding social capital R201, R203 NO Whether any household member holds savings R309 NO Access to remittances R305, R306 YES Asset ownership BL-H7.02 YES Shock preparedness and mitigation R601d, R801, R802, R401a,f,m NO Household has agricultural hazard insurance BL-G09 NO Adaptive capacity index Bridging social capital R201 - R204 NO Linking social capital R205-R210 NO Human capital BL-21 NO Livelihood diversification R501 YES Exposure to information R401, 1-15 YES Adoption of improved practices BL-G3b, BL-G16, BL-G18, BL-G21 YES Asset ownership BL-H7.02 YES Access to financial resources R701a,b YES Transformative capacity index Access to formal safety nets R703 YES Access to markets R704-R707 YES Access to basic services R701c,d, BL-F4-F10 YES Access to infrastructure R701f,g,h, R702 NO Access to agricultural services R701e YES Bridging social capital R202, R204 NO Linking social capital R205-R210 NO Gender equitable decision making index BL-J07, J10, BL-K05, BL-K14, BL-K15, R311 YES Active participation in local decision￾making bodies R602 YES 2 4. Resilience module R02A. Respondent's line number (B01) from Module B, Question B10 R02B. OBTAIN WRITTEN CONSENT. DOES [NAME] AGREE TO PARTICIPATE IN THE SURVEY? 1=Yes 2=No  Go to Module H 3=Not available  Go to Module H Shock exposure and severity R100 R101 R102 R103 R104 R105 R106 In the past year (12 months) did your household experience [insert shock]? (READ OUTLOUD FOR EACH SHOCK/ STRESSOR) What were the main impacts of [insert shock] during the last 12 months? [DO NOT READ RESPONSES ALOUD] From whom, if anyone, did your household receive assistance to cope with the impact of [insert shock] within the last 12 months? [DO NOT READ RESPONSES ALOUD] What type of assistance? [DO NOT READ RESPONSES ALOUD] In the months following the shock, how severe was the impact on household INCOME? [READ RESPONSE OPTIONS ALOUD.] In the months following the shock, how severe was the impact on household FOOD CONSUMPTION? [READ RESPONSE OPTIONS ALOUD.] 1=Yes 2=No 8=Don’t Know If R101=2,8 skip to the next item See codes below. SELECT ALL THAT APPLY If R102='Y', skip to R105 See codes below SELECT ALL THAT APPLY If R103='Y', skip to R105 See codes below SELECT ALL THAT APPLY 1=Remained the same 2=Slight decrease 3=Severe decrease 1=Remained the same 2=Slight decrease 3=Severe decrease 1 Flooding/ landslide 2 Drought/ unpredictable or insufficient rain 3 Earthquake 4 Land/ forest degradation due to cutting for grazing or fuelwood without adequate replanting 5 Pests or disease outbreak (crop or livestock) 6 Hailstorm 3 Shock exposure and severity R100 R101 R102 R103 R104 R105 R106 In the past year (12 months) did your household experience [insert shock]? (READ OUTLOUD FOR EACH SHOCK/ STRESSOR) What were the main impacts of [insert shock] during the last 12 months? [DO NOT READ RESPONSES ALOUD] From whom, if anyone, did your household receive assistance to cope with the impact of [insert shock] within the last 12 months? [DO NOT READ RESPONSES ALOUD] What type of assistance? [DO NOT READ RESPONSES ALOUD] In the months following the shock, how severe was the impact on household INCOME? [READ RESPONSE OPTIONS ALOUD.] In the months following the shock, how severe was the impact on household FOOD CONSUMPTION? [READ RESPONSE OPTIONS ALOUD.] 7 Severe illness of household member/ death in the HH 8 Market price fluctuation 9 Theft/ Conflict CODES for R102 CODES for R103 CODES for R104 A Loss of life A Relatives A Food B Loss of labor B Group within the community (savings coop; farmer's coop, church) B Cash gift C Loss of agricultural land C Non-relatives within my ethnic/caste group C Cash loan D Loss of community forest D Non-relatives of other ethnic/caste groups D Clothing E Destruction or damage to home E NGO E Household items F Loss of livestock F Government F Shelter G Loss of crops G My employer G Seed H Loss of fishing waters H My wholesaler H Labor to rebuild/ repair structures I Loss of drinking water sources I Landowner I Agricultural labor 4 J Decline in crop production X Other________ J Transporting HH items to safety K Decline in livestock production Y Did not receive assistance K Childcare L Loss of HH assets (including distress sales) L Land parcel M HH member migrated X Other_________ N HH was displaced O Reduced ability to buy basic needs P Damage to critical infrastructure (roads, markets, health care service) X Other ________________ y No impact Shock exposure and severity (cont’d) CHECK SHOCKS ABOVE IN R101, ROWS 1-9 AT LEAST ONE YES FOR R101, ROWS 1-9: (R101=1) R107 ALL RESPONSES TO R101, ROWS 1-9 "NO" OR "DK": (R101=2,8) R201 R107 To what extent has your household’s ability to meet food needs returned to the level it was before all the shocks and stressors experienced in the last 12 months? [PROMPT] Ability to meet food needs is the same as before the shock…………………..1 Ability to meet food needs is better than before the shock…………… ……..2 Ability to meet food needs is worse than before the shock…………………. .3 R108 In light of the shocks and stressors your household faced in the last 12 months, to what extent do you believe your household will be able to meet its food Ability to meet food needs will be the same as before the shock…………………..1 Ability to meet food needs will be better than before the shock…………… ……..2 5 needs in the next year? [PROMPT] Ability to meet food needs will be worse than before the shock…………………. .3 SOCIAL CAPITAL (Bonding, Bridging and Linking Social Support) USE THE RESPONSE LIST AT RIGHT FOR QUESTIONS R201-R204. CIRCLE THE RESPONSE FOR EACH QUESTION BELOW. [SELECT ALL THAT APPLY] Relatives .............................................................................A Non-relatives within my ethnic / caste group .....................B Non-relatives of other ethnic/caste groups ........................C Other (specify).....................................................................X No one.................................................................................Y Don't know .......................................................................... Z R201 If your household had a problem and needed money or food urgently, who IN THIS COMMUNITY could you turn to for help? A B C X Y Z If X is selected, specify: _________________________________ R202 If your household had a problem and needed money or food urgently, who OUTSIDE THIS COMMUNITY could you turn to for help? A B C X Y Z If X is selected, specify: _________________________________ R203 Who INSIDE THIS COMMUNITY would your household help if they needed food or money urgently? A B C X Y Z If X is selected, specify: _________________________________ R204 Who OUTSIDE THIS COMMUNITY would your household help if they needed food or money urgently? A B C X Y Z If X is selected, specify: _________________________________ R205 Do you or does anyone else in your household personally know an elected government official? Yes 1 No 2 Skip to R208 Don't Know 8 Skip to R208 R206 How do you (or another household member) know the government official? Is he or she a… Family member or relative A 6 SOCIAL CAPITAL (Bonding, Bridging and Linking Social Support) [SELECT ALL THAT APPLY] Friend B Neighbor C Acquaintance D Don't know Z R207 Could you ask the official to help your family or community if help was needed? Yes 1 No 2 Don't know 8 R208 Do you or does anyone else in your household personally know a staff member of an NGO? Yes 1 No 2 Skip to R211 Don't Know 8 Skip to R211 R209 How do you (or another household member) know the NGO staff member? Is he or she a… [SELECT ALL THAT APPLY] Family member or relative A Friend B Neighbor C Acquaintance D Don't know Z R210 Could you ask the NGO staff member to help your family or community if help was needed? Yes 1 No 2 Don't know 8 CIRCLE YES OR NO RESPONSE FOR QUESTIONS R211, R213, R215 AND R217. READ THE RESPONSE LIST AT RIGHT FOR QUESTIONS R212, R214, R216 AND R218. [SELECT ALL THAT APPLY)] Relatives .............................................................................A Non-relatives within my ethnic/caste group .......................B Non-relatives of other ethnic/caste groups ........................C Other (specify).....................................................................X No one.................................................................................Y Don't know .......................................................................... Z R211 In the past 12 months, has your household helped anyone IN THIS COMMUNITY to recover after a shock? Yes 1 No 2 SKIP TO R213 R212 In the past 12 months, who IN THIS COMMUNITY has your household helped to recover after a shock, such as by sharing or giving food, money, supplies/materials, livestock, or labor? A B C X Y Z If X is selected, specify: _________________________________ R213 In the past 12 months, has your household helped anyone OUTSIDE THIS COMMUNITY to recover after a shock? Yes 1 No 2 SKIP TO R215 R214 In the past 12 months, who OUTSIDE THIS COMMUNITY has your household helped to recover after a shock, such as by sharing or giving food, money, supplies/materials, livestock, or labor? A B C X Y Z If X is selected, specify: _________________________________ 7 SOCIAL CAPITAL (Bonding, Bridging and Linking Social Support) R215 In the past 12 months, has anyone IN THIS COMMUNITY helped your household with food, money, supplies/materials, livestock, or labor? Yes 1 No 2 SKIP TO R217 R216 In the past 12 months, who IN THIS COMMUNITY has helped your household with food, money, supplies/materials, livestock, or labor? A B C X Y Z If X is selected, specify: _________________________________ R217 In the past 12 months, has anyone OUTSIDE THIS COMMUNITY helped your household with food, money, supplies/materials, livestock, or labor? Yes 1 No 2 SKIP TO R301 R218 In the past 12 months, who OUTSIDE THIS COMMUNITY has helped your household with food, money, supplies/materials, livestock, or labor? A B C X Y Z If X is selected, specify: _________________________________ 8 Use of remittances and savings R301 Currently, has any member of your family migrated TO SOMEWHERE ELSE IN NEPAL in order to earn an income? Yes 1 No SKIP TO 304 2 R302 How many of your family members are now living SOMEWHERE ELSE IN NEPAL in order to earn an income? R303 Does the household receive remittances from these family members living SOMEWHERE ELSE IN NEPAL? Yes, regularly 1 Yes, irregularly 2 No 3 R304 Currently, has any member of your family migrated TO ANOTHER COUNTRY in order to earn an income? Yes 1 No SKIP TO R307 2 R305 How many of your family members are now living IN ANOTHER COUNTRY in order to earn an income? R306 Does the household receive remittances from these family members living IN ANOTHER COUNTRY? Yes, regularly 1 Yes, irregularly 2 No 3 R307 CHECK ANSWERS TO R303 AND R306: IF R303=3 AND R306=3 SKIP TO R310 R308 How has household income changed, if at all, since these family members began sending remittances? [PROMPT] Large decrease 1 Slight decrease 2 No change 3 Slight increase 4 Large increase 5 R309 How are the remittances mainly used? SELECT ALL THAT APPLY. [PROBE.] Enumerator will categorize responses as follows:  Basic/essential household supplies (food, soap, cooking fuel, etc.)  Luxury and non-essential items. (Electronics, jewelry, cigarettes, alcohol)  To recover from shocks/ stressors. Examples: Health expenses; replace assets lost in the shock; housing (includes damage repair. Does not include improvements and upgrades to undamaged housing.)  Investment in the future. Examples: Education; housing (Includes improvements and upgrades. Does not include damage repair); agriculture (purchase land, tools, livestock, seed, fertilizer, other inputs); small business; savings (for any reason) Basic/essential household items A Luxury and non-essential items B To recover from shock C Investment in the future D Social and religious ceremonies E Loan repayment F To fund migration G Other (specify: _____________________) X 9  Social and religious ceremonies. Examples: Wedding, burial  Loan repayment. Examples: migration loan or other loan Use of remittances and savings (cont’d) R310 Do you or another household member regularly save cash? Yes 1 No SKIP TO R400 2 R311 In the past 12 months, how has your household used its savings? (Critical to make distinction from loans) SELECT ALL THAT APPLY. Enumerator will categorize responses as follows:  Basic/essential household supplies (food, soap, cooking fuel, etc.)  Luxury and non-essential items. (Electronics, jewelry, cigarettes, alcohol)  To recover from shocks/stressors. Examples: Health expenses; replace assets lost in the shock; housing (includes damage repair. Does not include improvements and upgrades to undamaged housing.)  Investment in the future. Examples: Education; housing (Includes improvements and upgrades. Does not include damage repair); agriculture (purchase land, tools, livestock, seed, fertilizer, other inputs); small business  Social and religious ceremonies. Examples: Wedding, burial  Loan repayment. Examples: migration loan or other loan Basic/essential household items A Luxury and non-essential items B To recover from shock C Investment in the future D Social and religious ceremonies E Loan repayment F To fund migration G Other (specify: ________________) X R312 Who primarily decides how savings are used? Respondent 1 Spouse living in household 2 Spouse who has migrated 3 Respondent and spouse jointly 4 Other (specify: ________________) 5 10 Access to Information R400 R401 R402 R403 Did your household receive any information on [topic] within the last year? [READ ALOUD EACH ITEM] 1 Yes 2 No >> Skip to next row 8 DK >> Skip to next row What was your household’s main source of information about [topic]? Use codes below Did the information influence any decisions made by household members? [READ ALOUD EACH ITEM] 1 Yes, result of decision benefitted my HH 2 Yes, result of decision was detrimental to my HH 3 Yes – no positive or negative effect 4 No 1 Early warning for natural hazards (flooding, landslide, hailstorm) 2 Threats to crop health (e.g., pest, disease) 3 Threats to animal health (e.g., disease, epidemic) 4 Rainfall / weather prospects for the coming growing season 5 Long-term changes in weather patterns 6 Disease prevention 7 Safe migration opportunities 8 Methods to improve crop production 9 Methods for improved animal health/husbandry 11 10 Business and investment opportunities 11 Opportunities for borrowing money 12 Available government services near this VDC 13 Natural resource management for this community 14 Equal rights for all ethnic groups 15 Gender equality / gender-based violence CODES FOR R402 - Main Information sources 1 Relatives, friends, neighbors 6 Local market 2 Community leaders 7 Gov’t: rural development agents, health/agriculture extensionists, other gov’t staff/officials 3 VDC 8 NGOs 4 School teachers 9 Newspaper /Radio / TV 5 Group in community (e.g., savings, forest users, farmers) 10 Internet or SMS Livelihood Activities R501 What were the sources of your household’s food/income over the last 12 months? [SELECT ALL THAT APPLY] [PROBE] Farming/crop production and sales A Livestock production and sales B Wage labor (agriculture-crop/livestock) INSIDE COMMUNITY C Wage labor (agriculture crop/livestock) OUTSIDE COMMUNITY D Wage labor (non-agriculture) INSIDE COMMUNITY E Wage labor (non-agriculture) OUTSIDE COMMUNITY F Sale of wild/bush products (e.g., honey, herbs, charcoal) INSIDE COMMUNITY G 12 Sale of wild/bush products (e.g., honey, herbs, charcoal) OUTSIDE COMMUNITY H Other self-employment/own business (agriculture crop/ livestock) INSIDE COMMUNITY I Other self-employment/own business (agriculture crop/ livestock) OUTSIDE COMMUNITY J Other self-employment/own business (non-agriculture) INSIDE COMMUNITY K Other self-employment/own business (non-agriculture) OUTSIDE COMMUNITY L Remittances coming from INSIDE NEPAL M Remittances coming from OUTSIDE NEPAL N Other (specify: __________________) X 13 Group participation R600 R601 R602 Are any of the following groups active in this community? [READ EACH GROUP ALOUD] Which household members participate in this group? [SELECT ALL THAT APPLY] Yes………………1 No………………2 DK………………8 IF R601=2 OR 8, SKIP TO NEXT GROUP Adult male(s).......... A Adult female(s) .......B Male youth .............C Female youth ......... D No one.....................Y 1 Savings / credit cooperative 2 Farmers’/ fishermen’s cooperative 3 Community forest users group 4 Disaster management group 5 Water users group 6 Mutual help group 7 Trade or business associations 8 Village committee 9 Charitable group (helping others) 10 Religious group 11 Political group 12 Mothers group 13 Women’s group 14 Youth group 14 Access to infrastructure and services R701 PLEASE TELL ME WHETHER THESE SERVICES ARE AVAILABLE IN YOUR COMMUNITY: Yes No DK a. Institutions where people can borrow money 1 2 8 b. Institutions where people can save money 1 2 8 c. Primary school within 5 kilometers 1 2 8 d. Health center within 5 kilometers 1 2 8 e. Agricultural extension services 1 2 8 f. Electricity 1 2 8 g. Cell phone service 1 2 8 h. A public telephone 1 2 8 R702 Is your community connected to a market or nearby town by a paved road? 1 2 8 R703 Are there any programs available to this community that help households with food or income when they are faced with a shock? 1 2 8 R704 How far away from this community is the nearest market for selling agricultural products? km R705 How far away from this community is the nearest market for purchasing agricultural inputs? km R706 How far away from this community is the nearest livestock market? km R707 How far away from this community is the nearest market for selling forest products? km 15 Collective action R801 In the past 12 months, have you worked with others in your village to do something for the benefit of the community? Yes 1 No >> Go to Module H 2 R802 What activities did you participate in to benefit the community? [SELECT ALL THAT APPLY] [PROBE] Protecting crop land from flooding A Protecting structures from flooding / landslides B Stopped tree-cutting C Improving access to drinking water D Improving access to electricity E Improving access to health services F Improving road quality G Forming cooperative H Promoting tourism to improve local economy I Other (specify: __________________) X End time: Hour Minute 16 5. Calculation of shock exposure and measures of resilience Throughout this document, after the explanation of each index or indicator calculation, we list the question numbers from the baseline survey and proposed resilience module used for each index (in red print). Those from the baseline household questionnaire are preceded by “BL” and those from the resilience module are preceded by “R”. 5.1. Index of shock exposure A measure of shock/ stressor exposure and severity is created that takes into account households’ exposure to shocks or stressors (out of 10), and the perceived severity of the shock on income and food consumption. Nine of the shock/stressors are listed in question R101. The 10th is designed to capture migration when it manifests as a stressor rather than a positive livelihood change. It is calculated from the response to a question that determines how household income has changed since a migration. If the household replies there have been a “large decrease” or “slight decrease” migration is added as a stressor. Perceived severity is measured using two variables: impact on income security, and impact on food consumption. The variables are based on respondents’ answers to the questions, “In the months following this shock, how severe was the impact on your income?” and “In the months following this shock, how severe was the impact on household food consumption?” which are asked of each shock or stressor experienced. The possible responses are: 1. Remained the same = value of 1. 2. Slight decrease = value of 2. 3. Severe decrease = value of 3. The responses to the two questions are combined into one variable that has a minimum value of 2 and a maximum value of 6. The shock exposure measure is then a weighted average of the incidence of experience of each shock (a variable equal to one if it was experienced and zero otherwise), weighted by the perceived severity of the shock. The shock exposure index ranges from 1 to 81. Survey questions: R101, R105, R106 5.2. Index of cumulative impact of shock exposure The cumulative impact measure is designed to objectively capture the cumulative and interactive impact of all shocks and stressors experienced by a household. It is the sum value of the total number of impacts (out of 17) reported for all shocks experienced. For example, if a household reports “loss of community forest” as a result of flooding/landslides and as the result of forest degradation, and reports no other shocks or impacts, the score for “cumulative impact” is 2. The maximum value for this measure is 153. Survey question: R102 17 5.3. Absorptive capacity index The absorptive capacity index is constructed from eight indicators, some of which are themselves indices. The indicators and explanations of their calculation are as follows. 1. Ability to recover index Ability to recover index is based on estimation of the ability of households to recover from the typical types of shocks that occur in the SABAL and PAHAL areas based on data on the shocks households experienced in the year prior to the survey. Since each survey household did not experience the same types of shocks/ stressors of the same severity, it is necessary to create a “shock exposure corrected” index to measure ability to recover. First, a base “ability to recover” variable is calculated based on responses to two questions: “To what extent has your ability to meet food needs returned to the level it was before the shocks and stressors you experienced in the last 12 months?” With possible responses and weighted values: 1. Ability to meet food needs is the same as before the shocks (= value of 2) 2. Ability to meet food needs is better than before the shocks (= value of 3) 3. Ability to meet food needs is worse than before the shocks (= value of 1) AND “In light of the shocks you faced last year, to what extent do you believe you will be able to meet your food needs in the next year?”, with possible responses and weighted values: 1. Ability to meet food needs will be the same as before the shocks (= value of 2) 2. Ability to meet food needs will be better than before the shocks (= value of 3) 3. Ability to meet food needs will be worse than before the shocks (= value of 1) The responses to the two questions are combined into one variable that has a minimum value of 2 and a maximum value of 6. Next, calculate a shock-exposure-corrected index to create a measure of ability to recover that assumes households experienced the same shock exposure and thus is comparable across them. See the baseline survey report for PRIME Ethiopia for methods.1 Survey questions: R107 and R108. 2. Access to informal safety nets. This indicator is the total number of community organizations potentially providing safety nets that are available in each household’s community. The household￾level information on availability of groups is used to determine availability in the community. Survey question: R601. 3. Bonding social capital index. The bonding social capital index is based on the responses to two questions: one asking whether the household would be able to get help from various categories of people living inside of their community if they need it and one asking whether the household would be able to give help to people in need living inside of their community. The possible responses are 1 Feed the Future. 2014. Ethiopia Pastoralist Areas Resilience Improvement and Market Expansion (PRIME) Project Impact Evaluation. Baseline Survey Report. Volume 1. 18 “relatives”, “non-relatives within my ethnic/caste group”, “non-relatives of other ethnic/caste groups” and “no one”. An additive index ranging from 0 to 6 is calculated based on these responses. Survey questions: R201, R203 4. Whether any household member holds savings. This indicator is a binary (dummy) variable equal to 1 if the respondent reported that a household member regularly saves cash. Survey questions: R309 5. Access to remittances. This indicator is a binary (dummy) variable equal to 1 if the respondent reported that the household receives remittances. Survey questions: R305, R306 6. Asset ownership index. Asset ownership is measured using the number of consumer durables owned out of a total of 15.2 Survey questions: BL H7.02 7. Shock preparedness and mitigation. Summary variable ranging from 0 to 3 summing up the points assigned to the following: - Household lives in a community with a disaster management group (1); - Household reports participating in any of the following activities: protecting crop land from flooding, protecting structures from flooding and landslides, preventing tree-cutting, or improving access to health services (1); - Household reports receiving any of the following types of information in the last year: early warning for natural hazards, disease prevention, natural resource management (1). Survey questions: R601d, R801, R802 R401a,f,m 8. Household has agricultural hazard insurance. This indicator is a binary (dummy) variable equal to 1 if the respondent reported that the household have agricultural hazard insurance. Survey questions: BL G09. Combine the eight indicators described into an absorptive capacity index using polychoric factor analysis. 5.4. Adaptive capacity index The adaptive capacity index is constructed from eight indicators, some of which are indices themselves. The indicators and calculation explanations are as follows. 1. Bridging social capital. The bridging social capital index is based on the responses to two questions: one asking whether the household would be able to get help from various categories of 2 Information on the ownership of productive assets should be included in this index. It is not clear whether this information is being collected in the baseline. 19 people living outside of their community if they need it and one asking whether the household would be able to give help to people in need living outside of their community. The possible responses are “relatives”, “non-relatives within my ethnic/caste group”, “non-relatives of other ethnic/caste groups” and “no one”. An additive index ranging from 0 to 6 is calculated based on these responses. Survey questions: R202, R204 2. Linking social capital. The linking social capital index is based on answers to questions regarding whether household members know a government official and/or NGO leader, how well they know them, and whether they believe the official/leader would help their family or community if help was needed. The index ranges from 0 to 6. Survey questions: R205-R210 3. Human capital. This binary (dummy) variable is equal to 1 if any household adult has a primary or higher education. Survey questions: BL21. 4. Livelihood diversification. The total number of livelihood activities engaged in in the last year. The question asked to identify these livelihoods is “What were the sources of your household’s food/income over the whole last 12 months?” The possible options are: – Farming/crop production and sales; – Livestock production and sales; – Wage labor; – Salaried work; – Sale of wild/bush products (eg., honey, charcoal); – Other self-employment/own business; – Rental of land; – Remittances; – Other. Survey questions: R501. 5. Exposure to information. The number of topics the respondent has received information on in the last year, out of a total of 15. Survey questions: R401a-o 6. Adoption of improved practices. This binary (dummy) variable is equal to 1 if respondents report adopting three or more improved practices for crop production (including vegetables) OR respondents report adopting three or more improved practices for livestock production OR respondents report following one natural resource management practice or technique not related directly to on-farm production OR respondents report using any improved storage method. Survey questions: BL G3b, BL G16, BL G18, BL G21 7. Asset ownership index. See above. 20 8. Access to financial resources. The variable is equal to zero if there is no institution in a household’s community providing credit or savings support, to one if there is one only, and to two if there are institutions that provide both types of support. Survey questions: R701a,b Combine the indicators into an index using polychoric factor analysis. 5.5. Transformative capacity index The transformative capacity index is constructed from nine indicators, some of which are indexes themselves. The indicators and calculation explanations are as follows. 1. Access to formal safety nets. This community-level variable is based on a binary (dummy) variable equal to 1 if the household response to the following question is “yes”: Are there any programs available to this community that help households with food or income when they are faced with a shock? The information from the household-level responses is used to compute a binary (dummy) variable equal to 1 if there are programs available in the community that help with food or income in response to shocks. Survey questions: R703 2. Access to markets. This community-level variable is the number of markets available within 10 kms of the household’s community, as determined from households’ responses to questions asking whether the following markets are available in their community: – Markets for selling agricultural products – Markets for purchasing agricultural inputs – Livestock market – Market for selling forest products. Survey questions: R704-R707 3. Access to basic services. This community-level variable is the number of basic services available in the households’ community, as determined from households’ responses to questions asking whether the following services are available in their community: – A primary school within 5 km – A health center within 5 km – Access to safe drinking water. Survey questions: R701c,d, BL F4-F10 4. Access to infrastructure. This community-level variable is the number of types of infrastructure available in the households’ community, as determined from households’ responses to questions asking whether the following types are available in their community: – Electricity 21 – Cell phone service – Public telephone – Paved road. Survey questions: R701f,g,h, R702 5. Access to agricultural services. This community-level variable is based on a binary (dummy) variable equal to 1 if the household reports that agricultural extension services are available in their community. The information from the household-level responses is used to compute a binary (dummy) variable equal to 1 if agricultural extension services are available in the community. Survey questions: R701e 6. Bridging social capital. See above. 7. Linking social capital. See above. 8. Gender equitable decision-making index. This community-level variable3 is based on binary (dummy) variables created regarding three types of decision-making control within households: control of income, control over use of savings, and control over health and nutrition decisions. The first variable, gender-equitable control of income, uses responses from the first male and female eligible persons from the roster who state they have been paid in “cash only” or “cash and kind” for work done in the past 12 months. Households without a male and female responding to Module J are excluded. The variable is equal to one if male respondents report they participate (solely or jointly) in decisions on how cash they themselves have earned is used AND female respondents also report they participate (solely or jointly) in decisions on how cash they themselves have earned is used. The variable is equal to 0 if either males or females in a household report that “spouse/partner” or “other person” makes this decision. The variable gender-equitable decision-making control over savings is equal to 1 if respondents report that males and females jointly determine how savings will be used. The variable gender-equitable control over health and nutrition decisions uses responses from the first male and female from the household roster who state they have a child under 2 years (K05). Households without a male and female responding “yes” to K05 are excluded. The variable is equal to one if female respondents report they make decisions about their own health and nutrition (response 1 “yourself” is only valid response) AND female respondents also report they participate jointly in decisions about their child’s health and nutrition AND male respondents report they participate jointly in decisions about their child’s health and nutrition. The variable is equal to 0 if all three conditions are not met. 3 This variable cannot be calculated at the household level because all households do not satisfy the conditions for inclusion. For example, not all households have been male and female adults, and not all households have both male and female adults who have earn cash income. After the data are collected, it will become clearer whether the proposed method of measuring gender-equitable decision-making at the community level will be viable in practice. 22 The information from the survey households in each community is used to create the community￾level index as follows: The three dummy variables are employed to calculate the percent of eligible (that is, who the dummy variable can be calculated for) households in each community satisfying the condition for gender-equitable decision making. Following, the mean of the three indexes is used as the indicator of gender-equitable decision making control for each community. Survey questions: BLJ07, J10; BL K05, BL K14, BL K15, R 311 9. Active participation in local decision-making bodies. A binary (dummy) variable equal to one if the respondent reports any household member’s level of participation in any group’s decision-making as “leader”, “very active”, or “somewhat active”. Survey questions: R602, R603 Combine the indicators into a transformative capacity index using polychoric factor analysis. 5.6. Index of household resilience capacity The overall index of resilience capacity is calculated using PCA, with the indexes of absorptive capacity, adaptive capacity, and transformative capacity as inputs. 6. Responses to Shocks and Stresses Program interventions that focus on resilience strengthening should be designed and implemented so that they lead to intermediate outcomes (e.g., strengthened resilience capacity of the target population), which themselves should then lead to an appropriate response outcomes. Fundamentally, resilience interventions are about strengthening the ability of households (or society) to choose – from a whole 'portfolio' of options – what they perceive at that time as the “right” response(s). An appropriate response (e.g., using social capital, accessing savings) increases the chances of positive well-being outcomes, while an inappropriate or ill-chosen one often leads to vulnerability. Resilience analysis should measure the effect of different resilience responses at multiple levels (i.e., households, communities, local, provincial and national authorities). In the context of food security, the Coping Strategies Index (CSI) represents a viable response indicator as it measures the occurrence of specific detrimental coping strategies. However the CSI focuses on short-term consumption-related behavior after a shock or stressor. Other short-term ex-post responses might also be relevant such as these focusing on cash or money-borrowing strategies, easily measured by indicators that capture access to or utilization of financial services (e.g., savings groups, credit). Improved resilience capacity, however, is not simply about avoiding detrimental short-term response strategies that undermine absorptive capacity in particular. It is also about nurturing or fostering the ability of actors to engage in positive and sustainable responses that improve all three resilience capacities, i.e., absorptive, adaptive, and transformative capacity. Thus a reduction in the adoption of detrimental coping strategies (i.e., a lower CSI) might serve as one universal indicator in resilience programs for improving absorptive responses. However, resilience response indicators should also measure changes in adaptive and transformative behavior. These responses have to be understood in relation to the specific social and ecological contexts and constraints within which these households are operating. 23 Absorptive Responses Coping strategy index (BL C22-C37) Use of savings to absorb shocks (R 310) Use of remittances to absorb shocks (R 308) Use of hazard insurance (BL GO9) Use of bonding social capital to absorb shock (R 211, 213) Transformative Responses Active participation in decision making (R 602,603) Participation in collective action (R 802) Gender equitable decision making index (BLJ07, J10; BL K05, BL K14, BL K15, R 311) Use of formal safety nets (R703) Adaptive Responses Application of information (R 403) Adoption of Improved Agricultural Practices (BL G3b, BL G16, BL G18, BL G21) Use of savings for adaptation ( R310) Use of remittances for adaptation (R 308) Use of bridging social capital (R 212) 24 7. Suggested research questions for qualitative component 1. Background information What kind of shocks and stressors is the community experiencing now? 2. Shock / stressor In what ways are these shocks/ stressors affecting the entire community? 3. Community response What actions are members of the community taking to support each other to respond to the shock/ stressor? Which are successful? Which are less successful? 5. Participation Are community leaders effective at organizing support for all members of the community? Why or why not? What collective action is the community taking to protect or maintain resources important to the whole community? Which resources and why? 4. Social rules, norms (caste, gender, other) and cultural attitudes How do social rules and norms and cultural attitudes affect the resilience capacities of different groups?  Savings, investment, decision-making control, disaster preparedness, participation, involvement in community action; access to services, etc.