Purchase Order No. AID-685-O-14-00029 June 9, 2014 This publication was produced at the request of the United States Agency for International Development (USAID). It was prepared independently by Chemonics International. The author’s views expressed in this publication do not necessarily reflect the views of USAID or the U.S. government. Yaajeende Agricultural Development Project Mid-Term Performance Evaluation Contents ACRONYMS....................................................................................................................................................................................... I EXECUTIVE SUMMARY .....................................................................................................................................................................II BACKGROUND ................................................................................................................................................................................. 1 General Background.................................................................................................................................................................... 1 USAID|Yaajeende’s Results Framework........................................................................................................................................... 2 Development Hypothesis............................................................................................................................................................ 2 Major Activities ............................................................................................................................................................................ 4 Critical Assumptions ................................................................................................................................................................... 5 EVALUATION PURPOSE .................................................................................................................................................................... 5 EVALUATION QUESTIONS ................................................................................................................................................................ 6 EVALUATION METHODS AND LIMITATIONS ..................................................................................................................................... 7 Review and Analysis of Relevant Documents........................................................................................................................... 7 Data Collection, Management, and Analysis............................................................................................................................ 7 Limitations of the Analysis.......................................................................................................................................................... 8 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS ..................................................................................................................... 9 Question 1: Has the project demonstrated effective, efficient, and sustainable vehicles/approaches for promoting adoption of innovation (technology, practices, behaviors) and diffusion of products and new technologies among the poor, women, and socially marginalized? ................................................................................................................................. 9 FINDINGS ............................................................................................................................................................................................ 9 CONCLUSION ..................................................................................................................................................................................19 RECOMMENDATION.......................................................................................................................................................................19 Question 2: Has the CBSP model proven to be an effective and sustainable private sector-driven approach to reduce undernutrition in targeted areas?...............................................................................................................................20 FINDINGS ..........................................................................................................................................................................................20 CONCLUSION ..................................................................................................................................................................................23 RECOMMENDATION.......................................................................................................................................................................24 Question 3: Have the activities that increase household assets and income led to improved participation of the rural poor in rural growth?................................................................................................................................................................. 24 FINDINGS ..........................................................................................................................................................................................24 CONCLUSION ..................................................................................................................................................................................26 RECOMMENDATION.......................................................................................................................................................................26 Question 4: What project activities have positively enabled value-chain investments to lead to improved consumption of diverse diets and quality foods? ................................................................................................................................................................. 26 FINDINGS ..........................................................................................................................................................................................26 CONCLUSION ..................................................................................................................................................................................30 RECOMMENDATIONS .....................................................................................................................................................................31 Question 5: What investments in human and institutional capacity development have effectively generated large-scale nutrition outcomes? .....................................................................................................................................................................................31 FINDINGS ..........................................................................................................................................................................................31 CONCLUSION ..................................................................................................................................................................................36 RECOMMENDATION.......................................................................................................................................................................36 Question 6: To what extent has the integrated nutrition and agriculture approach led to a reduction of undernutrition among the target population? .........................................................................................................................................................................36 FINDINGS ..........................................................................................................................................................................................36 CONCLUSION ..................................................................................................................................................................................39 RECOMMENDATION.......................................................................................................................................................................39 Question 7: To what extent has the project’s water and sanitation activities led to improved healthy behaviors of the target population? ...................................................................................................................................................................................39 FINDINGS ..........................................................................................................................................................................................39 CONCLUSION ..................................................................................................................................................................................41 RECOMMENDATION.......................................................................................................................................................................42 Question 8: To what extent has the project been implemented effectively, including timely completion of project activities, effective use of project resources, reach of target groups/beneficiaries, quality of partnerships and collaboration, and contribution to overall USAID/Senegal economic growth objective goals?......................................42 FINDINGS ..........................................................................................................................................................................................42 CONCLUSION ..................................................................................................................................................................................45 RECOMMENDATION.......................................................................................................................................................................45 Question 9: What is the likelihood that project approaches/practices and results will be sustained?...........................46 FINDINGS ..........................................................................................................................................................................................46 CONCLUSION ..................................................................................................................................................................................48 RECOMMENDATION....................................................................................................................................................................... 48 Question 10: What are the outcomes of the project’s approach to addressing gender, environmental compliance, and governance issues?..............................................................................................................................................................49 FINDINGS ..........................................................................................................................................................................................49 CONCLUSION ..................................................................................................................................................................................53 RECOMMENDATION.......................................................................................................................................................................53 LESSONS LEARNED .........................................................................................................................................................................54 ANNEX I: EVALUATION STATEMENT OF WORK ...............................................................................................................................57 ANNEX II: QUALITATIVE DATA COLLECTION INSTRUMENT .............................................................................................................64 ANNEX III: BIBLIOGRAPHY OF DOCUMENTS REVIEWED ..................................................................................................................70 ANNEX IV. LIST OF PERSONS INTERVIEWED ....................................................................................................................................74 ANNEX V. EVALUATION DESIGN MATRIX .......................................................................................................................................78 i ACRONYMS BCC behavior change communications BDS business development services CBSP community-based solution provider CWG citizen working group DRDR Direction Régionale du Développement Rural EIA Environmental Impact Assessment FtF Feed the Future GHI Global Hunger Index HICD Human and Institutional Capacity Development IEE Initial Environmental Examination IFPRI International Food Policy Research Institute ISRA Institut Sénégalais de Recherches Agricoles IYCF Infant and Young Child Feeding M&E monitoring and evaluation MOU memorandum of understanding PMP performance management plan SAED Société Nationale d'Aménagement et d'Exploitation des Terres du Delta du Fleuve Sénégal UNICEF United Nations Children's Fund USAID United States Agency for International Development WASH water and sanitation for health ii EXECUTIVE SUMMARY BACKGROUND USAID|Yaajeende1 is a five-year Feed the Future (FtF) project designed to reduce malnutrition in the Matam and Kédougou regions as well as the Department of Bakel — an area representing the northeastern one￾third of Senegal. The project is implemented by NCBA/CLUSA (prime awardee) and Counterpart International, Heifer International, and Sheladia Associates (sub-awardees). The project’s goal is to accelerate the participation of the very poor in rural economic growth and to improve the population’s nutritional status. The development hypothesis of USAID|Yaajeende, one of USAID’s first FtF projects, is that an integrated approach to agriculture, economic growth, and nutrition can lower the rate of undernutrition much more rapidly than by focusing on agriculture or economic growth alone. USAID|Yaajeende activities are divided into five major areas: increasing the availability of food by improving the diversity and sustainability of agricultural production and promoting sustainable land management; increasing and diversifying revenues from agriculture by stimulating key agricultural markets and value chains; reducing undernutrition and ensuring a healthy diet through improved utilization of foods; improving capacity for local governance of food-related resources; and cross-cutting activities, such as expanding the role of women in food security and nutrition. EVALUATION PURPOSE The purpose of this evaluation, conducted after approximately 35 months of project implementation (November 2010 to September 2013), is to assess progress to date and identify improvements that will facilitate the attainment of planned project results. Of particular importance is whether mid-term results have supported the underlying development hypothesis that an integrated approach to agriculture, economic growth, and nutrition can reduce undernutrition more rapidly than a focus on agriculture or economic growth alone. It is expected that identification of good practices and lessons learned will serve as a basis for providing recommendations to maintain momentum, scale up activities, and make strategic modifications that will guide future government of Senegal and USAID programming. EVALUATION QUESTIONS Component 1: Accelerate the Participation of the Rural Poor in Rural Growth 1. Has the project demonstrated effective, efficient, and sustainable vehicles/approaches for promoting adoption of innovation (technology, practices, behaviors) and diffusion of products and new technologies among the poor, women, and socially marginalized? 2. Has the CBSP (community-based solution provider) model proven to be an effective and sustainable private sector-driven approach to reduce undernutrition in targeted areas? 3. Have the activities to increase household assets and income led to improved participation of the rural poor in rural growth? 1 Yaajeende means prosperity, abundance, and generosity in bestowing gifts upon others in the Pulaar language. iii Component 2: Reduce Undernutrition 4. What project activities have positively enabled value-chain investments to lead to improved consumption of diverse diets and quality foods? 5. What investments in human and institutional capacity development have effectively generated large￾scale nutrition outcomes? 6. To what extent has the integrated nutrition and agriculture approach led to the reduction of undernutrition among the target population? 7. To what extent has the project’s water and sanitation activities led to improved healthy behaviors of the target population? Cross-Cutting Questions 8. To what extent has the project been implemented effectively, including timely completion of project activities, effective use of project resources, reach of target groups/beneficiaries, quality of partnerships and collaboration, and contribution to overall USAID/Senegal economic growth objective goals? 9. What is the likelihood that project approaches/practices and results will be sustained? 10. What are the outcomes of the project’s approach to address gender, environmental compliance, and governance issues? EVALUATION METHODS AND LIMITATIONS The evaluation team used a mixed data-collection approach, including a thorough document review, direct observations, key informant interviews, and focus-group discussions. The team conducted interviews with representatives from USAID/Senegal and USAID|Yaajeende; government officials at the central, regional, and local levels; and other stakeholders in Dakar and at project sites. The team also visited a cross-section of USAID|Yaajeende’s activity sites to understand the project’s technical assistance approach and beneficiary experience. In addition, the team conducted focus-group discussions with producer groups, community￾based solution providers, community nutrition volunteers, women’s groups, agro-entrepreneurs, and project beneficiaries, including mothers of malnourished children and poor rural farmers. In total, the team held 264 focus-group discussions and conducted 40 key informant interviews. Since this evaluation was designed as a performance evaluation, the team did not perform causation or attribution analysis. Causal inference between the project and its outcomes was limited to statements of “plausible contributions,” not as conclusions that project interventions were the major or only cause of observed results. The evaluation relied in part on secondary data, including performance and survey results that the project has generated. It also relied on a qualitative survey using rapid-appraisal methods based on direct observations, key informant interviews, and focus-group discussions. Although such methods are widely used in analyzing project performance, they have several limitations, including informal sampling procedures, a lack of unambiguous validation techniques to test answers in the survey tools, and researchers’ inability to go beyond what is reported by informants. iv MAJOR FINDINGS Mid-way through its project cycle, USAID|Yaajeende has been highly successful in reaching its target groups and beneficiaries: the poor and the vulnerable, especially women. USAID|Yaajeende’s achievements are universally recognized by government officials at the central, regional, and local levels in Senegal, as well as by the private sector, civil-society organizations, and beneficiary households. USAID|Yaajeende’s success is evidenced by communities’ and beneficiaries’ acceptance of project activities — there is consensus among the 40 key interviewees and 264 focus-group participants that project activities are highly relevant to the development issues that beneficiary communities face. USAID|Yaajeende’s contributions to USAID/Senegal’s economic growth objective goals are considerable, and so is its relevance to the Senegalese government’s national strategy for economic and social development. This strategy stresses food security; agricultural and private sector development; inclusive development to improve the nutritional status of mothers and children via dietary diversity and by meeting the nutritional needs of vulnerable groups (under-five children, pregnant women, and nursing mothers); building the capacity of institutions and local government units; and strengthening the leadership and entrepreneurial skills of women. Gender integration has been a central pillar of project interventions. USAID|Yaajeende has broadened women’s equitable participation in project interventions and developed equalizing strategies to promote increased access to resources and opportunities for women. USAID|Yaajeende’s partnerships and collaboration are of the highest quality, enabling the project to introduce new products, technologies, and innovations to strengthen food security in its intervention zones. The project’s diverse approaches to promoting adoption and dissemination of new technologies, practices, and behaviors have been equally effective. The effectiveness of the CBSP network, a private sector-driven model consisting of entrepreneurs who are motivated by the desire for financial success, is illustrated by its members’ increasingly strong commercial ties with suppliers and their success in offering an increasingly wide range of products and services to their communities. The emerging transition of the community nutrition volunteer network into community-based solution providers is another sign of project success. USAID|Yaajeende has used three major mechanisms to increase household assets and income for the rural poor: the pass-on-the-gift program, bio-reclamation of degraded land, and dissemination of innovations and efficient farming techniques among smallholders. The pass-on-the-gift program and the reclamation of degraded land have increased household assets significantly, and higher horticultural yields have enabled smallholders to market most of their production and raise household revenue. Key informant interviews and focus-group discussions reveal that project activities have resulted in more diversified diets and consumption of higher-quality foods in project areas. Similarly, non-project survey data, as well as qualitative information gathered in key informant interviews and focus-group discussions, clearly indicate that the nutrition situation has improved in USAID|Yaajeende’s intervention zones. However, the two conclusions cannot be corroborated with quantitative data, in part because the mid-term survey scheduled for mid-2013 has yet to be conducted. In the absence of reliable time-series data on dietary v diversity in USAID|Yaajeende’s interventions and control areas, it is not possible to assess whether project activities have contributed to improved food consumption in the target area and whether improvements can be attributed to project activities. Nor is it possible, without impact evaluation data, to rigorously investigate the extent to which the integrated nutrition and agriculture approach has reduced undernutrition among the target population. The mid-term survey scheduled for the third quarter of Year 4 will fill these data gaps. Sustainability is achieved when host-country partners and beneficiaries maintain results beyond the life of a project. USAID|Yaajeende has paid special attention to project sustainability from the outset. Ample qualitative evidence and quantitative results show that many of the project’s agricultural and nutrition activities are likely to have lasting effects. However, USAID|Yaajeende did not elevate sustainability to the results-framework level, nor did it develop a comprehensive and clearly articulated sustainability plan from the start. It also did not develop adequate and direct sustainability indicators to monitor and evaluate progress in a more rigorous way. The number of USAID|Yaajeende’s performance indicators was reduced from 52 in Year 1 to 47 in Year 2 to 42 in Year 3. At 42, the number of indicators places a heavy burden on the project, especially when combined with a variety of process indicators — a finding emphasized by USAID|Yaajeende’s staff in the field. In addition, more than 40 percent of those indicators consist of output indicators. Nearly 50 percent consist of output indicators when the four impact indicators used in the baseline survey are excluded. After three years of limited activity and unclear, undocumented results due to funding limitations and the inherent difficulties associated with water and sanitation for health (WASH) activities in USAID|Yaajeende’s intervention zones, the WASH program continues to face severe constraints with likely implications on project sustainability. MAJOR RECOMMENDATIONS A follow-up survey to compare current undernutrition indicators with their baseline values is necessary to assess with minimum ambiguity whether (and the extent to which) USAID|Yaajeende’s value-chain investments have led to dietary diversity and consumption of high-quality foods. It is also necessary to assess whether (and to what extent) human and institutional development has generated large-scale nutrition outcomes. Given the one-year delay in the mid-term survey’s implementation, and given that the project is scheduled to end in 2015, it may be more cost-effective to conduct the survey in the beginning of Year 5. Unless the project is extended, it would be wasteful to collect two sets of the same impact evaluation data less than a year apart. USAID|Yaajeende should formulate a comprehensive and more rigorous monitoring and evaluation (M&E) mechanism to assess the sustainability of the CBSP network and the citizen working groups. Such a mechanism would feature an adequate number of sustainability indicators, clearly defined graduation benchmarks, phase-out plans, and focused evaluation reports. Since service provision and governance are two central components of project interventions in agriculture and nutrition, the performance management plan (PMP) should include a limited number of sustainability indicators to monitor outcomes. vi USAID|Yaajeende should strengthen its collaboration with USAID’s Community Health Program and other partners involved in WASH activities in its intervention zones, such as the government of Senegal’s Millennium Water and Sanitation Program. Such collaboration would enable Yaajeende to play a supporting role, rather than a leading role, through governance program activities. Just as the results framework should consist of a set of necessary and sufficient conditions to achieve the overall project objective, the PMP should contain no more than the necessary and sufficient indicators to measure project achievements. USAID|Yaajeende’s performance indicators should be anchored in higher￾level strategic thinking about what must be achieved for project success. The overall set of indicators should be streamlined with a view to discarding those that may not be needed. Following USAID guidelines, these indicators should be adequate, direct, and cost-effective. USAID|Yaajeende should develop a comprehensive and systematic stand-alone sustainability plan. Experience from other countries demonstrates that, to achieve optimal results, sustainability should be an integral part of project design and should be embedded throughout implementation to withdrawal. In particular, private sector organizations, community organizations, and other stakeholders should be aware of their post-exit roles and responsibilities from the outset. The sustainability plan should include decisions about approach (phase-out, gradual phase-over), explicit benchmarks for progress and timelines, allocation of responsibilities, graduation criteria and progressive withdrawal of project support, and a focus on building the capacity of community and government organizations to progressively take up the management and provision of project services. It is understood that not all activities are expected to be fully sustainable at project conclusion. When such is the case, the sustainability plan should define the degree of sustainability considered essential to the success of those activities. The objective of the sustainability plan is to spell out what results will continue and how sustainability will be targeted and measured. It should describe the process through which the movement toward sustainability will occur, and specify the ways in which required early and intermediate outcomes related to achieving the next higher order of change will be brought about and documented. The sustainability plan should contain explicit benchmarks for progress and timelines, and a set of performance indicators to measure the stated results. 1 BACKGROUND General Background The food security situation in Senegal in 2013, illustrated by the Global Hunger Index (GHI) score of 13.8, was characterized by the International Food Policy Institute (IFPRI) as “serious.” 2 This situation is all the more serious in that, at 13.7 in 2005, the country’s GHI score has remained unchanged for nearly a decade. Food insecurity in Senegal affects between 15.6 and 24 percent of the population, with higher numbers concentrated in the northern and eastern rural regions.3 According to the Senegal National Food Security Plan, agricultural development in the country faces many challenges: low soil fertility, limited use of agricultural inputs and access to agricultural credit, limited availability of good-quality seeds, obsolete farming equipment, inadequate storage and processing infrastructure, under-developed marketing networks, and monopolies within key agricultural value chains. Senegal’s Lettre de Politique de Développement de la Nutrition outlines additional constraints, such as insufficient food availability and access, lack of labeling and certification of food quality, lack of nutrition technical specialists, limited involvement of communities in developing nutrition projects, absence of a nutritional information system, and socio-cultural factors affecting traditional nutritional practices. The government of Senegal has taken major steps to address these constraints. Its National Food Security Plan, Lettre de Politique de Développement de la Nutrition, and Country Investment Plan for Agriculture reflects its commitment to alleviating poverty and reducing hunger. USAID/Senegal supports the Senegalese government’s efforts to alleviate constraints through FtF, a strategy based on the development hypothesis that poverty and hunger can be sustainably reduced by stimulating the national agriculture sector and enhancing the nutritional status of the population, especially women and children. USAID|Yaajeende was designed to support those efforts; its major features are summarized below. Yaajeende Project General Information Project Name USAID/Senegal Yaajeende Agricultural Development Project Cooperative Agreement Number 685-A-00-10-00002-00 Period of Agreement November 1, 2010 – September 30, 2015 Funding $39,999,065 Implementing Organization Cooperative League of the USA/National Cooperative Business Association Agreement Officer’s Representative Papa Nouhine Dieye, USAID/Senegal’s Agricultural Specialist 2 The GHI is a tool designed to comprehensively measure and track hunger globally, by region, and by country. It combines three equally weighted indicators into one index consisting of: (1) under-nourishment -- the proportion of undernourished people as a percentage of the population; (2) child underweight -- the proportion of children younger than age five who are underweight; and (3) child mortality -- the mortality rate of children younger than age five (IFPRI 2013). 3 This section draws heavily on the scope of work and the program description. 2 USAID|Yaajeende’s Results Framework Yaajeende is a five-year FtF food security project designed to reduce malnutrition in the Matam and Kédougou regions as well as the Department of Bakel — an area representing the northeastern one-third of Senegal. The project’s goal is to accelerate the participation of the very poor in rural economic growth and to improve the population’s nutritional status. As illustrated in Figure 1 (next page), USAID|Yaajeende is an integral part of FtF. It addresses three of USAID’s economic growth results framework first-level objectives: inclusive agriculture sector growth; improved nutritional status, especially of women and children; and improved management of natural resources.4 Development Hypothesis USAID|Yaajeende’s development hypothesis is that an integrated approach to agriculture, economic growth, and nutrition can lower the rate of undernutrition much more rapidly than focusing on agriculture or economic growth alone. Greater food security will be achieved as rural populations shift from subsistence agriculture to commercial agriculture via thriving small agro-enterprises and participation in dynamic markets and value chains; use more nutritional and fortified foods and adopt best practices; and work with local governments and citizen groups to govern food and food-related resources in a transparent, participative way. USAID|Yaajeende’s agriculture interventions focus on helping emerging and ultra-poor farmers in rural communities to organize into producer groups or producer organizations, enabling them to take advantage of economies of scale and gain access to new skills, technologies, and financial resources. The community￾based solution providers serve as a private sector-driven extension service. They help producer groups and producer organizations establish linkages to important regional and national actors and leading firms within key value chains. Over the course of the project, farmers are expected to produce more food for local consumption and become stakeholders in dynamic new agro-enterprises that improve household revenue for owners and workers, leading to increased availability and access to food. USAID|Yaajeende’s nutrition interventions focus on increasing public demand for nutritious foods through education, training, and behavior change communications (BCC), and on improving the supply of diverse, nutritious foods, including fortified foods, through local agents. USAID|Yaajeende works to strengthen the existing network of community nutrition volunteers set up by the Nutrition Enhanced Program. These volunteers provide information to mothers and children on nutrition best practices, guide producer groups and producer organizations on what commodities to grow, and link people to public nutrition services. 4 First-level objective 2 (increased trade) is not directly addressed by USAID|Yaajeende, an initiative that focuses on poor, food￾deficit areas, because production in those areas is consumed locally or marketed within Senegal. 3 USAID/Senegal Economic Growth Development Objective: Increased Inclusive Economic Growth FtF Goal: Sustainably Reduce Global Poverty and Hunger Level 1, Objective 1: Inclusive Agriculture Sector Growth Level 1, Objective 3: Improved Nutritional Status, Especially of Women and Children Level 1, Objective 4: Improved Management of Natural Resources IR 1.1: Improved Agricultural Productivity IR 1.2: Improved Rural Infrastructure IR 1.3: Improved Access to Finance IR 1.4: Improved Markets Sub‐IR 1.1: Increased agriculture value chain and off‐farm jobs Sub‐IR 1.1.2: Increased private sector investment in agriculture Sub‐IR 1.2.1: Off‐farm revenues diversified Sub‐IR 1.2.2: Local warehouse capacities strengthened Sub‐IR 1.2.3: Irrigation expanded and improved Sub‐IR 1.3.1: Agribusiness and BDS enhanced Sub‐IR 1.4.1: Role of farmer organizations strengthened Sub‐IR 1.4.2: Agricultural revenues diversified IR 3.1: Bring to Scale Essential Nutrition Actions Sub‐IR 3.1.1: Improved access to diverse and quality foods Sub‐IR 3.1.2: Improved health and nutrition behaviors Sub‐IR 3.1.3: Improved use of maternal and child health and nutrition services IR3.2: Integrate nutrition into the value‐  chain approach Sub‐IR 3.2.1: Increased private sector investment in nutrition IR 4.1: Improved Water Management IR 4.2: Improved Conservation of Bio‐ diverse Areas IR 4.3: Increased Ability to Adapt to Climate Change Figure 1. USAID|Yaajeende Abridged Results Framework 4 Major Activities Nutrition-led agriculture is at the core of USAID|Yaajeende’s approach. It is built on the notion that agriculture has the potential to do more than contribute to basic food and income needs; it can also improve nutrition and health. USAID|Yaajeende’s activities can be divided into five major areas as follows: 1. Increase the availability of food by improving the diversity and sustainability of agricultural production and by promoting sustainable land management  Promote resilient farming systems, including conservation agriculture, to improve soil health  Restore degraded lands  Increase rainfed, irrigated, and flood recession crop productivity  Improve water resource management and promote effective small irrigation technologies  Intensify and diversify agricultural systems through improved market gardening  Increase livestock assets via animal placements and increase the availability of livestock products  Introduce technologies and techniques to increase the production of highly nutritious crops, including biofortified crops 2. Increase and diversify revenues from agriculture by stimulating key agricultural markets and value chains  Map proximity of vulnerable populations to markets and value chains  Strengthen the producer organization sector  Enhance provision of services in rural areas by establishing a CBSP network  Promote agribusiness and enhance business development services in target zones  Increase revenues by facilitating market linkages within key value chains  Help beneficiaries diversify agriculture-related income by creating off-farm agribusinesses  Help transition community nutrition volunteers to community-based solution providers focusing on sales of nutrition oriented products and services  Create a voucher system for the ultra-poor 3. Reduce undernutrition and ensure a healthy diet through improved utilization of foods  Conduct nutritional gap analysis  Promote micronutrient rich vegetable gardening in households to expand consumption of vegetables  Strengthen the storage and processing of foods to expand year-round access to nutritious foods  Increase social marketing and BCC to encourage healthy behaviors and build public demand for diverse and nutritious foods  Improve sanitation and expand access to clean drinking water 5 4. Improve capacity for local governance of food-related resources  Establish multi-sectoral institutional food security framework  Develop community land-use plans  Enhance civil society’s ability to work with government to manage food security issues  Strengthen government-citizen collaboration on agriculture, food security, and nutrition issues  Strengthen the capacity of government solution providers to deliver nutrition products and services and to upgrade nutrition health services and policy  Improve local government capacity to manage and monitor food security at the local/regional level 5. Cross-cutting activities  Expand the role of women in food security and nutrition  Improve management and monitoring of food security through innovative and effective knowledge management and metrics  Support innovative local initiatives via a grants and enterprise fund Critical Assumptions The first critical assumption of USAID|Yaajeende is that poor farmers and livestock producers are willing to adopt new technologies. Although the project intends to assume some of the risk associated with new technologies and provide technical assistance and training, it will not fully subsidize any activity. Subsidies are not sustainable, do not allow scaling up, and do not promote client ownership. For irrigation, potable water, and livestock watering schemes, USAID|Yaajeende assumes the ground and surface water situation will remain relatively stable, or at worst, trend gradually toward greater scarcity. Although allowance will be made for climate change in the planning and design of project activities, the effects of climate change on limited water supplies in semi-arid environments are not yet known. If water resources deteriorate rapidly in some areas, project investments may not yield their expected results. EVALUATION PURPOSE The purpose of this mid-term evaluation, conducted after approximately 35 months of project implementation (November 2010 to September 2013), is to assess progress to date and identify improvements that will facilitate the attainment of planned project results. To this end, the evaluation assesses the relevance of the two project components to Senegal’s FtF strategy; soundness of project approaches; overall quality of project management; relevance, timeliness, and cost-effectiveness of service delivery; beneficiary coverage and response; and overall sustainability of project results beyond September 2015. 6 Of particular importance is whether mid-term results have supported the underlying development hypothesis that an integrated approach to agriculture, economic growth, and nutrition can reduce undernutrition more rapidly than a focus on agriculture or economic growth alone. It is expected that identification of good practices and lessons learned will serve as a basis for providing recommendations to maintain momentum, scale up activities, and make strategic modifications that will guide future government of Senegal and USAID programming. EVALUATIONQUESTIONS Component 1: Accelerate the Participation of the Rural Poor in Rural Growth 1. Has the project demonstrated effective, efficient, and sustainable vehicles/approaches for promoting adoption of innovation (technology, practices, behaviors) and diffusion of products and new technologies among the poor, women, and socially marginalized? 2. Has the CBSP model proven to be an effective and sustainable private sector-driven approach to reduce undernutrition in targeted areas? 3. Have the activities to increase household assets and income led to improved participation of the rural poor in rural growth? Component 2: Reduce Undernutrition 4. What project activities have positively enabled value-chain investments to lead to improved consumption of diverse diets and quality foods? 5. What investments in human and institutional capacity development have generated large-scale nutrition outcomes? 6. To what extent has the integrated nutrition and agriculture approach led to the reduction of undernutrition among the target population? 7. To what extent has the project’s water and sanitation activities led to improved healthy behaviors of the target population? Cross-Cutting Questions 8. To what extent has the project been implemented effectively, including timely completion of project activities, effective use of project resources, reach of target groups/beneficiaries, quality of partnerships and collaboration, and contribution to overall USAID/Senegal economic growth objective goals? 9. What is the likelihood that project approaches/practices and results will be sustained? 7 10. What are the outcomes of the project’s approach to address gender, environmental compliance, and governance issues? EVALUATION METHODS AND LIMITATIONS The evaluation was conducted by a multidisciplinary team consisting of Bechir Rassas, agricultural and nutrition economist and team leader; Malamine Savané, agriculture specialist; Valerie Quenum Ndiaye, nutrition specialist; and Ya Cor Ndione, economist. To address the evaluation questions, the evaluation team used a mixed-data collection approach, including a thorough document review, direct observations, key informant interviews, and focus-group discussions. Review and Analysis of Relevant Documents The evaluation team began with a thorough review of qualitative and quantitative data on project performance in key documents, such as USAID/Senegal’s FtF strategy; Senegal’s Agriculture Country Investment Plan; Stratégie nationale de sécurité alimentaire du Sénégal; USAID|Yaajeende agreement; USAID|Yaajeende annual work plans, annual and quarterly reports, and PMPs; sector action plans and reports; training manuals; field trip reports; and other relevant documents. Data Collection, Management, and Analysis The evaluation team collected data for this evaluation during the document review phase. USAID|Yaajeende’s PMP, a tool used by the project team to assess and report on progress toward achieving development objectives, and associated survey data were major sources of data. USAID|Yaajeende’s data included input, activity, and output monitoring data; a rapid village-level reconnaissance survey; a household, producer-organization, and enterprise baseline survey; an annual household, producer-organization, and enterprise survey; case studies; and focused evaluations. The team collected qualitative data through direct observations, key informant interviews, and focus-group discussions, using interview guides that reflected the evaluation questions. The team conducted interviews with representatives from USAID/Senegal, USAID|Yaajeende, and the Senegalese Ministry of Agriculture (Institut Sénégalais de Recherche Agricole, Institut de Technologie Alimentaire, and other offices), the Senegalese Ministry of Health (Nutrition Enhancement Program and the Cellule de Lutte contre la Malnutrition), and other stakeholders in Dakar and at project sites. The evaluation team visited a cross-section of USAID|Yaajeende’s priority activity sites to gain a more in￾depth understanding of the project’s technical assistance approach and beneficiary experience. The team conducted focus-group discussions with producer groups, community-based solution providers, community nutrition volunteers, women’s groups, agro-entrepreneurs, and project beneficiaries, including mothers of malnourished children and poor rural farmers. (The guides for key informant interviews and focus-group discussions are provided as an annex to this evaluation.) 8 Sampling methodologies for selecting interview and focus-group participants varied. The team attempted to collect data from most of the key stakeholders, using chain or “snowball” sampling to identify additional interviewees as needed and asking key stakeholders to identify others who would enrich the information base. For other informants, the team applied heterogeneity sampling to include diverse audiences who could answer questions such as, “Have USAID|Yaajeende’s outreach efforts been effective in changing the public’s understanding of prevailing practices related to water, sanitation, and hygiene?” and “Have USAID|Yaajeende’s activities changed mothers’ understanding of their children’s nutritional needs?” The evaluation team worked with USAID|Yaajeende staff to identify a wide variety of focus groups representing a cross-section of project beneficiaries, including community-based solution providers, community nutrition volunteers, citizen working groups, and mother-to-mother groups. Qualitative data were used to interpret and shed light on the collected quantitative data. (See Table 1 for the number and location of key informant interviews and focus-group discussions.) Table 1. Key Informant Interviews and Focus-Group Discussions Number of Persons Contacted by Geographic Location Location Key Informant Interviews Focus-Group Discussions Men Women Total Men Women Total Dakar 12 2 14 0 0 0 Bakel 5 0 5 29 35 64 Matam 13 1 14 38 88 126 Kédougou 7 0 7 5 69 74 Total by Category of Respondents 37 3 40 72 192 264 Average Number of Respondents in Focus Groups 3 8 11  Total number of persons contacted in key informant interviews and focus group discussions: 304  Number of key informant interviews: 40  Number of focus group discussions: Bakel Matam Kédougou Total 7 11 5 23 Limitations of the Analysis Since this evaluation was designed as a performance evaluation, the evaluation team did not perform causation or attribution analysis. Causal inference between the project and its outcomes was limited to statements of “plausible contributions,” not as conclusions that project interventions were the major or only cause of observed results. This evaluation relied in part on secondary data, including performance and survey results generated by the project. It also relied on a qualitative survey using rapid-appraisal methods based on direct observations, key informant interviews, and focus–group discussions. Although such methods are widely used in analyzing project performance, they have several limitations, such as informal sampling, which can lead to imperfectly representative samples; a lack of unambiguous validation procedures to test respondents’ 9 answers; researchers’ inability to go beyond what is reported by informants; informants’ individual biases; evaluators’ individual biases; and difficulties that often occur in low-resource settings in logistical recording, coding, and analyzing of qualitative data. To lessen bias and strengthen validity of findings, the evaluation team used multiple data collection and analysis methods so that data collected using one method could be compared to information collected by other means. The team used direct observation, key informant interviews, and focus-group discussions to supplement a thorough document review, project surveys, performance monitoring data, and statistics collected by government institutions and bilateral and multilateral donors. To collect qualitative data, the evaluation team used a large sample of key informants and focus-group participants of more than 300 respondents. They selected informants for representativeness, credibility, knowledge, and diverse viewpoints, and used probing techniques to encourage informants to give fuller, clearer responses and to detail the basis for their views, conclusions, and recommendations. For focus￾group discussions, the team used similar sampling and probing methods to ensure no relevant groups were overlooked and no perspectives discounted. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS Question 1: Has the project demonstrated effective, efficient, and sustainable vehicles/approaches for promoting adoption of innovation (technology, practices, behaviors) and diffusion of products and new technologies among the poor, women, and socially marginalized? FINDINGS USAID|Yaajeende has introduced a wide variety of new products, technologies, and innovations to strengthen food security in its intervention zones. USAID|Yaajeende is one of the major components in USAID/Senegal’s program that supports the Senegalese government’s efforts to strengthen national food security. This initiative is part of the broader FtF program, a USAID strategy based on the development hypothesis that poverty and hunger can be sustainably reduced by stimulating the national agriculture sector and by enhancing the nutritional status of the population, especially of women and children. Nutrition-led agriculture has been the cornerstone of USAID|Yaajeende’s strategy to strengthen food security in its intervention zones. Underlying USAID|Yaajeende’s intervention design and implementation is the assumption that an integrated approach to agriculture, economic growth, and nutrition can lower the rate of undernutrition much more rapidly than focusing on agricultural and economic growth alone. In particular, greater food security is achieved as rural populations shift from subsistence to commercial agriculture that increases the availability of, accesses to, and utilization of highly nutritious foods. This can 10 be achieved through production and consumption of nutrient-rich crops (horticultural and biofortified crops), higher productivity in cereal cultivation, and innovative livestock management activities that increase assets of the most vulnerable population. Achieving higher production efficiency has been at the core of USAID|Yaajeende’s efforts to reduce food insecurity in its intervention zones. To achieve higher production efficiency, USAID|Yaajeende has employed innovative methods, such as:  Bio-fortification, a process by which the nutritional quality of staple crops is enhanced through conventional plant breeding and modern technology. To date, the project has implemented various bio-fortification activities to secure, field-test, and disseminate a range of biofortified crop varieties: o Provitamin A carotenoid-biofortified sweet potatoes o Short-cycle iron- and zinc-biofortified millet o Quality protein maize (Obatampa variety) o Iron- and zinc-biofortified common beans  Conservation agriculture, a package of resilient farming techniques aimed at improving soil health through higher use of organic fertilizer (either compost or fine dried cow manure when compost is not available) and lower use of chemical fertilizer and through maximizing water use by ripping5 (rather than plowing) and other catchment techniques to retain soil moisture and increase yield 5 If the soil is fairly light and if the compaction or hardpan is near the surface especially on lighter soils (as is the case in the zones of intervention), a ripper to loosen the soil can be used. A ripper is a chisel-shaped implement that breaks up surface crusts and opens a narrow slot or furrow in the soil, about 5–10 centimeters deep. Unlike a plough, a ripper does not turn the soil over. The ground in between the rows is undisturbed, except for controlling weeds. Rainwater concentrates in the planting lines and sinks into the 11  Improved flood recession agriculture, using improved fertilization, planting, and phytosanitary treatment. Flood recession agriculture is practiced along the river banks and in the bas fonds, or lowland, as water recedes. Practiced from October to March in the Senegal River Valley, it typically involves maize and sorghum, and increasingly horticulture, especially potatoes and sweet potatoes.  Improved irrigated rice, maize, and sorghum cultivation, using deep urea placement and improved varieties, such as New Rice for Africa, a cultivar group of interspecific hybrid rice developed by the Africa Rice Center, to improve the yield of African rice cultivars.  Bio-reclamation of degraded land, a practice used to restore degraded agricultural land, increasing food production while empowering women by giving them access to newly restored land. (The restored land is formally attributed to women’s groups for 25 years or more.) The technique involves scarifying degraded land to break down the surface crust, building micro-catchments to channel water run-off to plants and trees, and digging holes for compost or manure to restore fertility. Micro￾catchments are then planted with traditional vegetables, such as okra and bissap, and irrigated by the captured water run-off.  Improved animal breeding using improved-race Guera goats capable of producing three liters of milk per day and improved-race rams (Touabir), larger-sized rams that increase offspring value.  Gardening activities such as home micro-gardens, and community and school gardens to increase availability of and access to nutritious food.  Agroforestry products and technology, such as live fencing and agroforestry plant production, including trees with high-nutrition value (Moringa) and high nitrogen-fixing capability to fertilize soils (Faidherbia soil where the crop roots are growing. Using the ripper allows farmers to sow the crop earlier and faster than if they plough the soil and then plant. 12 alba and Acacia mellifera). With local government involvement, the project also sponsored widespread planting and domestication of other species, such as madd and shea. USAID|Yaajeende used a wide variety of approaches to promote adoption and dissemination of new technologies, practices, and behaviors. To promote adoption and dissemination of biofortified crops, USAID|Yaajeende assisted the Institut Sénégalais de Recherche Agricole (ISRA)6 in securing 24 varieties of orange-fleshed sweet potatoes from HarvestPlus in Kenya,7 improved millet and common beans, and horticulture. After field-testing to ascertain which varieties worked best in Senegal, ISRA provided vines to community multipliers8 and mother-to- 6 Founded in 1974, the Senegalese Institute for Agricultural Research (ISRA) develops and conducts research on crops, livestock, fisheries, forestry to promote agricultural production in Senegal. 7 HarvestPlus is a leader in the global effort to end hidden hunger caused by a lack of essential vitamins and minerals in the diet, such as vitamin A, zinc, and iron. HarvestPlus is part of the consultative group for International Agricultural Research Program on Agriculture for Nutrition and Health, a program which helps realize the potential of agricultural development to deliver gender￾equitable health and nutritional benefits to the poor. The HarvestPlus program is coordinated by two of these centers, the International Center for Tropical Agriculture and the International Food Policy Research Institute. 8 Community multipliers are a network of community seed producers that USAID|Yaajeende has put in place to create base seeds from open-pollinated, non-proprietary varieties, particularly for biofortified millet and orange-fleshed sweet potatoes. Community multipliers purchase germplasm from ISRA, and after multiplication, sell the genetic resources to producers through community￾based solution providers. 13 mother groups9 for dissemination. Other mechanisms for adoption and dissemination included demonstration plots,10 farmer training, and communication campaigns. To promote horticulture and increased cereal productivity, USAID|Yaajeende uses demonstration plots, training via mother-to-mother groups and community nutrition volunteer networks, communication campaigns, techno-fairs, credit facilitation, producer field training via partner producer associations, government extension agents, project staff serving as field facilitators to coordinate training and monitoring, provision of inputs, services and technical assistance through community-based solution providers,11 and limited grants. Livestock development activities are carried out through USAID|Yaajeende’s animal-placement or pass-on￾the-gift program.12 A program beneficiary is typically selected among the most vulnerable female 9 Groups of 12 to 15 women who are pregnant, breastfeeding, or have a child under five years of age. Originally formed for educational purposes, they are led by a community nutrition volunteer. USAID|Yaajeende has used mother-to-mother groups for livestock placement, micro-gardening, bio-reclamation of degraded land, revenue-generating activities, cereal banking, and rollout of biofortified varieties. 10 The purpose of these plots is to demonstrate new or improved varieties of agricultural and horticultural crops; the handling and management of soils; the adaptability of certain soils to certain crops; improved cultural methods in the growing and harvesting of crops; and improved methods in farm management and in farm accountancy. 11 Community-based solution providers represent a network of trained community entrepreneurs that USAID|Yaajeende has created to make products, services, and information available to farmers. 12 The pass-on-the-gift program is implemented by Heifer International, one of USAID|Yaajeende’s implementing partners. 14 population in the community to receive goats, sheep, or hens, along with training in animal husbandry. The beneficiary will pass on these gifts to an equally needy family in the community, helping others as they have been helped. The recipient breeds the animals with an improved-breed animal (providing more meat, milk, or eggs) placed with the community nutrition volunteer. As the animals multiply, the beneficiary must pass on the same number of animals received to another vulnerable family. USAID|Yaajeende’s interventions to promote adoption and dissemination of new products, technologies, and practices have been highly effective. The effectiveness of USAID|Yaajeende’s interventions in horticultural production is demonstrated by an expansion in production areas, higher yields, higher consumption and sales, and higher adoption rates. Horticultural production was non-existent when the project started. Adoption rates (measured by the ratio of the number of producers applying the new technology to the number of producers trained) increased by approximately 59 percent in Year 2 and 89 percent in Year 3.13 Bio-degraded land gardening more than quadrupled between Year 2 and Year 3 (from 37 hectares in Year 2 to 201 hectares in Year 3). Horticultural yields nearly doubled during the same period (2,188 kilograms per hectare in Year 1 and 3,977 kilograms per hectare in Year 2). Higher yields and expansion in cultivated areas have resulted in higher sales of surplus production (from 70 percent in Year 2 to 75 percent in Year 3). As reflected in Table 2 (next page), cereal productivity through conservation agriculture has increased significantly in 2013. Maize yields increased between 29 percent (Kédougou and Matam) and 136 percent (Bakel), and sorghum yields increased between 29 percent (Matam) and 177 percent (Bakel). In flood 13 Unless otherwise indicated, figures reported in this evaluation are USAID|Yaajeende’s. 15 recession agriculture, maize yields increased by more than 130 percent and sweet potato yields by more than 300 percent between Years 2 and 3 and Years 3 and 4. Table 2. Conservation Agriculture Versus Traditional Agriculture: Crop Yields in 2013 Crop Yield Kédougou Bakel Matam Maize Sorghum Maize Sorghum Maize Sorghum Yields Using Conservation Agriculture (Kilograms per Hectare) 2,138 2,280 3,101 1,996 1,054 480 Yields Using Traditional Agriculture (Kilograms per Hectare) 1,656 1,561 1,312 720 814 371 Percent Change +29 +46 +136 +177 +29 +29 Source: USAID|Yaajeende; Direction Régionale de Développement Rural, Tambacounda-Bakel, Kédougou and Matam Although poultry assets were more than 20 percent lower between Years 2 and Year 3 due to high mortality rates, sheep and goats — the two most valuable assets — increased by more than 50 percent and nearly 15 times, respectively (see Table 3).14 Table 3. Livestock Assets in Years 2 and 3, Including Assets Multiplied Through the Pass-on-the-Gift Program Livestock Asset Year 2 Year 3 Number Number Percent change Goats 302 4,712 1,460 Sheep 1,428 2,221 56 Poultry 7,714 5,845 -24 14 Due to a high mortality rate for poultry in Year 3, USAID|Yaajeende training has concentrated on poultry disease prophylaxis and rural aviculture for community nutrition volunteers and relays, as well as for facilitators and livestock agents, so that smallholder poultry activities can be monitored more effectively. 16 Source: Team calculations using data provided by Yaajeende The effectiveness of USAID|Yaajeende’s interventions is reflected in key informant interviews and focus￾group discussions. In key informant interviews, nearly 88 percent of the respondents (35 out 40) agreed with the statement that project interventions have been effective. Fewer than 3 percent disagreed, and the rest of the respondents did not know. A positive assessment of project activities was considerably higher in focus-group discussions, in which nearly 97 percent of the participants (255 out of 264) reported that project interventions have been effective. USAID|Yaajeende has made a considerable effort to ensure the sustainability of its vehicles and approaches to promoting adoption and dissemination of more efficient technologies and innovations. However, the likelihood that the results of these efforts will be sustained beyond the life of the project has yet to be supported by more conclusive evidence. According to USAID, “Sustainability is achieved when host-country partners and beneficiaries are empowered to take ownership of development processes, including financing, and maintain project results and impact beyond the life of the USAID project.” In assessing USAID|Yaajeende’s interventions to promote adoption and dissemination of more efficient technologies and innovations, several elements are likely to contribute to project sustainability.15 For example:  The USAID|Yaajeende team planned many activities with government personnel, and government agencies validated the introduction of improved varieties.  USAID|Yaajeende staff collaborated with the Institut Sénégalais de Recherche Agricole for in-vitro plant propagation and for introducing and disseminating improved varieties, including tests with common beans biofortified with iron and zinc, short cycle biofortified millet with extra iron and zinc, and orange￾fleshed potato cuttings.  USAID|Yaajeende staff collaborated with government technical personnel and extension agents from the Agence Nationale de Conseil Agricole et Rurale and the Direction Régionale de Développement Rural in each intervention zone.  The project trained local farmers on how to use new varieties and cropping methods, resulting in a transfer of valuable knowledge  The project trained producers through subcontracts with local NGOs and producer federation partners, such as the Union des Producteurs Horticoles de Bakel and the Fédération des Associations du Fouta pour le Développement.  USAID|Yaajeende used community-based solution providers, a central pillar of its institutional capacity-building program, to strengthen private sector capacity and boost private sector participation in developing local communities.  USAID|Yaajeende has enhanced the potential for program sustainability by using CBSPs, a private￾sector driven network (see Question 2), to ensure that the supply of agricultural inputs and services remain available to program beneficiaries after the program ends. 15 USAID|Yaajeende’s program sustainability will be investigated in Question 2 and more comprehensively analyzed in Question 9. 17  Adoption of new cultivars, improved varieties, and agricultural techniques has increased significantly, providing an early sign of program sustainability.  USAID|Yaajeende has not only created service demand (as evidenced by high adoption rates), but it has also paid attention to supply factors — the ability of solution providers to deliver their services effectively.  Adoption of new cultivars, improved varieties, and agricultural techniques has increased significantly, providing an early sign of project sustainability. The potential for sustainability in USAID|Yaajeende’s interventions is also reflected in key informant interviews and focus-group discussions. In key informant interviews, 26 respondents, or 65 percent, agreed with the statement that project interventions will be sustained after the project ends. Only one respondent disagreed with the statement and the rest of the respondents did not know. A positive assessment of project activities was considerably higher in focus-group discussions, in which nearly 88 percent of the participants (232 out of 264) reported that project interventions are likely to be sustained. Participants cited the relevance of new technologies and communities’ ability to apply what they have learned without future USAID|Yaajeende assistance. However, a number of factors continue to obscure the sustainability question as it relates to promoting adoption and dissemination of new technologies and innovations. For example:  Using CBSPs will not guarantee that services will be sustained beyond the life of the project (see detailed analysis in Question 2). Project assistance continues to be provided to all producers and other stakeholders, including those who have repeatedly received project assistance. 18  USAID|Yaajeende has collected data on adoption rates by tracking new technologies and farming practices applied. However, this indicator does not denote sustainability because adoption rates are likely to be correlated with continued project assistance and may not be sustained when project assistance ends. A better indicator of sustainability would measure adoption rates among farmers who no longer receive project assistance and who continue to apply new technologies and farming practices.  USAID|Yaajeende has not conducted a profitability analysis of new agricultural techniques. Available information suggests that only total revenue data have been collected for certain commodities and that no expenditure data in the form of labor, physical, and capital inputs have been estimated. A thorough profitability analysis using standard farm budgets would provide a necessary (though not sufficient) indication of enterprise sustainability, especially when project support (such as grants and subsidized inputs) is considered.16  USAID|Yaajeende has yet to formulate a well-articulated sustainability strategy for its interventions to promote adoption of new technology, complete with sustainability indicators to monitor progress and case studies to assess how constraints can be removed. This gap is reflected in the project’s PMP, 16 USAID/Senegal conducted a cost-benefit analysis of its investments in agricultural development, based in large part on USAID|Yaajeende’s interventions. However, cost-benefit analysis and farm budgeting have distinctly different purposes. Cost￾benefit analysis is used to determine if a given investment or decision is sound or to identify a basis for comparing projects. It involves comparing the total expected cost of each option against the total expected benefits, to see whether the benefits outweigh the costs and by how much. Benefits and costs are expressed in monetary terms and are adjusted for the time value of money, so that all project flows of benefits and costs over time are expressed in terms of their "net present value." If the discounted present value of the benefits exceeds the discounted present value of the costs then the project is worthwhile. Conducting a cost-befit analysis is different from developing standard farm budgets. Farm budgets are resource allocation mechanisms that help answer the question of how to best organize and manage the farm business – in terms of what crops can be produced, what production practices should be used, and what equipment will be needed -- so that net revenue can be maximized. 19 which does not contain indicators to show that the new varieties and production technologies are likely to be sustained once project assistance has been withdrawn. CONCLUSION USAID|Yaajeende has introduced new products, technologies, and innovations to strengthen food security in its intervention zones. Achieving higher production efficiency has been at the core of this initiative. The project has used diverse approaches to promote adoption and dissemination of new technologies, practices, and behaviors — and these approaches have been highly effective. USAID|Yaajeende has made a considerable effort to ensure the sustainability of its vehicles and approaches to promote adoption and dissemination of more efficient technologies and innovations. Such efforts are reflected in key informant interviews and focus-group discussions with stakeholders. However, more rigorous empirical evidence would have better demonstrated the likelihood of project sustainability. RECOMMENDATION USAID|Yaajeende should conduct a detailed profitability analysis of its agricultural innovations in a series of farm budgets to examine inputs and revenues generated under various scenarios, reflecting the range of conditions prevailing in intervention zones. To be meaningful, the analysis should mirror production and marketing conditions in the absence of project support, accounting for grants, subsidies, and financial incentives received from the project. 20 Data on adoption rates should be collected, not only for producers who continue to receive project assistance, but also for producers who no longer receive assistance and those who never received direct assistance from the project. A high adoption rate among those who no longer receive project assistance and those who never received assistance will shed light on the likelihood that a new technology or farming practice is likely to be sustained beyond the life of the project. An example of indicators to measure farmers’ sustainable adoption of new agricultural methods could be the “percentage of farmers who continued to apply at least [a project-defined number of] innovations (crop/livestock) promoted by the project, [a project-defined number of] agricultural seasons after project assistance ended.” Another example could focus on farmers’ sustainable access to credit by identifying the “percentage of farmers who continued to use financial services (agricultural credit or agricultural insurance) [a project-defined number of] agricultural seasons after project financial services facilitation ended.” A limited number of indicators capturing the results of these two investigations would form an integral part of a more comprehensive sustainability plan including other key project components. Question 2: Has the CBSP model proven to be an effective and sustainable private sector￾driven approach to reduce undernutrition in targeted areas? FINDINGS The CBSP model has been effective.  CBSPs represent a network of trained community entrepreneurs that USAID|Yaajeende formed to make products, services, and information available to farmers. The CBSP model is based on the premise that agriculture and nutrition activities are driven not only by demand for inputs and services for production, but also by the ability of the market to supply those inputs and services to producers. From this perspective, CBSPs’ effectiveness will be measured by their ability to meet producers’ needs in terms of quantities supplied and product diversification.  The number and types of products and services offered by CBSPs increased five-fold (from three to 15) in Year 2 and by an additional 45 percent (from 15 to 22 in Year 3), for an average increase of more than 300 percent in the past two years. From a low base of three products (agricultural inputs, nutrition products, and credit), services increased to 22 products in Year 3, including processing, insurance, nursery, and veterinary services.  The number of commercial firms that CBSPs rely on for their supplies at the national and regional levels has more than doubled between Year 1 (nine suppliers) and Year 2 (20 suppliers). The number of suppliers (19) was similar in Year 3 because fortified flour and iodized salt are now produced locally or imported from neighboring regions.  Suppliers’ increasing interest in using the CBSP network to expand operation and increase market share is reflected in their growing readiness to train CBSPs on how to use their products most effectively (via six training events in Year 2 and eight in Year 3). Suppliers also set up demonstration plots (five in Year 3) and participated in local fairs (nine in Year 2 and eight in Year 3) to promote their supplies. 21 22  CBSP sales have increased steadily in the past three years, from a low base of approximately $50,000 in Year 1, to more than $400,000 in Year 2, to more than $850,000 in Year 3. These results indicate that producers are increasingly relying on CBSPs for their production needs and that suppliers are increasingly relying on them to expand operation.  Key informant interviews and focus-group discussions support the effectiveness of the CBSP model. In interviews, 50 percent of the respondents (20 out 40) agreed with the statement that the model has been effective. Only 5 percent (two out of 40) disagreed and the rest of the respondents did not know. A positive assessment of project activities was considerably higher in focus-group discussions, in which more than 66 percent of participants (175 out of 264) reported that the model has been effective. Several characteristics of the CBSP model point to its potential sustainability. However, the likelihood that the model will be sustained beyond the life of the project has yet to be supported by more conclusive evidence.  USAID|Yaajeende has paid ample attention to the sustainability of input supply and to other value￾chain links, including marketing through commercial channels, and has not relied on donated inputs. In this context, the CBSP model is a private sector-driven network consisting of entrepreneurs motivated by the desire for financial success.  The emerging transition of the community nutrition volunteer network into CBSPs is another sign of project sustainability. The effectiveness and sustainability of an intervention is likely to depend on the behavior of two groups of people: households and solution providers (see for instance, Victora et al.; Sjoblom 2012; Di Vinadio 2013). Since effectiveness and sustainability hinge on supply-side factors — that is, the ability of providers to deliver services effectively — even women with better knowledge of good child nutrition practices may be limited in their ability to act on their knowledge if they lack access to nutrition-related products and services.  Starting from the premise that the CBSP model’s sustainability required financing arrangements not readily available to CBSPs, USAID|Yaajeende assisted CBSPs in obtaining credit though the local credit union community and banking sector. However, CBSPs’ access to credit without project assistance has yet to be demonstrated.  Key informant interviews and focus-group discussions underpin the CBSP model’s sustainability. During interviews, 60 percent of the respondents (24) agreed with the statement that the model is sustainable. Only 20 percent (eight out of 40) disagreed and the rest of the respondents did not know. A positive assessment of project activities was considerably higher in focus-group discussions, in which 69 percent of the participants (182) reported that the CBSP model is sustainable.  Yaajeende continues to provide technical assistance to CBSPs, including those who have repeatedly received assistance.  Focus-group discussions suggest that expectations for additional project support remain high among CBSPs, indicating that no gradual phase-out and graduation criteria have been set. 23  Available information suggests that only CBSP sales data have been collected and that no expenditure data have been estimated. A net margin analysis would have provided a necessary (though not sufficient) indication of enterprise sustainability — especially when project support (such as grants and free or subsidized ripping equipment) is taken into consideration.  The relationship between credit institutions and CBSPs remains unstable.17  USAID|Yaajeende has yet to conduct an in-depth evaluation of the sustainability of the CBSP model. For instance, with the exception of a general statement in the annual report for Year 3,18 there is no evaluation that accounts for why a number of CBSPs who received project assistance are no longer operational (or what accounts for the success of many others).  USAID|Yaajeende has yet to formulate a more rigorous strategy to assess the sustainability of the CBSP network, complete with sustainability indicators to ensure that the CBSP function is likely to be sustained once project assistance has ended.19  This gap is reflected in the project’s PMP, which does not contain any indicators to monitor the sustainability of the CBSP network. This task is important since the CBSP model is a critical element of USAID Yaajeende. CONCLUSION The effectiveness of the CBSP network is illustrated by its members’ increasingly strong commercial ties with suppliers and their success in offering an increasingly wide range of products and services to their communities. Several indications support the conclusion that the CBSP model is likely to be sustainable, including the private sector nature of the network, its nascent working relationship with credit unions, and its increasingly stronger links with national and regional suppliers. However, the absence of rigorous M&E mechanisms, such as graduation benchmarks, time-bound phase-out plans, and evaluation reports, makes it difficult to assess with any degree of certainty whether the CBSP network will be sustained. 17 The project notes that three years into the project, CBSP access to credit remains uneven. For instance, “in Kédougou credit awards were not as successful. This area constituted only 4.9% of the total CBSP financing. This is mainly due to the fact that a dozen CBSPs are in dispute with Agence de Crédit pour l'Enterprise Privée (ACEP) due to less than full repayment of their loans. The project is mediating this dispute and hopes to find resolution. ACEP has said that it cannot provide loans to CBSPs in Kédougou until the dispute has been resolved. Efforts will be made in Year 4 to improve access to credit for the CBSPs, and this will require a closer coaching of the CBSPs to facilitate access to credit for producers and facilitation of access to agricultural equipment through credit.” 18 The statement reads as follows: “In the first 3 years of experience, the CBSP networks have become dynamic and engaged in a broad range of products and services. However, with the rapid growth of this group of entrepreneurs, a few dishonest actors have created some problems for the entire group. In Year 3, the project helped the regional networks ‘clean house’ and reduce the number of CBSPs involved in order to encourage professionalism, increase skills, improve quality control and ensure that contract services provided to firms and producers are of the highest standard. The project has selected the top CBSPs to continue in the program. 86 CBSPs were selected in Bakel, 28 in Kédougou and 144 in Matam for a total of 258 to work more intensively with the project in the upcoming years. This represents about 1/3 of the CBSPs that [sic] were originally identified and trained. This network will grow in the future, but with limited expansion until this core group of 258 CBSPs have been strengthened and have proven their mastery of the CBSP methodology” (USAID|Yaajeende Annual Report, 2013). 19 Two examples of such indicators could be: “Percentage of CBSPs who continued to use financial services (agricultural credit and/or agricultural insurance), [a project-defined number of] agricultural seasons after project financial services facilitation ended.” Or, “Percentage of CBSPs who continued to have access to inputs from national and regional suppliers [a project-defined number of] agricultural seasons after project facilitation ended.” 24 RECOMMENDATION Yaajeende should formulate a comprehensive and rigorous M&E mechanism to assess the sustainability of the CBSP network. Such a mechanism would feature an adequate20 number of sustainability indicators, clearly defined graduation benchmarks, phase-out plans, and focused evaluation reports. Since the CBSP model is a central component of project interventions in agriculture and nutrition, the PMP should also include a limited number of sustainability indicators to monitor outcomes. Question 3: Have the activities that increase household assets and income led to improved participation of the rural poor in rural growth? FINDINGS USAID|Yaajeende has used three major mechanisms to increase household assets and income for the rural poor: the pass-on-the-gift program, bio-reclamation of degraded land, and dissemination of innovations and more efficient farming techniques among smallholders. Pass-on-the-gift is an asset-building program providing a pathway out of vulnerability as households transition from raising chickens to raising goats and sheep. Bio-reclamation of degraded land involves restoring productivity or use to lands degraded by human activities or natural phenomena. The project has disseminated innovations and more efficient farming techniques among smallholders through biofortification; conservation agriculture; improved flood recession agriculture; improved irrigated rice, millet, and sorghum cultivation; bio-reclamation of degraded land; and improved animal breeding. The pass-on-the-gift program has increased household assets significantly. As shown in Table 4, the number of vulnerable households and household members who benefited from the program increased by more than 150 percent (from 1,300 to more than 3,000) between Years 2 in 3. By 2013, more than 30,000 vulnerable household members benefitted from the program. For the pass-on-the gift program, beneficiaries trade livestock ownership from poultry to more prized livestock assets (sheep and goats). For example, a case study in the villages of Diakateli and Fongolimbi in the Kédougou region, one of USAID|Yaajeende’s beneficiary communities, shows that from 196 chickens received in early 2012, livestock assets in early 2014 grew to 343 chickens, seven sheep, and 15 goats, in addition to 21 chickens and 104 eggs consumed and 94 chickens sold. Table 4. Pass-On-The-Gift Program: Total Assets and Number of Beneficiary Vulnerable Households and Household Members, Years 2 and 3 20 USAID defines adequacy as: “Taken as a group, the indicator (or set of indicators) should be sufficient to measure the stated result. In other words, they should be the minimum number necessary and cost-effective for performance management” (USAID 2010b). 25 Beneficiaries Year 1 Year 2 Year 3 Number Number Number Percent Change (Year 2- Year 3 Vulnerable Households (*) 0 1,281 3,227 152 Vulnerable Household Members (**) 0 12,810 32,270 152 (*) Includes vulnerable households receiving pass-on-the-gift animals (**) Immediate members of the vulnerable households (approximately 10 members per household) Source: USAID|Yaajeende data It is important to note that household livestock assets are increasingly being protected by the new owners through the purchase of livestock insurance policies underwritten by the Compagnie Nationale d’Assurances Agricole du Sénégal — a USAID|Yaajeende innovation. (As of September 2013, 26 owners insured 216 animals, for a total value of $24,000). Livestock insurance has promoted household members’ access to credit: livestock assets are now increasingly being used as collateral to access credit. Household assets have increased through reclamation of biodegraded land Reclamation of biodegraded land has contributed to increased household wealth. Land reclamation and formal attribution of tenure to vulnerable groups for a period of 25 years has increased more than four-fold between Years 2 and 3 (see Table 5). Table 5. Increased Assets through Reclamation of Biodegraded Land Land Area Year 1 Year 2 Year 3 Hectares Hectares Hectares Years 2 to 3 (percent change) 0 37 201 440 percent Source: Yaajeende and team calculations Higher horticultural yields have enabled smallholders to market much of their production and raise household revenue. Improved agricultural inputs and new farming methods have enabled farmers to raise yields, sell surplus production, and boost revenue. By selling 75 percent of their horticultural production (Table 6), smallholders in the project areas have started to transition from subsistence agriculture to becoming an integral part of the local market economy. Table 6. Smallholder Horticultural Production: Marketed Surplus and Revenue, 2011-2013 Marketed surplus Year 1 Year 2 Year 3 Years 2 to 3 ( percent change) Metric Ton 0 842 1,833 118 percent Percent of production 0 70 75 Revenue (000 dollar) 0 217 840 288 percent Source: Yaajeende and team calculations 26 CONCLUSION USAID|Yaajeende has used three major mechanisms to increase household assets and income for the rural poor: the pass-on-the-gift program, bio-reclamation of degraded land, and dissemination of innovations and more efficient farming techniques among smallholders. The three programs have been successful. RECOMMENDATION The three programs should be continued and strengthened. Question 4: What project activities have positively enabled value-chain investments to lead to improved consumption of diverse diets and quality foods? FINDINGS Value-chain investments and associated activities are reflected in the increasingly wide variety of farm inputs used by smallholders and CBSPs’ expanding business size and services. Smallholders have invested in a wider variety of farm inputs, such as improved seeds, phytosanitary products, fertilizer, livestock feed, and farming tools. CBSP sales to producers have increased seventeen￾fold between Years 1 and 3 ($50,000 in 2011; $400,000 in 2012; and $850,000 in 2013). The number of products and services offered by CBSPs in the same period increased seven-fold, from three in Year 1 to 15 in Year 2 and 22 in Year 3. In response to the higher demand for CBSP services, the number of CBSPs has increased significantly (from 46 in Year 2 to 302 in Year 3). CBSPs’ expanded investment is reflected in the amount of loans they received from credit institutions. As shown in Table 7, CBSPs’ borrowing increased from zero in Year 1 to just under $40,000 in Year 2 and approximately $250,000 in Year 3. Table 7. Loans Accorded to CBSPs (Year 1-3) Year 1 Year 2 Year 3 Number of CBSPs 0 46 302 Loan amount 0 $38,708 $245,690 Source: Yaajeende 27 In addition, private firms offering improved inputs and new technologies, such as Hortis, Tropicasem, Senchim, Equip Plus, and Floridia, have invested funds and staff time to strengthen the capacity of the CBSP network through field training. These firms also participated in “techno-fairs” organized by USAID|Yaajeende in the region. No reliable impact evaluation data are available to assess whether value-chain investments have led to improved food consumption in terms of dietary diversity and quality21 and which project activities have contributed to improvement. Lack of dietary diversity is a severe problem among the poorest segments of the rural population because their diets are predominantly based on starchy staples, with little or no animal products and few fresh fruits and vegetables. These diets tend to be low in micronutrients, and the micronutrients they contain are often in a form that cannot be easily absorbed. Dietary diversity ensures adequate intake of essential nutrients and promotes good health. A more diversified diet leads to a number of improved outcomes in birth weight, child anthropometric status, and improved hemoglobin concentrations (see, for instance, Ruel 2003; Swindale et al. 2006; Kennedy et al. 2011). Dietary diversity scores are calculated by summing the number of food groups consumed in the household or by the individual respondent, usually in a 24-hour recall period. The score ranges from zero to 12, signifying the maximum number of food groups consumed over the reference period. Such food groups include cereals; root and tubers; vegetables; fruits; meat, poultry, and offal; eggs; fish and seafood; pulses, legumes, and nuts; milk and milk products; oils and fats; sugar and honey; and miscellaneous. More than 40 percent of interviewees (17 out 40) and nearly 90 percent of focus-group respondents (234 out of 264) stated that project activities had resulted in more diversified diets and consumption of high￾quality foods among the target population. The two project interventions most commonly cited for the improvement were fruit and vegetable production and livestock activities. Community meals were listed as an additional reason for the noted improvement.22 Higher production, consumption, and marketed surplus of horticultural and biofortified crops (Questions 1 and 3) lend support to this assessment. However, without impact evaluation data, it is not possible to assess with confidence whether value-chain investments have led to improved food consumption in terms of dietary diversity and quality and which project activities have contributed to improvement. 21 Dietary quality refers to nutrient adequacy. Adequacy refers to a diet that meets requirements for energy and all essential nutrients. 22 USAID|Yaajeende uses monthly community meals as an additional nutrition training tool. Community meals bring together specific target groups (e.g., mothers and their young children; women of child-bearing age) to prepare and serve food. Project staff use these opportunities to conduct behavior change communication (BCC) and Information, education, and communication (IEC) activities on child feeding, complementary foods, food preparation, preservation and consumption methods that optimize nutrient absorption. 28 USAID|Yaajeende’s baseline survey was conducted in 2011 but the mid-term survey has yet to be carried out. In 2013, USAID|Yaajeende conducted a household survey to collect dietary diversity data. These data, together with the baseline data, are summarized in Table 8. Table 8. Household Dietary Diversity Score for Intervention and Control Areas, 2011 and 2013 Year Intervention Areas Control Areas 2011 (*) 8.63 7.13 2013 (**) 7.47 5.97 Percent difference (2011-2013) 13 percent 16 percent Notes: (*) baseline survey; (**) household survey Source: Yaajeende As measured by a drop in the diversity score from 8.63 to 7.47, the diet became less diversified in the intervention areas two years into the project, although the drop (13 percent) was smaller than for control areas (16 percent). This is surprising because project interventions are expected to result in a higher diet diversity score for the intervention areas. This result points to potential data quality issues in the baseline data and/or the household survey.23 USAID|Yaajeende notes that the 2011 and 2013 results are difficult to compare because the baseline survey was based on a sample of nearly 3,000 households, a considerably larger sample than the one used in the 2013 household survey. For this reason, the project has focused on intra-year comparisons between intervention and control areas. It is evident, however, that questionable 23 As detailed in Question 6, the treatment and control groups do not necessarily need to have the same pre-intervention conditions. The two groups may well have different characteristics. However, many of those characteristics (such as seasonal variability and agricultural practices) can reasonably be assumed to remain constant over time, or at least over the course of the evaluation. 29 results due to inadequate sampling or any other unreliable survey implementation procedures cannot be used to make comparison over time or within years. Dietary diversity data can be used to compare household or individual consumption by food category across space and over time to assess the impact of a given intervention. Figure 2 (next page) depicts consumption patterns by food group in project and control areas in 2013. Data in Figure 2 shows that households in project areas are characterized by higher consumption of root crops; fruits; vegetables; meat; eggs; milk and dairy; and oils, fats, and butter. This pattern suggests that gardening, arboriculture, and livestock activities have contributed the most to dietary diversification in intervention areas. However, this inference cannot be confirmed with any degree of certainty due to the 2013 household data concerns outlined above. In the absence of reliable baseline and mid-term survey data, it is not possible to assess if the diet diversity status of households has improved over time in intervention areas and how that improvement compares with changes in the diet diversity status of households in control areas. This conclusion also applies to the undernutrition data in Question 5. 30 Figure 2: Consumption of Food Groups in Project Areas and Control Areas in 2013 (Source: USAID|Yaajeende) CONCLUSION Dietary diversity as a measure of household food access and food consumption is an important indicator that may shed light on which future project interventions could be most effective. When combined with other nutrition information, it could provide a more complete framework for analyzing the food and nutrition security status of a population. 31 Interviews and focus-group discussions, together with production, consumption and marketed-surplus data, indicate that project activities resulted in more diversified diets and consumption of higher-quality foods in project areas. However, this conclusion cannot be corroborated with quantitative data, in part because the mid-term survey scheduled for mid-2013 has not yet been conducted. Due to the absence of reliable time-series data on diet diversity in USAID|Yaajeende’s intervention and control areas, it is not possible to assess whether project activities have contributed to improved food consumption in target areas and whether the improvements can be attributed to those activities. The mid-term survey scheduled for the third quarter of Year 4 will fill this data gap. RECOMMENDATIONS Given the one-year delay in implementing the mid-term survey, and since the project is scheduled to end in 2015, it may be more cost-effective to conduct the survey in the beginning of Year 5. Unless the project is extended, it would be wasteful to collect two sets of the same impact evaluation data less than a year apart. Since women are a key target group, the impact evaluation should consider performing a women’s dietary diversity analysis to examine the micronutrient adequacy of diets for women of reproductive age (see, for instance, Ruel et al. 2011). Assuming that similar data are included in the baseline survey, the food groups considered in women’s diet diversity would emphasize micronutrient intake more than economic access to food. In addition to calculating mean dietary diversity scores, it is important to investigate which food groups are predominately consumed at different levels of the scores. Such an investigation would provide information on foods eaten by people with the lowest dietary diversity, and which foods are added by people with a higher score. Comparing dietary diversity scores for intervention and control groups may also shed light on which project interventions would be most effective in the future. Question 5: What investments in human and institutional capacity development have effectively generated large-scale nutrition outcomes? FINDINGS USAID|Yaajeende has invested heavily in human and institutional capacity development. USAID|Yaajeende’s institutional capacity building has focused on private sector development, governance, nutrition education, and innovative farming methods. Since private sector development was discussed in Question 2, it will not be repeated here. Governance has been at the core of the USAID|Yaajeende project. Training in agriculture has focused on horticultural farming. Nearly 6,000 people applied horticultural training in Year 3 in the rainy season and 8,000 applied it in the off-season. Approximately 7,000 people applied training in flood recession agriculture in Year 3, and more than 6,500 people applied livestock rearing training through the pass-on-the-gift program. Major activities to build institutional capacity in agriculture included assisting ISRA in building genetic stock, 32 building a network of community seed multipliers who procured their stocks from ISRA, and enabling mother-to-mother groups to fully participate in biofortified crop production. Through Year 3, mother-to￾mother groups received 8,800 orange-fleshed sweet potato vines and 1.4 metric ton of biofortified millet seed, along with training on how to cultivate these new varieties. To reduce malnutrition, the mother-to-mother nutrition program has targeted pregnant women, mothers of children less than two years old, and their children. The major programmatic areas addressed by USAID|Yaajeende were  Good nutrition in the first 1,000 days of a child’s life (the “window of opportunity” from conception to two years of age)  Exclusive breastfeeding for the first six months and continued breastfeeding for at least two years  Adequate complementary feeding for children between six and 23 months; for malnourished children, between 24 and 59 months  Adequate intake of vitamin A, iron, and other micronutrients for women and children  De-worming, vaccinations, and diarrhea treatment  Clean water, hand washing, and use of latrines 33 By targeting pregnant women, mothers of children less than two-years old, and their children, USAID|Yaajeende has followed industry norms for effectively reducing food insecurity in poor populations. USAID|Yaajeende’s approach is a preventive strategy24 reflecting accepted best practices to improve child nutrition.25 According to USAID research, decades of epidemiologic findings, and United Nations Children's Fund (UNICEF) recommendations, children should receive nothing but breast milk (exclusive breastfeeding) for the first six months of life. Complementary foods should be introduced after a child turns six months old to reduce the risk of malnutrition. UNICEF also recommends that breastfed children between six and 23 months should be fed four or more other food groups daily. Non-breastfed children should be fed milk or milk products, in addition to four or more food groups. Infant and Young Child Feeding (IYCF) guidelines also recommend that children should be fed a minimum number of times per day (UNICEF 2010). Providing micronutrients and vaccinations is also essential. Applying best practices (see World Bank 2008; Horton 2008), USAID|Yaajeende’s interventions included promoting immunization services, vitamin A distribution, oral rehydration salt, zinc, and de-worming for children less than five years old. Clean water and improved sanitation were also emphasized. USAID|Yaajeende’s interventions are likely to have generated positive nutrition outcomes in project areas. However, in the absence of a rigorous mid-term evaluation survey to compare the current nutrition situation relative to the baseline for intervention and control areas, it is not possible to assess with any degree of certainty whether improvement in the population’s nutrition status can be attributed to project activities. Based on SMART survey data, stunting and wasting — two indicators of undernutrition — declined significantly between 2010 and 2013 in the project’s three regions, by as much as one-third (stunting) and nearly one-fourth (wasting) in Kédougou. (See Table 9 for more information.) Table 9. Change in Stunting and Wasting Rates for U-5 Children in USAID|Yaajeende’s Intervention Zones (2010-2013) 24 USAID/Food for Peace defines prevention as follows: “As in any public health intervention, prevention means population-based coverage. As an analogy, consider the polio vaccine: all children in a population are entitled to, and should get the polio vaccine no matter the socioeconomic status of the household. Similarly, all children in a population with high stunting rates…are at risk of becoming malnourished during the 1000 days between conception and two years of age and thus should be protected from the ravages of nutritional deficiencies” (USAID/FFP 2012). A study (Menon et al. 2007) conducted in Haiti over a three-year period in communities randomly selected to receive a preventive approach of a Title II maternal and child health and nutrition program found that the prevalence of stunting, under-weight, and wasting respectively was 4, 6, and 4 percentage points lower after three years of activity compared to communities with access only to the recuperative program approach. The preventive approach was also found to be more cost-effective than a treatment-after-the-fact approach. 25 There is consensus that one of the most effective ways to ensure good childhood nutrition is to focus efforts on the 1,000-day “window of opportunity,” the nine months preceding a child’s birth and the two years following a child’s birth. There is also consensus that the damage to physical growth, brain development, and human capital formation that occurs during this period is extensive and largely irreversible. Any interventions after this critical period are much less likely to improve nutrition. Starting at birth, improved nutrition yields benefits that cascade through life and even future generations. Undernutrition affects health and survival through higher mortality and morbidity among neonates, infants, and children, with losses in the future of economic output and increased future spending on health (see, for instance, USAID/FFP 2010; World Bank 2006; Alderman et al. 2006; Alderman et al. 2014). 34 Location Stunting Wasting Year Percent change Year Percent change 2010 2013 2010 2013 Kédougou 36 24 -33 percent 12 9.2 -23 percent Matam 18 14.2 -21 percent 23 18.8 -18 percent Bakel 20 17.8 -11 percent 17.8 14.3 -20 percent Source: SMART survey In key informant interviews, 40 percent of respondents (16 out of 40) agreed with the statement that project activities to enhance human and institutional capacity development have generated positive large￾scale nutrition outcomes. The rest of the respondents did not know. Most commonly cited reasons included agricultural training and nutrition education. Approximately 46 percent (121 out of 264) of focus-group discussion participants positively assessed project activities. The rest of the participants did not know. Table 10 describes the undernutrition status in USAID|Yaajeende’s intervention areas and control areas in 2013. Although the three indicators in control areas show improvement (with wasting declining by as much as 20 percent), the malnutrition situation in intervention areas has generally worsened. These results, when combined with the large discrepancy between wasting in the SMART survey data and the USAID|Yaajeende survey data for 2013, strongly suggest that USAID|Yaajeende’s undernutrition data for 2013 have limited reliability. As discussed in Question 4, such deficiency may be explained by the inadequate sample size used to gather the 2013 survey data. Table 10. Change in Stunting and Wasting Rates for U-5 Children USAID|Yaajeende’s Intervention Zones and Control Areas (2010-2013) Anthropometric Indicator Yaajeende Intervention Zones Control Areas Year Percent change Year Percent change 2011 2013 2011 2013 Stunting 21.8 23 6 percent 25 24.5 -2 percent Underweight 20.6 22.3 8 percent 24.3 23.5 -3 percent Wasting 14.3 13.7 -4 percent 10 8 -20 percent Source: USAID|Yaajeende 35 36 CONCLUSION The SMART survey data clearly indicates that the nutrition situation has improved in USAID|Yaajeende’s intervention zones — a conclusion corroborated by qualitative information gathered via key-informant interviews and focus-group discussions. However, as with the dietary diversity data analyzed earlier, this conclusion cannot be corroborated with dependable quantitative data for intervention and control areas. RECOMMENDATION A follow-up survey to compare current undernutrition indicators with their baseline values is necessary to assess whether (and the extent to which) USAID|Yaajeende’s human and institutional development has effectively generated large-scale nutrition outcomes. As mentioned in Question 4, it would be more cost￾effective to conduct this survey in the beginning of Year 5, rather than immediately. Question 6: To what extent has the integrated nutrition and agriculture approach led to a reduction of undernutrition among the target population? FINDINGS Only an impact evaluation can determine the extent to which the integrated nutrition and agriculture approach led to a reduction of undernutrition among the target population. USAID|Yaajeende’s development hypothesis states that integrating agriculture and nutrition can reduce undernutrition faster than by focusing on agriculture alone. The same hypothesis underlies USAID/Senegal’s FtF strategy. For project interventions to have the highest impact, they must strengthen the link between increased food production, higher income, and greater consumption of quality foods, particularly by women and children. An impact evaluation is necessary to validate the importance of the agriculture-nutrition nexus. Impact evaluations are in line with current USAID efforts to revitalize project assessments by including impact evaluations that “measure the change in a development outcome that is attributable to a defined intervention.” These evaluations “are based on models of cause and effect and require a credible and rigorously defined counterfactual to control for factors other than the intervention that might account for the observed change” (USAID 2011). Thus, impact evaluations focus on outcomes that reflect changes in well-being that can be attributed to a particular intervention, such as whether people are healthier, better educated, or less vulnerable to adverse shocks. Evaluating the impact of that intervention hinges on a fundamental question: What would the situation have been if the intervention had not taken place? 37 Although impact evaluations vary in complexity, from randomized designs to quasi-experimental methodology, to statistical controls and simulations using computable general equilibrium models, a common methodology is to compare a treatment group (such as the population living in USAID|Yaajeende project areas) with a control group (such as the population living outside of USAID|Yaajeende project areas). This comparison determines the changes in outcomes over time between the population participating in the project (the treatment group) and the population not participating in the project (the comparison group).26 This comparison will de facto involve four groups, not only two (see Figure 3, next page). The distinction between the four groups will be based on two determining factors: time and space. In addition to the group that received the treatment (population benefitting from USAID|Yaajeende), the three other groups not affected by the treatment include the treated group prior to its treatment (population living in USAID|Yaajeende project areas before the project was introduced), the control group in the period before the treatment occurred (before the project was introduced), and the control group in the current period. The rationale behind this empirical approach is that if the two USAID|Yaajeende and two control groups are subject to the same time trend, then potential confounding factors are removed and we can estimate the outcome — USAID|Yaajeende’s impact. Figure 3: Yaajeende’s Treatment and Control Groups over Time and across Space Without a follow-up survey to compare the baseline with the new undernutrition situation in intervention and control areas, we cannot accurately estimate the extent to which the integrated nutrition and agriculture approach has led to a reduction of undernutrition among the target population. 26 It is important to note that treatment and control groups do not necessarily need to have the same pre-intervention conditions. The two groups may have different characteristics. However, many of those characteristics can reasonably be assumed to remain constant over time, or at least over the course of the evaluation. 38 Malnutrition data for Matam, Bakel, and Kédougou in 2010 and 2013 suggest that USAID|Yaajeende’s interventions may have contributed to an improvement in the population’s nutrition status. However, since there are no separate sets of data for intervention and control areas, a reduction in undernutrition cannot be attributed to USAID|Yaajeende. Thus, the extent to which the integrated nutrition and agriculture approach has led to a reduction in undernutrition among the target population cannot be estimated prior to a follow-up survey. Nonetheless, it should be noted that experience from other countries demonstrates that agriculture and nutrition programs are mutually reinforcing, and that integrating nutrition and agricultural interventions is likely to yield optimal results (see, for instance, Rogers et al. 2004 and Rassas et al. 2014).27 Qualitative data corroborate this conclusion. More than 85 percent of those who participated in focus-group discussions agreed that USAID|Yaajeende’s integration of nutrition and agriculture has reduced undernutrition among the target population. None of the respondents thought that limiting project activities to nutrition or agriculture would have led to a higher reduction in undernutrition among the target population. 27 It is for this reason that USAID/Food for Peace guidance notes that in implementing multi-year assistance programs, “PM2A [Preventing Malnutrition in Children Under 2 Approach], along with the rest of a Title II program’s maternal and child health component, should be consistently linked with the program’s agriculture and livelihoods components” (USAID/FFP 2010). 39 CONCLUSION Experience from other countries has demonstrated that integrating agriculture and nutrition is likely to reduce undernutrition much more rapidly than focusing on agriculture or nutrition alone. Interviews and focus-group discussions corroborate this result. However, an impact evaluation is necessary to validate this conclusion in the context of USAID|Yaajeende’s intervention zones. RECOMMENDATION USAID|Yaajeende has scheduled a follow-up survey to compare baseline data with the current nutrition status of the targeted population. This survey should receive priority consideration in planning future activities. Question 7: To what extent has the project’s water and sanitation activities led to improved healthy behaviors of the target population? FINDINGS In the first three years, USAID|Yaajeende’s WASH activities were limited to training, communication, and promotion of WASH technologies. In Years 1 and 2, the project team focused on training and communications because there was no dedicated funding stream for WASH technologies and infrastructure. Training and communication activities focused on food preparation; hygiene and improved latrine use; drainage areas near food preparation sites; use and construction of hand-washing stations (also known as “tippy taps”);28 and food preparation tools, such as portable dish-drying racks. In Year 3, WASH activities consisted of additional training and promotion of WASH technologies, with an emphasis on tippy taps for hand washing, potable water purification, and latrine construction. USAID|Yaajeende received $1.5 million to strengthen WASH activities in Year 4. Activities have increased access to low-cost water and sanitation technologies via the CBSP network, boosted social marketing efforts to create social pressure for change, enlisted Citizen Working Groups (CWGs) and youth to help with project coordination, encouraged public debate (see Question 10 for more information), piloted new technologies (such as moringa purification , low-cost sand and carbon filtration, and multi-use systems), and increased collaboration with USAID’s Community Health Program to promote community ownership of WASH activities through CWGs. 28 A tippy-tap is a simple no-touch, low–water use, hand-washing device that can be made out of a jerry can, sticks, twine, or rope. The device is installed in kitchens and latrine areas and allows families to control water flow with a foot pedal, avoiding hand contamination. 40 The WASH M&E system does not measure intervention outcomes and program sustainability. Only 12 percent of key informants (four out of 40) agreed with the statement that project activities in water and sanitation had led to improved health behaviors among the target population. The majority of respondents did not know. However, more than 85 percent of focus-group respondents (245 out of 264) agreed with the statement. When some of the focus-group participants were asked how many of the tippy taps in their communities were functional, a relatively low number (generally corresponding to less than 20 percent) was cited. Participants hastened to note, however, that even though there were few functional tippy taps and latrines in their communities, they now understand the importance of water and sanitation for their health. USAID|Yaajeende’s WASH M&E system consists of two output indicators: the number of people trained in child health and nutrition (including WASH) and the number of school children taught about nutritional issues (including WASH). In Year 3, the project’s PMP listed one outcome indicator to record the number of households adopting improved WASH practices due to USAID|Yaajeende. There were no indicators to measure the sustainability of USAID|Yaajeende’s WASH activities. This shortcoming is all the more important because the WASH program in USAID|Yaajeende’s intervention zones continues to face daunting constraints in water availability29 and target communities are unable to pay for latrine construction or WASH supplies, such as soap, jugs, and Aquatabs®.30 29 For instance, USAID|Yaajeende notes that “in the rural community of Velingaro Ferlo, in Matam region, the nearest source of water during the dry season is 25km away. Ground or river water is brought to the community and sold for approximately $1 per 5 liters, the same price as approximately15 liters of bottled water in Dakar, and only affordable for the richest people.” (USAID|Yaajeende program description) 30 Aquatabs® are effervescent tablets which kill micro-organisms in water to prevent cholera, typhoid, dysentery, and other waterborne diseases. While Aquatabs® have a low per-unit cost, they must be purchased regularly. 41 CONCLUSION After three years of limited activity and undocumented results — due to limited funding and difficulties associated with implementing WASH activities in USAID|Yaajeende’s intervention zones — the WASH program continues to face severe constraints, which will likely have implications on program sustainability. 42 RECOMMENDATION USAID|Yaajeende should strengthen its collaboration with USAID’s Community Health Program and the Senegalese government’s Millennium Water and Sanitation Program. Such collaboration would enable USAID|Yaajeende to play a supporting role (rather than a leading role) in governance program activities (see Question 10 for more information). USAID|Yaajeende should use a limited number of indicators to measure progress toward sustainability. These may include:31  An indicator for improved access to potable drinking water: percent of households using an improved drinking water source (to be defined by the project) [a project-defined number of] months after project assistance ends.  An indicator for increased access to improved sanitation facilities: percent of households with access to an improved sanitation facility (to be defined by the project) at least [a project-defined number of] months after project assistance ends.  An indicator for improved hygiene practices: percent of households with children aged zero to 23 months that either have water and soap or a locally available cleansing agent at a hand-washing place at least [a project-defined number of] months after project assistance ends. Question 8: To what extent has the project been implemented effectively, including timely completion of project activities, effective use of project resources, reach of target groups/beneficiaries, quality of partnerships and collaboration, and contribution to overall USAID/Senegal economic growth objective goals? FINDINGS With one exception, USAID|Yaajeende has used project resources effectively.  USAID|Yaajeende’s project team consists of 76 local staff (representing 94 percent of the team), two regional staff (from Niger and Mali), and three U.S. staff (the chief of party, technical adviser, and director of finance).  Field staff are based as close to project sites as possible, with about 30 staff based in rural communities where the project implements activities.  To minimize costs and maximize workflow and collaboration, two to three colleagues, including senior staff, are assigned a single office space in regional offices.  The project uses grants to leverage matching contributions from beneficiaries, project partners, and private sector companies, and to broaden producers’ and CBSPs’ access to credit.  The project uses passage houses, rather than hotels, for staff travel.  In Year 2, the Tambacounda office was closed to move project staff closer to intervention zones. 31 Indicators adapted from USAID/DCHA/FFP, Revision to Food for Peace Standard Indicators Collected in Baseline Surveys and Final Evaluations. Information Bulletin (FFPIB) dated December 20, 2011. 43  A less effective use of project resources was USAID|Yaajeende’s household survey to collect undernutrition data for 2013. The survey was not operationally useful because it was based on an inadequate sample size that negated its value in comparing results over time and across space. USAID|Yaajeende has reached its target groups and beneficiaries. USAID|Yaajeende has targeted the poor, the vulnerable, and women. The poor are defined as smallholders with one hectare of land or less. The vulnerable are defined as the nutritionally vulnerable (women of maternal age and children less than 5 years of age), those who have no land and no livestock, and those who receive the zakat – an obligatory payment made annually under Islamic law on certain kinds of property and used for charitable and religious purposes. All project activities are centered on the poor and the vulnerable, especially women. Benefits for targeted groups and beneficiaries are linked to USAID|Yaajeende’s interventions to promote adoption and dissemination of new products and practices, mechanisms to increase household assets and income, and investments in human capacity development. USAID|Yaajeende’s partnerships and collaboration are of the highest quality. USAID|Yaajeende distinguishes between institutional partnerships, subcontracting arrangements, and collaboration. Institutional partnerships are between two co-equal and autonomous entities (government agencies, projects, private enterprises, and NGOs). Collaboration is relatively long-term in nature, encompassing a variety of activities. Partnership is based on a memorandum of understanding (MOU), a joint work plan, and joint M&E activities. Examples of institutional partnerships include a partnership with the Senegalese Institute for Agricultural Research to introduce and field-test new varieties, seed multiplication, and germplasm banking. The project also has a detailed MOU with the National Society for Management and Operation of the Senegal River Delta and an MOU and joint work plan with Teranga Gold Corporation to increase horticultural production in Kédougou, using mines as an engine to power growth in the horticultural sector. Subcontracting arrangements with local organizations to carry out specific tasks are based on a contract or purchase order, which includes a set of deliverables and a negotiated price. Examples of USAID|Yaajeende’s subcontracting arrangements include a contract with the Union des Producteurs Horticoles de Bakel to provide expertise and technical support to producers, and a contract with the Senegalese Institute for Agricultural Research to assist with seed production for fortified millet varieties. As needs arise, USAID|Yaajeende collaborates with private firms, sometimes with an MOU in place. No funds change hands under this arrangement: firms fund their activities and USAID|Yaajeende’s contributions are limited to logistical or communication assistance. The project has close working relationship with a variety of private firms to supply CBSPs and producers with horticultural inputs, agricultural tools, irrigation equipment, and credit. 44 The quality of USAID|Yaajeende’s partnerships stems from their coverage (spanning national, regional, and local government institutions; private sector organizations; NGOs, and civil-society organizations), their relevance to project interventions, and their clearly defined roles for each partner organization. Nearly 80 percent of key informant interviewees agreed that USAID|Yaajeende’s partnerships were generally of the highest quality. Nearly all partner organizations interviewed were satisfied with their partnerships with USAID|Yaajeende. With two notable exceptions, USAID|Yaajeende has completed project activities in a timely manner. In general, USAID|Yaajeende has completed its project activities on time. Quarterly and annual reports were submitted on schedule and all work plans have been implemented as scheduled. USAID has not communicated any dissatisfaction. Every quarter, the project team delivered a presentation detailing progress to USAID; comments were addressed during those presentations. However, the environmental mitigation and monitoring plan was signed only in Year 3 (see Question 10). As part of its M&E system, USAID|Yaajeende conducted a baseline impact evaluation survey in Year 1. A mid￾term survey was scheduled for the beginning of Year 3, but it has not yet been conducted.32 Delaying the mid-term evaluation survey by more than one year has several negative implications for implementation. First, conducting the survey on schedule would have provided much-needed mid-term guidance in several areas, particularly on whether the project has succeeded in reducing undernutrition in its intervention zones and whether adjustments are needed. Second, the survey would have enabled USAID|Yaajeende to test its hypothesis that integrating agriculture and nutrition is the most effective approach for reducing undernutrition. Third, conducting the survey would have enabled our mid-term evaluation to investigate a series of questions, which could not be investigated in the absence of mid-term survey data (see Questions 4, 5, and 6). Fourth, as noted earlier, unless the project is extended, a final impact evaluation will not be needed because collecting the same impact evaluation data twice in one year would be redundant and wasteful. USAID|Yaajeende’s contributions to USAID/Senegal’s economic growth objective goals are considerable. USAID|Yaajeende is anchored in USAID/Senegal’s economic growth objective goals. USAID|Yaajeende shares USAID/Senegal’s economic growth development objective (increased inclusive economic growth) and three of its four first-level objectives (inclusive agricultural sector growth; improved nutritional status, especially for women and children; and improved management of natural resources). Its fourth first-level objective (increased trade) is partially covered through two of USAID|Yaajeende’s intermediate results (increased access to finance and improved markets). USAID|Yaajeende covers all of USAID/Senegal’s economic growth development objective intermediate results and six of its seven sub-intermediate results. Although USAID|Yaajeende developed a number of indicators to reflect specific project activities, most of its PMP indicators are identical to those measuring USAID/Senegal’s economic growth objectives. 32 The mid-term survey was rescheduled due to budget cuts in Year 3 that reduced the amount of funding available for its timely completion. 45 USAID|Yaajeende’s achievements are universally recognized by Senegalese government officials at the central, regional, and local levels, as well as by the private sector, civil-society organizations, and beneficiary households. These achievements form an integral part of USAID/Senegal’s successful economic growth objectives. The number of performance indicators used to measure results is high, with a high proportion of output indicators and an absence of direct indicators to measure capacity building and sustainability. USAID|Yaajeende’s performance indicators were reduced from 52 in Year 1, to 47 in Year 2, to 42 in Year 3. At 42, the number of indicators places a heavy burden on the project, especially when combined with a variety of process indicators — a finding emphasized by USAID|Yaajeende’s staff in the field. In addition, more than 40 percent of those indicators consist of output indicators. Nearly 50 percent consist of output indicators when excluding the four impact indicators used in the baseline survey. USAID|Yaajeende used several indicators to measure capacity building. However, the indicators selected did not provide the most direct evidence of the condition or result they were measuring. (The lack of direct capacity-building and sustainability indicators is fully addressed in Questions 1, 2, 5, 7, 8, 9 and 10.) CONCLUSION USAID|Yaajeende has reached its target population, used project resources effectively, and — except for delays in signing the Environmental Mitigation and Monitoring Plan and in conducting the mid-term evaluation survey — completed project activities in a timely manner. Its contributions to USAID/Senegal’s economic growth objectives have been considerable. However, the project’s performance indicators do not effectively capture achievements. RECOMMENDATION Just as USAID|Yaajeende’s results framework should consist of necessary and sufficient conditions to achieve the project’s overall objective, the PMP should contain no more than the necessary and sufficient indicators to measure project achievements. USAID|Yaajeende’s performance indicators should be anchored in higher-level strategic thinking about what must truly be achieved to obtain project success. The overall set should be streamlined to discard any unnecessary indicators, and new indicators should be identified to directly measure capacity building and sustainability. 46 Question 9: What is the likelihood that project approaches/practices and results will be sustained? FINDINGS USAID|Yaajeende paid special attention to sustainability; many of its interventions are likely to have lasting effects. References to sustainability abound in USAID|Yaajeende’s major reports, and project staff in Dakar and in the field have stressed its importance.33 USAID|Yaajeende’s project description states that, “It is not enough for a food security program to have impact during the time that is funded. Food security is not the same as relief; it must have an institutional capacity-building focus that will lead to a holistic and financially sustainable system that will continue after the program has left.” USAID|Yaajeende has developed necessary conditions for sustainability by promoting the adoption and dissemination of more efficient technologies and innovations and by training local farmers on how to use new varieties and cropping methods. These efforts have created a transfer of valuable knowledge. Adoption of new cultivars, improved varieties, and agricultural techniques has increased significantly, providing an early sign of project sustainability. CBSPs also ensure that the supply of agricultural inputs and services will remain available to project beneficiaries after the project ends. CBSPs have built a working relationship with credit unions and have increasingly stronger links with national and regional suppliers. The emerging transition of the community nutrition volunteer network into CBSPs is a sign of greater sustainably in providing nutrition and health-care services in the local community.34 This transition was encouraged by USAID|Yaajeende’s strategic decision to build on the existing network of community nutrition volunteers, a network of local nutrition agents set up by the Nutrition Enhancement Program and partner NGOs.35 Another sustainability factor is USAID|Yaajeende’s relevance to the Senegal National Strategy for Economic and Social Development (2013-2017). This strategy stresses food security, private sector development, agricultural development through the National Agricultural Investment Program,36 inclusive development to improve the nutritional status of mothers and children via dietary diversity, meeting the nutritional needs of vulnerable groups (under-five children, pregnant women, and nursing mothers), capacity building for institutions and local government units, and leadership and entrepreneurial skill-building for women. 33 A word count reveals that sustainability was listed 10 times in USAID|Yaajeende’s 2013 annual report and work plan, and 16 times in its project description document. 34 USAID|Yaajeende is aware that community nutrition volunteers are still largely a volunteer network, and efforts are underway to assist their transition to CBSPs, bringing them closer to sustainability. The same conclusion applies to mother-to-mother groups. 35 A World Bank-financed initiative, the Nutrition Enhancement Program, aims to enhance nutritional conditions of vulnerable populations. The project collaborated with local NGOs to extend nutrition and growth promotion into rural areas. It contracted 12 NGOs to implement 34 district-level subprojects, while collaborating closely with local governments. It mobilized health and nutrition workers, who provided growth-monitoring services, counseling to pregnant women and mothers of young children, and delivery of essential services, such as vaccination, de-worming, and micronutrient supplementation. 36 The National Agricultural Investment Program addresses the challenges that the Common Agricultural Policy of the Economic Community of West African States, the sub‐regional counterpart of the Comprehensive African Agricultural Development Program, intends to meet. 47 USAID|Yaajeende’s sustainability potential is evidenced by the level and degree of acceptability of project activities among communities and beneficiaries, as reflected in key-informant interviews and focus-group discussions. Among the 40 key interviewees and 264 focus-group participants, there was a consensus that project activities are highly relevant to the development issues that beneficiary communities face. In key informant interviews, 60 percent of the respondents (24 out of 40) agreed with the statement that USAID|Yaajeende’s approaches, practices, and results are likely to last after the project ends in 2015. A positive assessment of project sustainability was considerably higher in focus-group discussions, in which nearly 92 percent of the participants (242 out of 264) agreed with the statement and only 2 percent disagreed. USAID|Yaajeende lacks a comprehensive and systematic sustainability plan, complete with an adequate set of direct indicators to guide progress. According to USAID, sustainability is achieved “when host-country partners and beneficiaries are empowered to take ownership of development processes, including financing, and maintain project results and impact beyond the life of the USAID project.” This statement echoes other donors’ definitions.37 Sustainability has several dimensions — the most important of which is financial sustainability, or how funding streams will continue at the end of the project and provide the necessary resources to carry on project achievements. Other components of sustainability include market-driven value-chain mechanisms, local participation and ownership, and awareness building and training.38 Partners’ roles in recipient countries are essential to extending project benefits beyond the life of the project. This raises a question of the relationship between sustainability and institutional capacity building. USAID|Yaajeende has raised awareness and provided training as a major method for building the capacity of local organizations, particularly in water and sanitation (see Question 7) and governance (see Question 10). Raising awareness and providing training are two critical pieces of the organizational development objective. However, these inputs will not be effective without a series of structured and integrated processes to remove barriers preventing institutions from achieving their goals and objectives (see, for instance, USAID 2010a; USAID 2011; and World Bank 2012). According to USAID’s Human and Institutional Capacity Development (HICD) guidance, training and other HICD interventions’ success is measured by improvement in organizational output and performance, and not simply by the number of individuals trained. Through a process of identifying performance gaps and designing performance solutions with clear goals and milestones to fill these gaps, organizations can achieve sustainability goals. USAID’s guidance also states that project teams and implementers “must build in monitoring and evaluation in order to track a project’s results in achieving sustainable outcomes during implementation.” The shortcomings of USAID|Yaajeende’s sustainability indicators have been analyzed in other questions (see Questions 1, 2, 7, and 10) and will not be repeated here. However, it should be emphasized that 37 For instance, sustainability is the “continuation of benefits after a major assistance from a donor has been completed” (Australian Development Agency); or the “durability of positive program or project results after the termination of the technical cooperation channeled through that program or that project” (UNDP). 38 See USAID Project Design Sustainability Analysis Tool (undated). 48 according to USAID guidance (USAID 2010b), performance indicators should be “adequate” (sufficient to measure the stated result) and “direct” (clearly measure the intended result). 39 USAID|Yaajeende’s capacity building and sustainability indicators do not feature these two criteria. CONCLUSION Sustainability is achieved when host-country partners and beneficiaries maintain project results beyond the life of the project. USAID|Yaajeende has paid special attention to project sustainability from the outset. Ample qualitative evidence and quantitative results show that many of the project’s agricultural and nutrition activities are likely to have lasting effects. However, USAID|Yaajeende did not elevate sustainability to the results-framework level or develop a comprehensive and clearly articulated sustainability plan from the start. It also did not identify adequate and direct sustainability indicators to monitor and evaluate progress in a more rigorous way. RECOMMENDATION USAID|Yaajeende should develop a comprehensive and systematic stand-alone sustainability plan. Experience from other countries (see, for instance, Rogers 2004 and 2012) demonstrates that to achieve optimal results, sustainability should be an integral part of project design and should be embedded throughout implementation. The private sector, community organizations, and other stakeholders should be aware of their post-exit roles and responsibilities from the outset. USAID|Yaajeende’s sustainability plan should at a minimum include: decisions about approach (phase-out and gradual phase-over), explicit benchmarks for progress and timelines, clear allocation of responsibilities, graduation criteria and progressive withdrawal of project support, and capacity building of local community and government organizations to progressively take up the management and provision of project services. We understand that not all activities are expected to be fully sustainable at project conclusion. When this is the case, the sustainability plan should define the degree of sustainability considered essential to those activities’ success. The objective of the sustainability plan is to spell out what results will continue and how sustainability will be targeted and measured. It should describe the process through which the movement toward sustainability will occur, and specify the ways in which the required early and intermediate outcomes achieving the next higher order of change will be brought about and documented. The sustainability plan should contain explicit benchmarks for progress, timelines, and a set of performance indicators to measure results. Following USAID guidelines, these indicators should be adequate, direct, and cost-effective. 39 For example, if the desired result is a reduction in teen pregnancy, the number of teenage girls receiving pregnancy counseling services would not be an optimal measure for this result (however well it may measure the service delivery necessary to reduce pregnancy rates). Achievement would be more directly measured by an outcome indicator such as the pregnancy rate among teens. 49 Question 10: What are the outcomes of the project’s approach to addressing gender, environmental compliance, and governance issues? FINDINGS Gender integration has been a central pillar of USAID|Yaajeende’s interventions. According to USAID, “Gender is a social construct that refers to relations between and among the sexes, based on their relative roles. It encompasses the economic, political, and socio-cultural attributes, constraints, and opportunities associated with being male or female…Gender Integration entails the identification and subsequent treatment of gender differences and inequalities during program/project design, implementation, monitoring, and evaluation” (USAID 2010e).40 Thus, mainstreaming a gender perspective involves assessing the implications for men and women of any planned interventions and integrating those implications into project design, implementation, and M&E. In this context, gender does not refer to biological and physiological characteristics that define men and women, but to the social roles that society assigns to each. USAID|Yaajeende has broadened women’s equitable participation in project interventions. USAID|Yaajeende is driven by three major considerations defining the status of women: women’s role in the agricultural sector, Senegal’s land tenure system and its implications on women, and women’s role in reducing undernutrition. Women play an active role in agriculture, from working in the field to participating in postharvest, processing, and marketing. However, their economic opportunities remain limited, as they have relatively limited access to new technologies and inputs and restricted control over family businesses. Although the 1996 Law on Decentralization entitled women to inherit land, many have not benefitted due to entrenched customary practices. High undernutrition rates among women, as well as women’s role in determining the nutritional status of their young children, shows that gender considerations must be taken into account to improve the targeted population’s nutritional status. 40 USAID’s definition echoes gender mainstreaming guidelines developed by other bilateral and multilateral organizations (see, for instance, ILO 2010; World Bank 2002). 50 USAID|Yaajeende has developed equalizing strategies that promote increased access to resources and opportunities for women. Women are USAID|Yaajeende’s primary target; nearly all of the project’s activities have focused on women as principal beneficiaries. Considered as two of the most vulnerable and food-insecure groups, pregnant and lactating women and their under-five children were USAID|Yaajeende’s primary targets in maternal and child health and nutrition, and they benefitted the most from project interventions in this area. Women received extensive training in mother and child health and nutrition (nearly 2,500 mother-to￾mother groups were formed, totaling more than 29,000 members). They also received training on restoring and managing biodegraded land (75 percent of the 15,000 participants were women). Women also accounted for more than 90 percent of producers who received training on horticultural production. Nearly 44,000 women (more than 70 percent of the target population) received nutrition training via community meals. Women acquired new assets through the pass-on-the-gift program (nearly 17,000 animals were placed, and approximately 80 percent of the placements were for women) and through reclamation of biodegraded land (nearly 240 hectares in Years 2 and 3 were attributed to women). Women now have increased access to credit. For instance, out of the 254 credit applications received in 2013, a total of 225 were funded. This credit benefitted 814 men and 2,277 women (nearly 74 percent of the credit recipients). Women have also benefited from off-farm employment opportunities (nearly 44 percent of CBSPs are women) and have a greater voice in governance matters (more than 60 percent of governance activity participants in Year 3 were women). 51 USAID|Yaajeende has complied with environmental regulations procedures, but its Environmental Monitoring and Mitigation Plan was signed with significant delay. Every USAID project is required to comply with the U.S. government’s environmental regulations process and procedures, defined in Chapter 22, Part 216, of the U.S. Code of Federal Regulations (22 CFR 216 ). These regulations describe USAID’s Environmental Impact Assessment (EIA) to identify potential effects — positive or negative — that a project or activity may have on the environment. The EIA contains plans to monitor and mitigate any negative effects and prevent activities that are likely to cause significant environmental harm. The primary objective is to ensure that project teams monitor and mitigate negative environmental effects. The EIA process starts with an initial screening that divides project activities into four categories: Exempt, Categorical Exclusion, Initial Environmental Examination (IEE) Required, and IEE Required and High Risk. No environmental documentation is required for exempt activities. Exemption forms are required for categorical exclusion, such as when activities are considered to have such a low risk of adversely affecting the environment that they are categorically excluded from environmental review. 52 If any project activities are not exempt or categorically excluded, an IEE is conducted. An IEE reviews the reasonably foreseeable environmental effects of a proposed project or activity. It is used as factual basis to decide whether to require a more comprehensive environmental assessment or to proceed with the project using plans to address and monitor detrimental effects on the environment, as outlined in the Environmental Mitigation and Monitoring Plan. USAID considers some types of activities to have such a high risk of adversely impacting the environment that such activities automatically trigger a detailed environmental assessment. Such activities include using pesticides and developing new land. As noted above, USAID|Yaajeende has complied with USAID’s environmental regulations procedures. However, the Environmental Monitoring and Mitigation Plan was signed only in Year 3 of project implementation. Governance is at the core of USAID|Yaajeende. USAID|Yaajeende’s governance project consists of a series of interventions (sequentially implemented) to train local governments and community leaders about food security, including food availability, access, utilization, and governance; to establish citizen working groups for food security, a multi-step process consisting of awareness campaigns and mapping of community-based organizations and civil-society organizations; and to hold a general assembly to nominate and elect citizen working group members. USAID helps each elected citizen working group and each rural community to draft a food security plan. The security plan is prepared via a participatory process. Participants assess community food security initiatives, identify strategic areas of improvement, establish inclusive governance structures to enable long-term partnerships between citizen working groups and the government, identify synergies with local development plans, analyze existing policy frameworks, and draft an annual work plan to implement food security plans. Once the annual food security work plan is completed, USAID|Yaajeende personnel embark on an intensive effort to assist citizen working group members in monitoring and evaluating field activities. They conduct regular monthly coordination meetings with CBSPs, community nutrition volunteers, and producer organization representatives to evaluate activities. Activities include reclamation of biodegraded land, conservation agriculture, communication campaigns, land tenure deeds for women’s groups, and town-hall meetings and budget hearings with the local government. By Year 2, all rural communities had received training in food security, focusing on techniques for encouraging citizen participation, organizing village and community meetings, evaluating seasonal activities, preparing operational action plans, and addressing advocacy issues. Training sessions in 24 communities involved more than 300 participants, 66 of whom were women. By the end of Year 3, all 25 rural communities had formed citizen working groups. In the same period, all 25 rural communities had drafted and validated their food security plans. Many food security plans were also in the process of implementation. 53 To date, USAID|Yaajeende’s governance program achievements do not demonstrate that the current structure is necessarily sustainable. USAID|Yaajeende trained rural communities on food security issues and assisted them in drafting local food security plans. However, these efforts do not ensure the sustainability of the nascent governance structure. As mentioned in Question 9, sustainability has several dimensions, including local participation and ownership, awareness building, HICD, and financial sustainability. While progress has been made in some of these areas — such as training and technical assistance for citizen working groups in internal governance, resource mobilization, financial management, proposal writing, and transparency and accountability — much needs to be done to make the citizen working group model sustainable. In particular, institutional capacity building and financial resources to carry on project achievements.41 CONCLUSION USAID|Yaajeende has successfully promoted increased access to resources and opportunities for women and incorporated governance as a central pillar of its interventions. However, much remains to be done to sustain the nascent governance structure. RECOMMENDATION USAID|Yaajeende should conduct a thorough evaluation of the citizen working groups and monitor their progress toward sustainability. This evaluation would be based on a series of steps defined by USAID|Yaajeende based on its knowledge of the current citizen working group structure. Possible steps could include:  Defining the desired performance of the citizen working groups based on a manageable set of objectives. Desired performance should be based on a series of project-defined performance factors (i.e. indicators). These indicators may include incentives, internal management structure, and resource mobilization.  Assessing actual performance in relation to desired performance based on direct observations and interviews with citizen working group members, as well as record and document reviews.  Assessing the performance gap as the difference between desired and actual performance; the performance gap should be measured against indicators used to define the desired performance.  Analyzing the most fundamental reasons or root causes for the performance gap.  Identifying performance solutions to address the root causes of the performance gaps, as well as related indicators or performance factors.  Implementing performance solutions. 41 USAID|Yaajeende is aware of the many challenges facing the sustainability of the citizen working groups, particularly their financial sustainability (see USAID|Yaajeende undated). 54  Monitoring and evaluating performance solutions by re-measuring the performance gap against the desired level of sustainability. LESSONS LEARNED There is increasing evidence that agriculture affects nutrition. However, there are very few examples to demonstrate how agricultural livelihood programming could be used to improve nutrition and identify opportunities for scaled-up nutrition across the range of current agricultural interventions. USAID|Yaajeende is one of the longest-running FtF projects to date that has pioneered nutrition-led agriculture, an approach based on the premise that agriculture can do more to improve nutrition and health than providing the poor with a steady supply of staple crops that meet calorie requirements at low prices. Three years into implementation, USAID|Yaajeende has contributed to the evidence base on how nutrition-led agricultural interventions can be successful and how future FtF projects could build on preliminary lessons learned from the USAID/Senegal experience. Relevance of project activities to host-country government priorities is essential. USAID|Yaajeende’s interventions are in line with the Senegal National Strategy for Economic and Social Development, the National Food Security Plan, the Lettre de Politique de Développement de la Nutrition, and the National Agricultural Investment Plan ― all of which stress food security, private sector development, and inclusive development to improve the nutritional status of mothers and children via dietary diversity and by meeting the nutritional needs of vulnerable groups. The National Agricultural Investment Program also addresses the challenges that the Common Agricultural Policy of the Economic Community of West African States, the sub‐regional counterpart of the Comprehensive African Agricultural Development Program, intends to meet. Such alignment has facilitated collaboration with national institutions (for example, to introduce and disseminate improved varieties), local government involvement (for example, in widespread planting and domestication of forest-tree species), and collaboration with technical government personnel and extension agents at field level. Acceptability of project activities among communities and beneficiaries is necessary to the success of field activities. The level and degree of USAID|Yaajeende’s acceptability is evidenced by the consensus among interviewees and focus-group participants that project activities are highly relevant to the development issues that beneficiary communities face. Gender mainstreaming helps realize the potential of agricultural development to deliver gender￾equitable health and nutritional benefits to the poor. A key element of USAID|Yaajeende’s success has been its ability to broaden women’s equitable participation in project interventions and to develop innovative equalizing strategies that promote increased access to resources and opportunities for women. 55 High-quality partnerships are a key determinant of project success. Three major characteristics determine partnership quality: relevance to project interventions, clearly defined roles and responsibilities for each partner organization, and the extent to which suitable partnerships cover key project activities.  Two examples that illustrate the relevance of USAID|Yaajeende’s partners to project activities include the partnership with the Senegalese Institute for Agricultural Research to introduce and field-test new varieties, seed multiplication, and germplasm banking; and the close working relationship with a variety of private firms to supply CBSPs and producers with horticultural inputs, agricultural tools, irrigation equipment, and credit.  USAID|Yaajeende partnerships are based on detailed MOUs, joint work plans and joint M&E activities, specific deliverables via subcontracting, and collaboration.  USAID|Yaajeende’s partnerships span national, regional, and local government institutions; private sector organizations; NGOs; and civil-society organizations. The private sector plays a key role in nutrition-led agriculture. Relying on the private sector has been at the core of USAID|Yaajeende’s success. The private sector-driven CBSP model has been particularly effective because it consists of entrepreneurs motivated by a quest for financial success. Project success is also apparent in the emerging transition of the community nutrition volunteer network into CBSPs. Attention to demand and supply factors is a major determinant of program success and sustainability. USAID|Yaajeende has not only created service demand (as evidenced by high adoption of the new technologies and farming practices it has promoted among beneficiary communities), but it has also paid attention to supply factors — the ability of CBSPs to deliver their services effectively and to ensure that the supply of agricultural inputs and services remain available to program beneficiaries after the program ends. Demand and supply factors play an equally important role in nutrition: even women with enhanced knowledge of good child nutrition practices may be limited in their ability to act on their knowledge if they lack access to nutrition-related products and services. Sustainability should be an integral part of project design and should be embedded throughout implementation. To be most effective, projects should develop comprehensive and systematic sustainability plans. The sustainability plans should include: decisions about approach (phase-out and gradual phase-over), explicit benchmarks for progress and performance indicators, clear allocation of responsibilities, graduation criteria and progressive withdrawal of project support, and capacity building of local community and government organizations to progressively take over the management and provision of project services. When certain activities are not expected to be fully sustainable at project conclusion, the sustainability plan should define the degree of sustainability considered essential to those activities’ success. The objective of the sustainability plan is to spell out what results will continue and how sustainability will be targeted and measured. It should describe the process through which the movement toward sustainability will occur, 56 and specify the ways in which the required early and intermediate outcomes achieving the next higher order of change will be brought about and documented. 57 Annex I: Evaluation Statement of Work I. Purpose and Use of the Evaluation The purpose of this mid-term evaluation is to assess progress to date and identify improvements that will facilitate the attainment of planned results of the USAID/Yaajeende Project. Specifically, the evaluation team will review and assess the adequacy 1 of the two components of the project stated in Section II within the context of the Senegal FTF strategy, the soundness of the project’s approaches, the quality of USAID/Yaajeende overall management, the adequacy and efficiency2 of CLUSA/NCBA’s assistance delivery, the beneficiary coverage and response, and the overall potential of sustaining the project results beyond September 2015. In addition to the evaluation findings, USAID/Senegal expects that good practices, lessons learned, and recommendations will provide ways to maintain momentum, scale-up the activities according to the FTF strategy, make the most suitable/strategic modifications in the project given realities of budget, and guide the Ministries of Agriculture and Health, and USAID in future programming. After about 35 months of implementation (November 2010 to September 2013), a comprehensive look at these components will help to identify strengths and needed adjustments to the intervention approaches, nature of services, and efficiency with which assistance is delivered. The team conducting this evaluation will gather a wide range of background information from USAID/Senegal, USAID/Yaajeende, Ministry of Agriculture (ISRA, ITA, and other Ministry of Agriculture offices), Ministry of Health, Nutrition Enhancement Program (NEP), and Cellule de Lutte contre la Malnutrition (CLM) staff, local partners, and beneficiaries to ensure that the findings, conclusions, and recommendations are based on an accurate understanding of the project and that multiple perspectives have been consulted to pave ways for utilization of the evaluation results. The evaluation report will be widely disseminated for the use of all stakeholders. II. Context Senegal has a current Global Hunger Index (GHI) of 17.3 which is classified as "serious" by the International F o o d P o l i c y R e s e a r c h I n s t i t u t e (IFPRI). Food insecurity in Senegal affects anywhere from 15.6%3 - 24% 4 of the population, with higher numbers concentrated in the northern and eastern rural regions. 1 Adequacy is meant to assess relevance of the project’s components. Are those are the right mix? 2 Efficiency is meant to ascertain whether the project is delivered in a timely and cost effective manner 3 http://www.countrystat.org/country/sen/documents/docs/Etude%20sur%20evolution%20du%20Secteur%20Agricole.pdf 4 Food security strategy. 58 According to Senegal’s National Food Security Plan, agricultural development in Senegal faces many challenges and constraints: limited use of agricultural inputs; low soil fertility; limited access to agricultural credit; few stores of good-quality seeds; obsolete farming equipment; inadequate storage and processing infrastructure; un-organization of marketing networks; and monopolies within key agricultural value chains.5 Senegal's Lettre de Politique de Développement de la Nutrition outlines further constraints with regards to nutrition: insufficient food availability and access; lack of labeling and certification of food quality; few technical specialists in nutrition; limited involvement of communities in the development of nutrition programs; absence of a nutritional information system; and socio- cultural factors with regards to traditional nutritional practices. Senegal has taken actions to alleviate the constraints that are listed above. The Government of Senegal (GOS) National Food Security Plan, the Lettre de Politique de Développement de la Nutrition, and the Country Investment Plan (CIP) for Agriculture evidence the Senegal’s commitment to reduce poverty and fight against hunger. USAID/Senegal supports GOS efforts through the Feed the Future (FTF) strategy, which is based on the development hypothesis that poverty and hunger can be sustainably reduced through transforming the national agriculture sector and nutritional status of the Senegalese population, especially women and children, through focused and scaled investment priorities. In Senegal, FTF focuses on the following five areas: 1. Agriculture driven economic growth – productivity increases through a value chain approach and promotion of sound land management; 2. Household behaviors that promote optimal nutrition; 3. Enhanced policy implementation; 4. Strengthened rural infrastructure and access to finance; 5. Increased human resource capacity, both at health facilities and every level of the agricultural sector including associated institutions. The development hypothesis that underpins the Yaajeende project is that an integrated approach to agriculture, economic growth and nutrition can decrease the rate of under-nutrition much faster than simply focusing on agriculture/economic growth alone. Greater food security will be achieved as rural populations (1) shift from subsistence agriculture to commercial agriculture via thriving small agro￾enterprises and participation in dynamic markets and value chains; (2) use more nutritional and fortified foods and adoption of best practices; and (3) work together with local governments and citizen groups to govern food and food related resources in a transparent, participative way. 5 http://siteresources.worldbank.org/INTPRS1/Resources/Senegal-PRSP(Sept2007).pdf and Food Security Strategy 59 USAID/Yaajeende agriculture interventions focus on helping both emerging and ultra-poor farmers within a Communaute Rurale (CR) to be organized into Producer Groups (PGs) or Producer Organizations (POs), enabling them to take advantage of economies of scale and gain access to new skills, technologies and financial resources that USAID/Yaajeende introduces via its Community Based Service Provider (CBSP) network. The CBSPs also serves as a private sector driven extension service. It also helps PGs and POs establish linkages to important regional and national actors and lead firms within key value chains. Over the course of the project, farmers will produce more food for local consumption and, at the same time, become stakeholders in dynamic new agro-enterprises that improve the household revenues of owners and workers, and lead to the increased availability and access to food. USAID/Yaajeende nutrition interventions focus on increasing public demand for nutritious foods via education, training and behavior change communications and improving the supply of diverse, nutritious foods, including fortified foods through local agents. USAID/Yaajeende works to strengthen the existing network of Community Nutrition Volunteers (CNVs) set up by the Nutrition Enhanced Program (NEP). These CNVs furnish information to mothers and children on nutrition best practices, provide guidance to PGs and POs about what foods to cultivate, and link people to public nutrition services. USAID/Yaajeende is a five-year Feed the Future Food Security project designed to reduce malnutrition in the regions of Matam and Kédougou and the Department of Bakel-an area that represents the northeastern 1/3 of Senegal. The goal of USAID/Yaajeende is to accelerate the participation of the very poor in rural economic growth and to improve the nutritional status. The long term objective of the project is to double food production, reduce the vulnerability to food shocks, improve nutrition status, and double intra-regional trade. The project is a comprehensive, structural approach to food security organized according to the four pillars 6 of food security: Availability, Access, Utilization and Governance (also referred to as Stability). The two components of the project are stated below: 1. Accelerate the Participation of the Rural Poor in Rural Growth; 2. Reduce Undernutrition. For more details, see the USAID/Yaajeende Project Results Framework in Annex A and a list of illustrative indicators in Annex B. 6 A country's food security level is calculated by examining several interrelated dimensions: (1) the availability of sufficient quantities of diverse, nutritious foods; (2) the population's ability to access these diverse, nutritious foods through gift or purchase; (3) the ability of the population to optimize the utilization of food to maximize health; and (4) the government's ability to effectively govern food stores and food related resources. 60 The table below provides a summary of the project. Yaajeende Project General Information Project Name USAID/Senegal Yaajeende Agricultural Development Project Cooperative Agreement Number 685-A-00-10-00002-00 Period of Agreement November 1, 2010 – September 30, 2015 Funding $39,999,065 Implementing Organization Cooperative League of the USA/National Cooperative Business Association Agreement Officer’s Representative Papa Nouhine Dieye, USAID/Senegal’s Agricultural Specialist III. Issues and Questions to be Investigated The Evaluation Team will investigate the extent to which the implementing partner has contributed to USAID/Senegal/EGO IR 1: Inclusive agriculture sector growth and IR 4: Improved nutritional status especially of women and children. Specifically, the evaluation will measure achievements made through the four pillars of the two USAID/Yaajeende components listed in section II above, plus certain cross-cutting themes such as gender equity, climate change, governance, science, technology and innovation. The Evaluation Team will answer the following questions related to cross-cutting themes and the two components of the project. Cross-Cutting Questions:  To what extent has the project been implemented effectively, including timely completion of project activities, effective use of project resources, reach of target groups/beneficiaries, quality of partnerships and collaboration, and contribution to overall USAID/Senegal EGO goals?  What is the likelihood that project approaches/practices and results will be sustained?  What are the outcomes of the project approach to address gender, environmental compliance, and governance issues? Component 1: Accelerate the participation of the rural poor in rural growth Has the project demonstrated effective, efficient and sustainable vehicles/approaches for promoting adoption of innovation (technology, practices, behaviors) and diffusion of products and new technologies among the poor, women, and socially marginalized? Has the CBSP model proven to be an effective and a sustainable private sector driven approach to reduce under-nutrition in targeted areas? How the activities to increase household assets and income led to improved participation of the rural poor in rural growth? 61 Component 2: Reduce undernutrition What project activities have positively enabled value chain investments to lead to improved consumption of diverse diets and quality foods? What investments in human and institutional capacity development have effectively generated large scale nutrition outcomes? To what extent the integrated nutrition and agriculture approach led to the reduction of under￾nutrition among the target population? To what extent the project’s water and sanitation activities led to improved healthy behaviors of the target population? IV. Methodology The project period to be reviewed ranges from November 2010 to September 2013. 1. Evaluation design Since this is a performance evaluation, a non-experimental design seems to be more appropriate. The evaluation team shall propose its own methodology, however it is expected that the evaluation will be implemented through document review, key informant interviews, and focus group meetings. We have annexed (Annex C) an illustrative evaluation design matrix. Offerors are requested to fill this matrix as much as they can; based on their proposed approach to this evaluation. The Evaluation Team will develop the specific approach to evaluation methodology and analytical framework, a detailed sampling plan, and data collection tools (e.g. survey and interview questionnaires, discussion guide for focus groups, etc.). In developing the methodology, USAID/Senegal expects that the analysis will consider all Section III components, topics and issues to be investigated. 2. Data sources and collection methods The Evaluation Team shall familiarize itself with USAID and project documentation. USAID/Senegal will ensure that all relevant documents are available to the Team prior to the field work. The documents will include, but will not be limited to:  USAID/Senegal Feed the Future (FTF) strategy  Government of Senegal Agriculture Country Investment Plan; Stratégie Nationale de Sécurité alimentaire du Sénégal; USAID/Yaajeende agreement  USAID/Yaajeende annual work plans, annual and quarterly reports; Performance Management Plan prepared by USAID/Yaajeende; Sector action plans and reports  Training manuals, field trip reports; and other documents, as appropriate and/or required. 62 The Evaluation Team will review documentation provided by USAID/Senegal and USAID/Yaajeende as well as other available information. The team will conduct in-depth interviews with beneficiaries and project staff, and analysis of project results. The Evaluation Team will meet and interview representatives from USAID/Senegal, USAID/Yaajeende, the Government of Senegal (Ministry of Agriculture and Ministry of Health), other donors, and other stakeholders in Dakar and in targeted localities. 3. Data analysis methods The Evaluation Team should include approaches to data disaggregation and integrate gender in data analysis. 4. Limitations of the methodology The Evaluation Team should include to the reports (draft and final) any methodological limitations to the evaluation. V. Illustrative tasks and timeframe The tasks in this SOW will be implemented over a period of about 10 weeks (6 –working days per week), starting around December 2013. The schedule below is illustrative and will be discussed and revised as required: Literature review and development of evaluation methodology (2 weeks) Interviews and Field visits (4 weeks) Data Analysis and Draft evaluation report (3 weeks) One-day debriefing of findings at USAID/Senegal (1 working day) Final evaluation report (1week) Tasks Dates/Time Frame Literature Review December 2013 Meeting with USAID to discuss the evaluation SOW and finalize evaluation questions December 2013 Draft work plan, methodology and tools submitted to USAID/Senegal December 2013 USAID review of work plan, including data collection methods/ tools December 2013 Meeting with USAID/Senegal to (a) discuss the draft work plan; (b) review and confirm planned dates of submission of deliverables; and (c) brainstorm on key accomplishments, weaknesses, opportunities and threats December 2013 Meetings with USAID/Yaajeende to (a) review the information sources and contact list; (b) discuss appointment dates and times; and (c) brainstorm on key accomplishments, weaknesses, opportunities and threats December 2013 Final Work plan, methodology and tools Submitted for approval December 2013 63 Tasks Dates/Time Frame Meetings with key informants: GOS representatives (Ministry of Health, Ministry of Agriculture), ISRA, ITA, CLM, DANSE, NEP, Private sector, producers organizations, other USAID implementing partners, etc… December 2013 – January 2014 Field travel and data collection December 2013 – January 2014 Drafting brief summary of key findings January 2014 Draft report submitted to USAID/Senegal January 2014 Debriefing for USAID/Senegal and Stakeholders February 2014 Full draft report submission February 2014 Feedback from USAID/Senegal and USAID/Yaajeende February 2014 Revisions to Report and Submission of final report February 2014 64 Annex II: Qualitative Data Collection Instrument Date _____________________________________________________________ Location __________________________________________________________ Interviewer(s) ______________________________________________________ Respondents (separate list) 1. a. The project has promoted the adoption of innovations (such as new technologies, new practices and new behaviors) and has disseminated new products and new technologies among the poor, women, and socially marginalized. And those approaches have produced their intended effects. Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 1. b. The newly adopted technologies, practices and behaviors will last after the project ends in 2015. Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 2. a. The community-based solution providers have been effective in reducing undernutrition in the targeted areas Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why 65 Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 2.b. The community-based solution providers will last after the project ends in 2015 Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 3. The project has had a positive impact on the rural poor in terms of increased household assets and income Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 4. a. Project activities have resulted in more diversified diets and consumption of higher quality foods in the project areas Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why 66 Number of participants: ________Men _________Women Reasons (list): 4. b. What are in your view the project activities that have contributed the most to more diversified diets and consumption of higher quality foods in the project areas? Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 5. a. Project activities to enhance human and institutional capacity development have generated positive large-scale nutrition outcomes Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 5. b. What project activities to enhance human and institutional capacity development have been most effective in generating positive large-scale nutrition outcomes? Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 67 6. a. The fact that the project has integrated both nutrition and agriculture in its activities has reduced undernutrition among the target population Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 6. b. Limiting project activities to nutrition would have led to higher reduction in undernutrition among the target population Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 6. c. Limiting project activities to agriculture would have led to higher reduction in undernutrition among the target population Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 7. Project activities in water and sanitation have led to improved health behaviors among the target population 68 Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 8. a. The project has used project resources effectively Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 8. b. Project partnerships and collaboration have been of high quality Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 9. Project approaches, practices and results are likely to last after the project ends in 2015 Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why 69 Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 10. Project activities have been highly beneficial to women Agree (indicate number) ______________________ Disagree (indicate number) ____________________ Don’t know (indicate number) __________________ If you agree, please explain why Number of participants: ________Men _________Women Reasons (list): If you disagree, please explain why Number of participants: ________Men _________Women Reasons (list): 70 Annex III: Bibliography of Documents Reviewed Alderman, Harold and John Hoddinott. 2014. The Economic Rationale for Investing in Undernutrition. The Guardian, Tuesday 28 January 2014 (http://www.theguardian.com/global-development-professionals￾network/2014/jan/28/child-malnutrition-stunting-cost-effective-interventions/print). Accessed 6/2/14. Alderman, H. and J. Behrman. (2006). “Reducing the Incidence of Low Birth Weight In Low-Income Countries Has Substantial Economic Benefits.” World Bank Research Observer 21 (1): 25-48. Bilinsky P. and Swindale A .2006. Score de Diversité alimentaire des Ménages (SDAM) pour la mesure de l’accès alimentaire des Ménages : Guide d’indicateurs: Projet d’assistance technique en matière d’Alimentation et de Nutrition, l’Académie pour le Développement de l’Education, 14 pages Washington, D.C Canadian International Development Agency (CIDA). 2001. Assessing Sustainability. Di Vinadio, Tommaso Balbo, Priyanka Sinha, and Paramjit Sachdeva. 2013. Strengthening Inclusive Ownership through Capacity Development: Operational Lessons from Case Studies. World Bank Institute Capacity Development and Results. Government of Senegal. 2011. Programme Nationale d’Investissement Agricole (PNIA), Plan d’Investissement 2011-2015. Horton, S., H. Alderman, and J. Rivera. (2008). “Hunger and Malnutrition.” Copenhagen Consensus 2008 Challenge Paper. Copenhagen Consensus Center, Copenhagen, Denmark. Horton, S., H. Alderman, and J. Rivera. (2008). “Hunger and Malnutrition.” Copenhagen Consensus 2008 Challenge Paper. Copenhagen Consensus Center, Copenhagen, Denmark. IFPRI. 2013. Global Hunger Index: The Challenge of Hunger: Building Resilience to Achieve Food and Nutrition Security. Kennedy, Gina, Terri Ballard and Marie Claude Dop. 2011. Guidelines for Measuring Household and Individual Dietary Diversity. Nutrition and Consumer Protection Division, Food and Agriculture Organization of the United Nations. Menon, P., and M. T. Ruel, with (in alphabetical order) M. Arimond, J.-P. Habicht, B. Hankebo, C. Loechl, J. Maluccio, M. N. Mbuya, and G. Pelto. 2007. Prevention Is Better than Cure. Final Report of the Evaluation: Prevention or Cure? Comparing Preventive and Recuperative Approaches to Targeting Maternal and Child Health and Nutrition Programs in Rural Haiti. Washington, DC: Food and Nutrition Technical Assistance Project. Rassas, Bechir, Louis Herns Marcelin, Bernard Crenn, and Felipe Tejeda. 2014. The Haiti Title II Multi-Year Assistance Program (MYAP): Final Evaluation. 71 Rogers, Beatrice Lorge, Jennifer Coates, Katie Houk, Elizabeth Kegode, Leslie Sanchez. 2012. TOPS FNS Knowledge Management Workshop, Addis Ababa, June 11, 2012Exit Strategies: How to Ensure Sustainability of Impact after Program Exit. Rogers, Beatrice Lorge and Kathy E. Macias. 2004. Program Graduation and Exit Strategies: Title II Program Experiences and Related Research. Washington, D.C.: Food and Nutrition Technical Assistance (FANTA) Project, Academy for Educational Development (AED). Ruel, Marie T, Megan Deitchler, and Mary Arimond. 2010. Developing Simple Measures of Women’s Diet Quality in Developing Countries: Overview. J. Nutr. November 1, 2010 vol. 140 no. 11 2048S-2050S. Ruel, Marie T. 2003. Operationalizing Dietary Diversity: A Review of Measurement Issues and Research Priorities. J. Nutr. November 1, 2003 vol. 133 no. 11 3911S-3926S. Sjoblom, Mirja, Alix Beith, and Rena Eichler. March 2012. Performance-Based Incentives for Child Health: Taking Stock of Current Programs and Future Potentials. Bethesda, MD: Health Systems 20/20, Abt Associates. Swindale, Anne and Paula Bilinsky. 2006. Household Dietary Diversity Score (HDDS) for Measurement of Household Food Access: Indicator Guide (v.2). Washington, D.C.: FHI 360/FANTA. World Bank. 2012. Guide to Evaluating Capacity Development Results: A Collection of Guidance Notes to Help Development Practitioners and Evaluators Assess Capacity Development Efforts. Washington, DC: World Bank. _________. 2006. Repositioning Nutrition as Central to Development: A Strategy for Large-Scale Action. Washington, DC: World Bank. _________. 2002. Integrating Gender into the World Bank’s Work: A Strategy for Action. Washington, DC: World Bank. _________. 2001. Annual Review of Development Effectiveness, From Strategy to Results. Washington, DC: World Bank. United States Agency for International Development (USAID). 2011a. Performance Monitoring and Evaluation, TIPS Number 15: Measuring Institutional Capacity. _________. 2011b. USAID Evaluation Policy, USAID, 1300 Pennsylvania Avenue, NW, Washington, D.C. _________. 2010a. Human and Institutional Capacity Development Handbook, A USAID Model for Sustainable Performance Improvement. _________. 2010b. Performance Monitoring and Evaluation, TIPS Number 6: Selecting Performance Monitoring Indicators (second edition). _________. 2010c. Performance Monitoring and Evaluation, TIPS Number 7: Preparing a Performance Management plan (second edition). 72 _________. 2010d. Performance Monitoring and Evaluation, TIPS Number 13: Building a Results Framework (second edition). _________. 2010e. Guide to Gender Integration and Analysis: Additional Help for ADS Chapters 201 and 203. (http://www.usaid.gov/sites/default/files/documents/1865/201sab.pdf) _________. 1988. A Conceptual Framework for Institutional Sustainability. USAID and the University of Maryland, International Development Management Center (http://pdf.usaid.gov/pdf_docs/PNABF611.pdf) _________. Undated. Project Design Sustainability Analysis Tool (http://pdf.usaid.gov/pdf_docs/pnadz042.pdf) (accessed 6/4/2014) USAID/FFP. 2012. Questions and Answers on Health and Nutrition Programming: Title II Development Programs. _________. 2010. PM2A, Title II Technical Reference Materials. USAID/Senegal. 2012. Senegal Country Development Cooperation Strategy (2012-2016) _________. 2011. Feed the Future, Senegal. FY 2011-2015 Multi-Year Strategy USAID|Yaajeende. 2013a. Annual Report 2013 _________. 2013b. Survey report. _________. 2013c. Performance Monitoring Plan _________.2013d. Convention de partenariat entre le projet Yaajeende et Equi Plus. _________.2013e. Convention de partenariat entre le projet USAID|Yaajeende et le Centre Régional de Recherches en Ecotoxicologie et Sécurité Environnementale (CERES). _________.2013f. Protocole d’Accord entre le projet USAID|Yaajeende et le CAFFAL et le FREDDAS _________.2013g. Baseline Study Report _________.2013h. Project Environmental Monitoring and Mitigation Plan _________. 2012a. Annual Report. _________.2012b. Convention de partenariat tripartite entre le projet USAID|Yaajeende et le réseau Aster￾International-ONG Sahel. _________.2012c. Convention de partenariat entre le projet USAID|Yaajeende et ETS Adiou SENE. _________.2012d. Convention de partenariat entre le projet USAID|Yaajeende et la société Agripro. _________.2012e. Convention de partenariat entre le projet USAID|Yaajeende et Agrophytex. _________.2012f. Convention de partenariat entre le projet USAID|Yaajeende et la société TROPICASEM. 73 _________.2012g. Convention de partenariat entre le projet USAID|Yaajeende et SENCHIM. _________.2012h. Protocole de partenariat entre le projet USAID|Yaajeende et le Service Régional de l’Elevage de Matam. _________.2012i. Convention de partenariat entre le projet USAID|Yaajeende et ETS Matrix. _________.2012j. Convention de partenariat entre le projet USAID|Yaajeende et l’Union des Mutuelles du Partenariat pour la Mobilisation de l’Epargne et le Crédit au Sénégal (UM-PAMECAS). _________.2012k. Convention de partenariat entre le projet USAID|Yaajeende et la Compagnie Nationale d’Assurance Agricole du Sénégal (CNAAS). _________.2011a. Cadre de partenariat entre le projet USAID|Yaajeende et le PRODAM. _________.2011b. Protocole d’Accord cadre Institut de Technologie Alimentaire-USAID|Yaajeende. _________.2011c. Convention de Partenariat USAID|Yaajeende et la Société Nationale d'Aménagement et d'Exploitation des Terres du Delta du Fleuve Sénégal (SAED). _________.2011d. Convention de Partenariat entre USAID|Yaajeende et l’Institut Sénégalais de Recherches Agricoles (ISRA). _________.2011e. Protocole d’Accord diffusion single USAID|Yaajeende-RTS Tamba _________.2011f. Protocole d’Accord dans le cadre du contrat entre le projet USAID|Yaajeende et Radio Dunya Tamba _________.2011g. Protocole d’Accord entre le projet USAID|Yaajeende et l’Agence Régionale de Développement(ARD) de Tambacounda _________.2011h. Convention de partenariat entre le projet USAID|Yaajeende et le Projet d’Appui à la Petite Irrigation Locale (PAPIL) _________. 2011i. Grants and Enterprise Fund Manual Policies and Procedures, USAID/Senegal. _________. Undated. Proposition d’un Systėme de Graduation des InstitutionsPartenaires et des Coaches PRL Victora, Cesar, Jean Pierre Habicht, and Jennifer Bryce. 2014. “Evidence-Based Public Health: Moving Beyond Randomized Trials.” American Journal of Public Health 94: (3):400-05. 74 Annex IV. List of Persons Interviewed USAID/Senegal Name Title Office ADRIEN Jack Contracting Officer Regional Acquisition and Assistance Office BAQAI Razia Environment Specialist Economic Growth Office DIEYE Papa Nouhine Agriculture Specialist/AOR Yaajeende Project Economic Growth Office DEVER Jeseph Private Sector Specialist Economic Growth Office NDIAYE Alioune Mody Acquisition and Assistance Specialist Economic Growth Office THIAM Fatou Monitoring and Evaluation Specialist Program Office WILLIAMS Anne EGO Director Economic Growth Office USAID|Yaajeende, Dakar Name Title DIOUF Awa Taye Sarr Specialist M&E CROSBY Todd V. COP LA CREMA CFO SENE Papa Coordinator WANE Coudy LY Nutrition specialist WANE Aissata S. Financial direction USAID|Yaajeende, Bakel Name Title CISSE Mamadou Yaajende prestataire DABO Siré PM Finance and agricultural insurance DIAGNE Modou M. Regional coordinator DIALLO Alagassimou Agriculture supervisor DIOUFJoseph Ibrahima Nutrition Coach PMCC DIOP Ibrahima Cissé PM Nutrition entreprise LAH Mohamed ACCESS Coach MBODJ Adjaratou Assane M&E assistant NDAO Daouda PM livestock entreprise NDIAYE Mouhamadou Bachir PM agricultural mecanisation and postharvest THIOUNE Penda Finance and administration 75 USAID|Yaajeende, Matam Name Title AW Alassane LPM irrigated agriculture BA Mamadou Access coach DJIBRILLA Issa LPM/MC/SB DIOUF Mamaddou LPM/EHA/ Nutrition coach DIOP Serigne Malick M and E assistant FALL Bilal Livestock trainer Ka Rougiatou A. LPM Governance trainer SALL Fatou Kiné Diop Regional coordinator SOW Ndèye Nio RAF USAID|Yaajeende, Kédougou Name Title DABO Ladji Accès DIAKO Mamadou Coordinateur DIAWARA Aliou Superviseur agriculture GUEYE Adama Logisticien LY Ibrahima Technicien communication SECK El Hadji Amadou RAF SECK Rosalie Formatrice élevage TOURE Cheikh Tidiane Coordinateur Senegalese Government, Dakar Name Title CAMARA Maty Diagne Chef De Division Direction de la Nutrition (Ministère de la Santé du Sénégal) COLY Victor Directeur Direction de la Protection des Végétaux (Ministère de l’Agriculture) DIA Seydina Alioune Coordinateur de la Grappe Agriculture-Agro-industrie Primature DIAGNE Mamadou Makhtar Directeur général Direction de l'Appui au Secteur Privé (DASP)/ Ministère de l’Economie et des Finances DIALLO Bassirou Ingénieur/coordonnateur des cultures in vitro, Institut Sénégalais de la Recherche Agricole ISRA/LNERPV DIATTA Paterne Ingénieur, Responsable de l'arboriculture fruitière, Centre pour le Développement de l’Horticulture (CDH) de Cambérène, Point focal YAAJEENDE ISRA/CDH FALL Mouhamadou Directeur Général Adjoint Compagnie Nationale d’Assurances Agricoles du Sénégal (CNAAS) KA Abdoulaye Coordonnateur national Cellule Lutte contre la Malnutrition NDOUR Yacine Badiane chef du Laboratoire National de Recherches sur les Productions Végétales ISRA/LNERPV SOW Insa Responsable Production CNAAS 76 Local Authorities, Bakel Name Title TIMERA Hamidou Sada 1 Président Communauté Rurale de Aroundou DIAGANA Yankhoba Chef division promotion rurale et développement SAED et de la Falémé DIAW Pape Mbaye Chef du Service Départemental de l’Elevage (SDEL) DIEDHIOU Alpha Adjoint au chef de service, charge des statistiques SDDR Local Authorities, Matam Name Title DIALLO Mamadou Chef du service Départemental de l’Elevage (SDEL) FAYE Paul Marie Ingénieur délègué de Matam SAED GUEYE Youssoupha Directeur Régional, Direction Régionale du Développement Rural (DRDR) Matam NIANE Mamadou Inspecteur de L’Education et de la Formation Ministère de l’Education Nationale SALL Aliou SUPERVISEUR Région Médicale SOGNANE Salamata Agent de santé Case de santé Local Authorities, Kédougou Name Title NDAO Tall Saba Directeur, DRDR BOUBANE Kély Chef de service départemental de l’élevage DIOUF Mignane DISEM/ SDDR Private Sector, Dakar Name Title EPOK Georges C. Directeur Commercial HORTIS/ GREEN SEEDS FALL Moctar Directeur général AGROSEED NDOUR René Directeur Equip Plus TOURE Assane Directeur Général AGRIPRO Private Sector, Bakel Name Title NIANG Ousseynou Chef D’agence ACEP 77 Private Sector, Matam Name Title BA Sidy Directeur général Medy Industries Moderne BOCCOUM Amadou Tidiane Représentant FLORIDIA à Ourossogui DIACKO Mamadou Président de l’association Organisation des Producteurs d’Oignons de la Vallée KEBE Baidy Responsable financier Medy Industries Moderne SARR Djibril Stagiaire IMCEC SYLLA Babacar Superviseur IMCEC TOURE Hamady Chargé du crédit IMCEC Private Sector, Kédougou Name Title DIAKHABY Karamba APS arboriculture FOFANA Moh APS arboriculture SAM Boubacar Eleveur émergent SAMOUSA Demba Arboriculteur 78 Annex V. Evaluation Design Matrix USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method Component 1: Accelerate the Participation of the Rural Poor in Rural Growth Question 1 Has the project demonstrated effective, efficient and sustainable vehicles/approaches for promoting adoption of innovation (technology, practices, behaviors) and diffusion of products and new technologies among the poor, women, and socially marginalized? 1. a. Identify new technologies, practices and behaviors introduced and disseminated by the project and how they have been promoted and disseminated Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights 1. b. Have the new technologies, practices and behaviors introduced and disseminated by the project been effective? Definitions: effective = worked well; brought desired results 3, 6, 7, 8, 9, 10, 13, 14, 15 Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights 1. c. Have the new technologies, practices and behaviors introduced and disseminated by the project been efficient? Definition: efficient = maximum output (result) Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; Guided interviews with tabulation of 79 USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method with minimum input private sector partners; selected beneficiaries key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights 1.d. Have the new technologies, practices and behaviors introduced and disseminated by the project been sustainable? Definition: sustainable = lasts after project ends Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights Question 2 Has the community￾based solution provider model proven to be an effective and a sustainable private sector driven approach to reduce undernutrition in targeted areas? 2. a. Has the community￾based solution provider model proven to be an effective private sector driven approach to reduce undernutrition in targeted areas? 23, 25 Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights 2. b. Has the community￾based solution provider model proven to be a Document review PMP, annual reports, other project documents Tabulation and discussion 80 USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method sustainable private sector driven approach to reduce undernutrition in targeted areas? Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights Question 3 How the activities to increase household assets and income led to improved participation of the rural poor in rural growth? Note: Improved participation implies that the rural poor play an active role (both quantitative and qualitative) in those activities that may result in structural changes in how those activities are implemented. 11, 12, 20, 21, 22, 41 Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights Component 2: Reduce Undernutrition Question 4 What project activities have positively enabled value chain investments to lead to improved consumption of diverse diets and quality foods? 32, 33, 34, 36 Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition Guided interviews with tabulation of 81 USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method volunteers; Citizen network groups; agriculture producers; artisans key issues and insights Question 5 What investments in human and institutional capacity development have effectively generated large scale nutrition outcomes? Indicators of nutrition outcomes: 28, 29, 30, 31, 32, 33 Indicators of human and institutional capacity developmen t: 38, 39, 40, Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights Question 6 To what extent the integrated nutrition and agriculture approach led to the reduction of undernutrition among the target population? Only an impact evaluation can rigorously answer this question (since answering the question requires a treatment group (people living in the project areas) and a comparison group (people living outside the project areas) Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights Question 7 To what extent the project’s water and Document review PMP, annual reports, other project documents Tabulation and discussion Key Project staff at HQ Guided 82 USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method sanitation activities led to improved healthy behaviors of the target population? informant interviews and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights Cross-Cutting Issues Question 8 To what extent has the project been implemented effectively, including timely completion of project activities, effective use of project resources, reach of target groups/beneficiaries, quality of partnerships and collaboration, and contribution to overall USAID/Senegal economic growth objective goals? 8.a.Timely completion of project activities Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes 8.b. Effective use of project resources Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes 8.c. Reach of target groups/beneficiaries 4, 5, 16, 17, 18, 20, 26, 37, 39 Document review PMP, annual reports, other project documents Tabulation and discussion 83 USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights 8.d. Quality of partnerships and collaboration Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes 8.e. Contribution to overall USAID/Senegal economic growth objective goals Mapping of indicators in USAID/Sene gal objective goals and USAID/Sene gal indicators Document review PMP, annual reports, other project documents Tabulation and discussion Question 9 What is the likelihood that project approaches/practices and results will be sustained No sustainabilit y indicators in PMP Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen Guided interviews with tabulation of key issues and 84 USAID|Yaajeende Evaluation Matrix Evaluation Question Evaluation Sub-Question Indicators (As Numbered in PMP) Data Collection Method Data Sources Data Analysis Method network groups; agriculture producers; artisans insights Question 10 What are the outcomes of the project approach to address gender, environmental compliance, and governance issues? 10.a. Gender Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights 10.b. Environmental compliance Document review PMP, annual reports, other project documents Tabulation and discussion 10.c. Governance Document review PMP, annual reports, other project documents Tabulation and discussion Key informant interviews Project staff at HQ and field levels; GOS officials in Dakar and field; private sector partners; selected beneficiaries Guided interviews with tabulation of key issues and insights from notes Focus group discussions With community￾based solution providers; community nutrition volunteers; Citizen network groups; agriculture producers; artisans Guided interviews with tabulation of key issues and insights