Agricultural Growth Program – Livestock Market Development (AGP-LMD) Project EXTERNAL MID-TERM PERFORMANCE EVALUATION REPORT May 2015         This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Tufts University. The authors of the report are Kurt Rockeman, Beyene Tadesse, and Tsehay Redda. Agriculture Knowledge, Learning, Documentation and Policy Project (AKLDP), Ethiopia Agricultural Growth Program – Livestock Market Development (AGP‐LMD) EXTERNAL MID‐TERM PERFORMANCE EVALUATION REPORT USAID Contract No. AID-663-C-14-00002 Agriculture Knowledge, Learning, Documentation and Policy (AKLDP) Project Implemented by: Feinstein International Center Friedman School of Nutrition Science and Policy Tufts University Africa Regional Office PO Box 1078 Addis Ababa Ethiopia Tel: +251 (0)11 618014 www.fic.tufts.edu Disclaimer The views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government.    CONTENTS List of acronyms ......................................................................................................................................................................... i List of tables ............................................................................................................................................................................... ii Acknowledgements ................................................................................................................................................................ ii EXECUTIVE SUMMARY....................................................................................................................................................... iii 1. INTRODUCTION...............................................................................................................................................1 1.1 Background to AGP‐LMD and the Mid‐Term Evaluation .......................................................................... 1 1.2 Project Objective and Intermediate Results .................................................................................................. 1 1.3 Theory of Change ...................................................................................................................................................... 2 1.4 Project Management, Coverage and Target Groups ................................................................................... 2 1.5 Evaluation Objectives and Questions ............................................................................................................... 3 1.6 Evaluation Design ..................................................................................................................................................... 4 1.7 Report Limitations .................................................................................................................................................... 4 1.8 Livestock Value Chain Productivity ................................................................................................................... 5 2. EVALUATION FINDINGS................................................................................................................................6 2.1 Program Performance ............................................................................................................................................. 6 2.1.1 General ......................................................................................................................................................................... 6 2.1.2 IR1: Increased Productivity and Competitiveness of Selected Livestock Value Chains ............. 8 2.1.3 IR2: Improved Enabling Environment for Livestock Value Chains .................................................. 13 2.1.4 IR3: Improved Quality and Diversity of Household Diet ....................................................................... 13 2.2 Program Impact on Government Policy, Process and Administration ............................................. 14 2.3 Effectiveness of the Implementation Approach ......................................................................................... 15 2.4 Relevance of Project Activities .......................................................................................................................... 17 2.5 Gender Issues ............................................................................................................................................................ 18 2.6 Nutrition Activity Performance ......................................................................................................................... 19 3. SUMMARY AND CONCLUSIONS................................................................................................................ 20 3.1 Program Performance ........................................................................................................................................... 20 3.1.1 General ....................................................................................................................................................................... 20 3.1.2 IR1: Increased Productivity and Competitiveness of Selected Livestock Value Chains ........... 21 3.1.3 IR2: Improved Enabling Environment for Livestock Value Chains .................................................. 24 3.1.4 IR3: Improved Quality and Diversity of Household Diet ....................................................................... 24 3.2 Program Impact on Government Policy, Process and Administration............................................. 24 3.3 Effectiveness of Implementation Approach ................................................................................................. 25 3.4 Effectiveness of Partnerships with GoE AGP Implementers and Other Government Offices .. 26 3.5 Relevance of Project Activities .......................................................................................................................... 26 3.6 Gender Issues ............................................................................................................................................................ 27 3.7 Nutrition Activity Performance ......................................................................................................................... 28 4. RECOMMENDATIONS.................................................................................................................................. 28 i List of acronyms ADLI Agricultural Development Led Industrialization AI Artificial Insemination AGP Agriculture Growth Program AGP‐AMDE Agricultural Growth Program – Agricultural Market Development AGP‐LMD Agricultural Growth Program – Livestock Market Development BDS Business Development Services CIG Common Interest Group CNFA Cultivating New Frontiers in Agriculture DA Development Agent DCA Development Credit Authority ENGINE Empowering New Generations to Improve Nutrition and Economic Opportunities Project, USAID EDDP Ethiopia Dairy Development Project FtF Feed the Future FMSEDA Federal Micro and Small Enterprise Development FtF ZOI Feed‐the‐Future Zone of Influence GoE Government of Ethiopia GRAD Graduation with Resilience to Achieve Sustainable Development Project, USAID GTP Growth and Transformation Plan HUNDEE Oromo Grassroots Development Initiative IP Implementing Partner IR Intermediate Result IGA Income Generating Activity JRIS Joint Review Implementation Strategy LITS Livestock Identification and Traceability System LMD Livestock Market Development LOL Land O’Lakes LTC Livestock Technical Committee LVC Livestock Value Chains LWG Livestock Working Group MLA Meat and Live Animal MOA Ministry of Agriculture MOH Ministry of Health MOT Ministry of Trade MSE Micro‐ and Small Enterprises MSP Multi‐Stakeholder Platform MSPM Multi‐Stakeholder Partnership Meeting MT Metric Ton ORDA Organization for Relief and Development in Amhara PEPFAR President’s Emergency Plan for AIDS Relief PCI Precise Consult International PLI II Pastoral Livelihoods Initiative, Second Phase PLWHA People Living with HIV/AIDs PMP Performance Management Plan PSNP Productive Safety Net Programme PRIME Pastoralists Areas Resilience Improvement and Market Expansion Project RDPAS Rural Development Program and Strategy REDFS Rural Economic Development and Food Security REST Relief Society of Tigray RFA Request for Application RMSDA Regional Micro and Small Enterprise Development SNNPR Southern Nations, Nationalities, and Peoples' Region SOW Statement of Work ii TA Technical Assistance TOT Training of Trainers VC Value Chain US United States USAID United States Agency for International Development List of tables Table 1: Ethiopia cattle population by age (2013‐2014)…………………………………………….. 5 Table 2a: Selected AGP‐LMD PMP IR1 Performance Indicators ……………………………………..9 Table 2b: AGP‐LMD IR1 Gross Performance Measures …………………………………………………11 Table 3: Selected AGP‐LMD PMP IR2 Performance Indicators …………………………………….13 Table 4 Selected AGP‐LMD PMP IR3 Performance Indicators …………………………………….14 Note In Ethiopia a woreda is an administrative unit, roughly equivalent to a district in other countries. A kebele is equivalent to a sub‐district, being the next level of administrative unit within a woreda. Acknowledgements The members of the evaluation team would like to express gratitude and sincere appreciation to the AGP‐LMD management and staff for their dedicated cooperation and assistance in organizing our travel and interview schedule and during our site visits; and the AKLDP staff for their substantive guidance and administrative and logistical support. iii EXECUTIVE SUMMARY The external mid‐term evaluation of the USAID/Ethiopia Agricultural Growth Program – Livestock Market Development (AGP‐LMD) project was carried out in March‐April 2015 to respond to Statement of Work (SOW) specifications and specific questions provided by USAID. AGP‐LMD is a five‐year, $41 million project implemented as part of the U.S. Government Feed‐ the‐Future (FtF) Initiative. The goal of the project is: Improved Smallholder Incomes and Nutritional Status. AGP‐LMD, in concert with the crop value chain program (AGP‐AMDe), serves as part of USAID’s contribution to the Government of Ethiopia’s Agricultural Growth Program (AGP). The AGP goal is “to end poverty and enhance growth”, which aligns with the FtF goal to “sustainably reduce poverty and hunger”. The project is implemented by CNFA. The objective of AGP‐LMD is to “foster growth and reduce poverty by improving the productivity and competitiveness of selected livestock value chains: dairy, meat and live animals.” It seeks to achieve this objective through three Intermediate Results and four cross‐ cutting issues: IR 1: Increased productivity and competitiveness of selected livestock value chains IR 2: Improved enabling environment for livestock value chains IR 3: Improved quality and diversity of household diets Cross cutting issues: gender, ICT, environment and pro‐poor. Specifically, AGP‐LMD implementation focuses on 7 main livestock growth corridors and milk shed areas, and includes 62 woredas (46 AGP woredas and 16 non‐AGP woredas) in Tigray, Amhara, Oromia and the Southern Nations, Nationalities, and Peoples' Region (SNNPR). Of these 10 woredas have been designated Deep Focused Woredas where the project will invest much of its efforts, with a specific emphasis on nutrition related activities. The AGP‐LMD evaluation team collected and analyzed quantitative and qualitative information to assess progress towards meeting the project goal and objectives using selected indicators and targets from the AGP‐LMD Performance Monitoring Plan (PMP). Quantitative information on accomplishments described in project documents and reports was verified and complemented with qualitative information from face‐to‐face interviews with over 70 project implementers, stakeholders and informed observers in six of the main livestock growth corridors/milk sheds. Key Findings and Conclusions The following discussion is a summary of the general findings, conclusions and recommendations of the evaluation team:  CNFA and its consortium partners are making a credible effort to implement the project, with the following very notable positives: o Capacity Building: Individuals interviewed were very positive about project technical and business training, which we believe has been the most successful element of the project to date. o Women’s Participation: The specially designed entrepreneurship, governance and leadership training focused on women, and the substantive involvement of women in all project activities. o Business‐to‐Business: This activity is greatly appreciated as a process for introducing firms to one another, breaking down resistance to cooperating with one another, and facilitating collaboration locally and internationally. o Dairy Value Chain: The project is doing a very good job of developing the dairy sector, especially with regard to collection centers and processors at all levels. o Investment: The project is providing grants totaling more than $6 million to leverage private investment, and has helped 9 borrowers obtain $2.2 million in DCA financing. iv o Regional Livestock Working Groups (LWGs): This is a highly regarded platform for bringing together government, NGOs, business interests, and producers to discuss practical issues and problems impacting the livestock sector and recommend solutions.  Based on our findings and conclusions, we believe that the project should concentrate its efforts during the remaining LOP on: o Greatly expanding support for micro‐ and small enterprise (MSE) development in the rural areas adjacent to livestock producers. o Substantially increasing the role and responsibilities of the local Implementing Partners, especially in designing and implementing MSE development activities, and measuring the impact of project activities (with CNFA technical advice and support). o Establishing private Business Development Services (BDS) providers to ensure essential services for grantees and industrial scale value chain operations. The principal findings and conclusions of the evaluation team which inform the summary and recommendations above are:  The link between AGP and FtF goals, and their link to the AGP‐LMD objective of increasing value chain productivity and competitiveness, is not clearly articulated in project documents. This results in an awkward geographic and targeting focus.  It can be argued that AGP‐LMD supports rural economic growth, and that such growth is a necessary (but not sufficient) condition for poverty reduction. It can also be argued that increasing value chain productivity and competitiveness can contribute to the achievement of both AGP and FtF goals. However, project documents do not make these arguments.  Individual project components, strategies and activities are not clearly defined and articulated, especially with regard to the separate dairy and meat/live animal value chains. This creates confusion among implementing partners and stakeholders, compromising implementation effectiveness.  AGP‐LMD activities (dairy value chain, meat and live animal value chain, training, grants program, enabling environment, and nutrition) resemble separate projects. There is little synergy between activities, which dilutes and complicates project management.  CNFA is roughly on track to meet most PMP targets, especially when compared with the percentage of the budget which has been expended. This is in spite of unrealistic implementation expectations; unreasonable expenditure targets during the first two years; and some ill‐considered decisions (e.g., suspending IR1 training in Year 3).  The grants program is designed to leverage large‐scale private investment, particularly processing capacity, in the livestock value chains. But the grant process is incredibly cumbersome, and the current rate of expenditure makes it unlikely that the program will contribute substantially to the achievement of project goals during the LOP.  Major PMP indicators, especially the mandatory FtF indicators, are inadequate to measure project impact or contribution to the achievement of AGP and FtF goals. PMP indicators are also insufficient to measure the performance and the contribution of the grants program and credit facilitation activities to the achievement of project goals  As stated previously, the project is doing a very good job of supporting dairy industry development. But the impact of increased capacity and consumer demand on productivity will not really be felt until after the project ends.  The project “pull” philosophy for drawing producers into the value chain, while focusing on working in the middle of the chain, makes sense. But in practical terms the project is working too far distant in the value chain from rural producers to impact productivity. v  Meat and live animal activities are too concentrated at the top end of the value chain, working with abattoirs and large scale feedlots, too focused on exports, and too far distant from producers to exert effective “pull”.  Ethiopian exports of live cattle and shoat carcasses are solidly competitive in the regional market. But they are unlikely to increase unless government‐imposed disincentives (policy and regulatory) are eased to draw more animals into formal export channels.  Beef exports are not presently, nor likely to be, competitive – at least in the near future. Major factors affecting competitiveness include product quality and cost.  Efforts to improve the enabling environment for livestock value chains are encouraging, but the impact on government policy and regulation is yet to be realized.  The project is on track to meet IR3 indicator targets, but the practical impact on nutritional practices and PLWHAs is not yet realized, and the scale of eventual impact is uncertain.  The project is not taking full advantage of the institutional and organizational strengths, and technical capabilities of local Implementing Partners. The culprit is the ineffectual sub‐contracting mechanism currently in use, which mitigates against effective performance. Recommendations The specific recommendations of the Evaluation Team for increasing the project contribution to both AGP and FtF objectives, and achieving the project goal and objective, are as follows: 1. Develop and incorporate in project documents and reporting a clear articulation of the relationship between the AGP and FtF goals, their linkage with the project goal and objectives, and exactly how project activities are expected to contribute to the achievement of those goals and the project objective.  At present the project goal and rationale, and the expected contribution of project activities to the achievement of the goal and LOP targets, is not clearly articulated. 2. Develop custom PMP indicators to measure and record the outcome and impact of the training programs and the grants program; and their contribution to the achievement of project goal and objective.  We suggest developing an indicator of increased sales/income for measuring the impact of training on MSEs and producers.  We also suggest indicators of increased product sales (volume and value), increased input purchases (volume and value), and a re‐evaluation of the job creation indicator (person/days of employment) to measure the impact of the grants program and loan facilitation. 3. Develop and implement a methodology to measure (rather than simply estimate) the impact of AGP and AGP‐LMD activities on livestock productivity (fertility, mortality, and yield/animal) and rural household income.  Specifically, we recommend joint design and implementation of the methodology with local Implementing Partners in line with Recommendation 5, in consultation with other stakeholders. This will both improve the effectiveness of FtF reporting, and help AGP measure the impact of its activities. 4. Greatly expanding support for micro‐ and small enterprise (MSE) development in the rural areas adjacent to livestock producers (first point of sale) as the focus of implementation during the remaining Life‐of‐Project.  Specifically, we recommend combining technical and business management training with small capital investment grants to leverage entrepreneurship (suggested amount vi $500 ‐ $1,000). This will promote value chain development, more directly impact the FtF target population, increase the number of beneficiaries, and contribute more clearly to the achievement of the AGP goal. 5. Substantially increasing the role and responsibilities of the local Implementing Partners, making them directly responsible for the design and implementation of the MSE development program recommended above.  Specifically, we recommend changing the current sub‐contracting modality to either a sub‐grant (preferred) or a multi‐year subcontract combining cost reimbursement for core operations with purchase orders for specific activities. This will ensure continuity in implementation, dramatically increase impact, and take full advantage of IP institutional and technical capacity and operational presence, enhancing sustainability. 6. Bring the large‐scale grants program to an orderly conclusion.  Specifically, we recommend that LMD not consider any grant in excess of $100,000. AGP‐LMD has done enough with the program to leverage new, larger‐scale value‐chain investments. It needs to finalize those grants currently in process, and bring the activity to a close. However, the project will need to continue providing TA during the remaining LOP to help the grantees succeed. 7. Design and implement a professional BDS program for providing essential TA and training services grantees and industrial scale value chain operations (milk processors, feedlots, feed processors, and abattoirs).  AGP‐LMD needs to develop individual programs of assistance with each client, defined in a written agreement which clearly states what the project will do in terms of providing technical assistance and advice, client obligations, and clearly defined results benchmarks. 8. Keep IR2 activities modest and focused.  The project should continue facilitating MSP and LWG activities, the pilot LITS, and modest engagement to support SPS issue resolution. The project should also seek to promote a new unified policy and regulatory framework for the livestock sector to replace the present burdensome and inconsistent combination of contradictory policies and regulations. 9. Design and incorporate pilot poultry activities into IR3, emphasizing patio production and home consumption of eggs.  We recommend a combination of including messages promoting the home use of eggs and poultry, and modest efforts to improve management and productivity of patio flocks. This will tie IR3 more directly to the overall project; emphasize the use of the animal source food (eggs and chicken) which is most accessible to the poor to improve nutrition; and lay the foundation for the design of future nutrition activities. In particular, the pilot poultry activity should incorporate value chain concepts where appropriate, perhaps engaging PLWHAs to raise chicks into mature birds which can be made available to households. Implementing the Recommendations We have crafted the recommendations to make implementation a fairly straightforward process. The changes we recommend will stimulate creativity and action, promote achievement, produce results, and make everyone look better. The two steps to implement the recommendations are: 1. Annual Work Plan: Recommendations 1, 2, 3, 6, 7, 8 and 9 can be implemented in the context of developing the next Annual Work Plan. USAID will need to provide guidance to CNFA to carry this out, but the recommendations are not complicated and do not appear to require substantive changes to the project SOW or budget. vii 2. Contract Amendment: Implementing Recommendations 4 and 5 will require an amendment to the contract SOW to describe the MSE development program, establish the guidelines for expanding Implementing Partner roles and responsibilities, and amending the budget to accommodate these changes. USAID may wish to consider a project extension to compensate for the time needed to make these changes. This need not be a long, drawn‐out and complicated process. But, it will take will and decisiveness: the will to embrace change; and the decisiveness to press on and make it happen. Summary CNFA is doing a credible job of implementing the project, and we believe that AGP‐LMD is contributing to the achievement of the AGP goal. But the project contribution to the achievement of the FtF goal is not clear, project activities lack synergy; and project results and impact are not being methodically measured. Therefore, in summary we recommend that implementation during the remaining LOP focus on MSE development in areas adjacent to rural livestock producers to increase impact and project contribution to the achievement of AGP and FtF goals. 1 1. INTRODUCTION 1.1 Background to AGP‐LMD and the Mid‐Term Evaluation The external mid‐term evaluation of the USAID/Ethiopia Agricultural Growth Program – Livestock Market Development (AGP‐LMD) Project was carried out in March‐April 2015 following the Statement of Work (SOW) specifications provided by USAID. The purpose of the evaluation is to examine what AGP‐LMD has achieved at the mid‐point in implementation. Specifically, the evaluation assessed whether AGP‐LMD is achieving or is likely to achieve the expected results by the end of the project, as well as the management and operation of the project. The findings, conclusions and recommendations are intended to inform and improve implementation of AGP‐LMD during the remaining life of the project. The evaluation findings will also help inform the design of similar projects in the future. AGP‐LMD is a five‐year project implemented as part of the U.S. Government Feed‐the‐ Future (FtF) Initiative. The goal of the project is: Improved Smallholder Incomes and Nutritional Status. AGP‐LMD, in concert with the crop value chain program (AGP‐AMDe), serves as part of USAID’s contribution to the Government of Ethiopia’s Agricultural Growth Program (AGP). The AGP goal is “to end poverty and enhance growth”, which aligns with the FtF goal to “sustainably reduce poverty and hunger.” AGP‐LMD is being implemented under a contract signed with Cultivating New Frontiers in Agriculture (CNFA)(AID‐663‐C‐12‐00009) on September 17, 2012 with an end date of September 16, 2017. Project activities focused on livestock growth corridors and milk sheds which encompass targeted AGP and food insecure woredas in Tigray, Amhara, Oromia and SNNPR. Implementation arrangements use and build upon the extensive professional and social networks of local organizations in the four regions. The initial Life of Project Budget was $37,673,362, subsequently increased to $41,173,362 on February 11, 2015. 1.2 Project Objective and Intermediate Results The objective of AGP‐LMD is to “foster growth and reduce poverty by improving the productivity and competitiveness of selected livestock value chains: dairy, meat and live animals.” It seeks to achieve this objective through three Intermediate Results (IRs), five Components, and numerous activities as follows: IR 1: Increased productivity and competitiveness of selected livestock value chains Component 1: From Analytics to Strategy to Learning - Community of practice - Innovations and good practices - Cases, studies, presentations and exchanges - Training of staff Component 2: Improve the Productivity and Competiveness of Livestock Value Chains - Link value chain actors to input and service providers - Strengthen input and service providers - Improve livestock management - Improve post‐production relationships, efficiencies and quality - Improve number, quality and functionality of the middle of the value chain - Market expansion – domestic and international - Strengthen governance and collaboration within value chain - Increase domestic consumption of dairy products - Increase women entrepreneurship and leadership development Component 3: Spur Investment and Innovation - Stimulate investments and access to finance throughout the value chain 2 IR 2: Improved enabling environment for livestock value chains Component 4: Improve the Enabling Environment of Livestock Value Chains - Facilitate and empower stakeholders for policy reform and advocacy - Build capacity of public and private stakeholders to provide services - Institutional coordination among stakeholders - Applied research for development - Linkage creation with others IR 3: Improved quality and diversity of household diet Component 5: Enhanced Nutrition of Rural Households (Nutrition and PLWHA) - Capacity building for IPs - Behavioral change communication - Community engagement - Focus on gender - Developing partnerships and network - Savings and Credit Group (SCG) formation and training (PLWHA) - Business training and resource mobilization Cross Cutting Gender, ICT, Environment, Pro‐Poor 1.3 Theory of Change AGP‐LMD is focused in the highly productive areas of Ethiopia where smallholder farmers have the productive assets required for at least minimal commercial engagement. It also includes activities which target extremely poor and subsistence level households and selected food insecure woredas where the graduates of the Productive Safety‐Net Program (PSNP) reside. In these areas it aims to pull their products into the marketplace upon graduation from USAID "push" initiatives, which will ultimately increase their incomes. The idea is for AGP‐LMD interventions to generate the market “pull” required to expand producer incomes, develop commercially‐oriented supporting services, and form sustainable supplier and buyer relationships. This expanded market pull will link new market participants graduating from subsistence agriculture by USAID's GRAD and PRIME initiatives. The key development hypothesis of AGP‐LMD is that market‐driven enterprise development can generate increased producer incomes by pulling previously marginalized populations into commercial value chains. Increased incomes, supported by effective planning and social behavior change communication, then lead to improvements in nutrition, household food security and health, and can equitably impact women, minorities and people living with HIV/AIDS. The project implementation approach is guided by a hypothesis which states that “empowering the private sector at all levels of the value chain, including the intersections with health and nutrition, is the most effective way to achieve sustainable growth of the livestock sector and to assure improvements in people’s nutrition and health.” To prove the hypothesis AGP‐LMD applies a holistic value chain development approach to achieve its objectives, with the middle of the value chain as an entry point and AGP woreda livestock and dairy producers as primary beneficiaries. To effectively implement this approach, AGP‐LMD used a “livestock growth corridor” or “milk shed” (cluster of woredas) approach to select intervention areas and targeted beneficiaries. Value chains are not limited to specific woreda boundaries or even to regional boundaries. Downstream value chain actors are often not present in the same woreda as their suppliers, while the improvement of the competiveness of a value chain requires working with the actors along the chain. 1.4 Project Management, Coverage and Target Groups AGP‐LMD is led by CNFA and supported by thirteen consortium partners. CNFA uses an implementation approach which combines Ethiopian and international consortium partners with proven livestock and dairy value chain development experience, with expertise in health, nutrition, gender equity, and ICT. In addition to CNFA, the AGP‐LMD consortium includes four large and experienced regional partners operating in the target regions: Relief Society of Tigray, 3 the Organization for Rehabilitation and Development in Amhara, the Oromo Grassroots Development Initiative, and Self‐Help Africa – Ethiopia. These organizations provide regional office facilities and regionally based personnel for front‐line implementation. Local business consulting firms Precise Consult International, BCad Consulting Management, and TREG Consult conducted end market studies, value chain gap analyses/strategies, and program learning strategies, as well as supporting program interventions to improve access to finance, business skills, and gender inclusiveness. This team is supported by international consortium partners, which include the Netherlands Development Organization, International Medical Corps, J.E. Austin & Associates, the Institute for International Education, and the International Institute for Communication and Development. In terms of geography the overall FtF implementation area includes the 111 AGP woredas that fall under the aegis of AGP‐AMDe, AGP‐LMD and ENGINE. It also includes an additional 38 woredas outside the AGP woredas where GRAD and PRIME are active. This diverse group of 149 woredas constitutes the Feed‐the‐Future Zone of Influence (FtF ZOI), with the population of vulnerable households within it the ultimate targeted beneficiaries of the FtF program. Within this context AGP‐LMD implementation focuses on 7 main livestock growth corridors and milk shed areas, and includes 63 woredas (46 AGP woredas and 16 non‐AGP woredas) in Tigray, Amhara, Oromia and the Southern Nations, Nationalities, and Peoples' Region (SNNPR). Of these 10 woredas have been designated Deep Focused Woredas where the project will invest much of its efforts, with a specific emphasis on nutrition related activities. In addition, the project includes a focus on engaging women as participants and increasing women’s role in leadership and decision making participants. Women are key actors in livestock value chains, but due to culture and social traditions do not benefit fully and freely from their participation. It also targets households with children, youth, and HIV/AIDS positive members. 1.5 Evaluation Objectives and Questions The objective of the AGP‐LMD project mid‐term evaluation is to provide USAID, CNFA, and stakeholders with an independent assessment of project performance to date. The evaluation focuses on progress in achieving the project goal, objectives, and planned results from September 2012 through March 2015; and on AGP‐LMD’s contribution to USAID Ethiopia’s FtF program delivery. Specifically, the evaluation assessed quantitatively measurable progress toward the achievement of the AGP‐LMD goal and objectives as articulated in the Performance Management Plan (PMP), and the impact of AGP‐LMD activities on selected livestock value chains. This is complemented by a qualitative assessment of other project results; the effectiveness of CNFA management, including collaboration/coordination mechanisms; the working relationship between implementing partners and stakeholders; and the program’s contribution to the achievement of FtF development objectives. The evaluation also provides recommendations to USAID, CNFA, and other stakeholders regarding the prioritization of project resources and activities to maximize impact and sustainability during the remaining life of the project, as well as to inform the design and implementation of similar projects in the future. The specific objectives of the AGP‐LMD evaluation are to: 1. Assess progress towards achieving the AGP‐LMD project goals and objectives in measurable, quantitative terms; 2. Assess other results, such as the impact of promotional and training activities, and women’s participation in project activities, in qualitative terms; 3. Evaluate CNFA’s methodologies, management, and activity coordination between partners and stakeholders in terms of programmatic and cost effectiveness in achieving project objectives and key results; 4. Provide information on project achievements and challenges, and make recommendations to USAID and CNFA for priority interventions during the remaining 4 life of the project to maximize impact, sustainability, and contribution to FtF development objectives; and 5. Make recommendations regarding activities which are candidates for being scaled up, those that should be discontinued, and those which can be used to inform new projects. The evaluation addresses the seven questions posed in the Statement of Work (Annex 1) as follows: 1. Are the selected program activities on track to reach the desired FtF results in Ethiopia? 2. Did the program activities lead to any significant changes in government policy, process, or administration that can be expected to contribute to increased production and investment in the sectors and/or increased export competitiveness? 3. What is the demonstrated effectiveness of the current implementation approach (e.g. working with local regional organizations such as ORDA, REST, HUNDEE and Self Help Africa ‐ Ethiopia; and with international consortiums)? 4. How effective is the partnership with GoE‐AGP implementers and other government offices such as Ministry of Agriculture, Ministry of Trade, Ministry of Industry, Federal Cooperative agency and ATA in terms of collaboration and coordination to implement AGP‐LMD? What factors contribute to success or challenges in partnerships, particularly with regard to the GoE‐AGP, and what is recommended to resolve the challenges? 5. Based on sustainability and cost effectiveness, which activities should be continued and which activities are irrelevant or no more important to contribute to the project objectives in the future? 6. How has the program addressed gender issues? 7. How was the performance of the nutrition sensitive activities implementation toward achieving expected outcome 1.6 Evaluation Design The AGP‐LMD evaluation team collected and analyzed quantitative and qualitative information on project performance to assess progress towards meeting the project goal and objectives using principal indicators and targets selected from the AGP‐LMD Performance Monitoring Plan (PMP). Quantitative information on accomplishments described in project documents and reports was verified and complemented with qualitative information from interviews with project implementers, stakeholders and informed observers. Interviews provided insight into implementing partner expectations; the effectiveness and expected impact of short‐term training and nutrition activities; and the potential for project sustainability. The evaluation team also assessed the data/information collected from project reports, site visits and interviews to respond to the specific evaluation questions. The evaluation team reviewed AGP‐ LMD achievements and linked them back to CNFA’s performance measures, especially regarding sustainability, enterprise development, management, quality, gender participation and nutrition issues. The AGP‐LM Evaluation Team worked with USAID and CNFA to develop a schedule of face‐to‐ face meetings with implementing partners and stakeholders. These were combined with visits to sites in six of the main livestock growth corridors/milk sheds. CNFA and USAID provided a list of key implementing partners and stakeholders for the AGP‐ LMD evaluation team to interview. The team developed a list of questions to guide interviews with implementing partners and stakeholders, which were used as a reference during the interview process to focus discussion. Further details on the evaluation design are provided in Annex 3 (AGP‐LMD Mid‐Term Evaluation Work Plan), and the CVs of the evaluation team members are provided in Annex 8. 1.7 Report Limitations There are important socio‐economic and agro‐ecological differences between the four regions included in USAID’s zone of influence, and the AGP‐LMD target woredas. In addition, given the time limitations and the geographic dispersion of project activities, it was not feasible to use a 5 statistically representative sample of target woredas and informants to examine AGP‐LMD’s work. Also, it was not possible to interview every regional and federal government counterpart and institutional stakeholder. However, the Evaluation Team did interview over 70 livestock value chain actors ranging from input supply and service providers to producers and processors, regional and federal level government officials, plus implementing partner and CNFA staff to obtain the information upon which our findings, conclusions and recommendations are based. Finally, the time available to the Evaluation Team was insufficient to carry out in‐depth analysis of each project strategy, component, and activity. Therefore, the evaluation report concentrates on the broad parameters of project design, implementation, and anticipated impact to identify the major actions needed to improve performance and impact. 1.8 Livestock Value Chain Productivity As previously stated, the objective of AGP‐LMD is to “foster growth and reduce poverty by improving the productivity and competitiveness of selected livestock value chains: dairy, meat and live animals.” In this context, Intermediate Result 1 (IR 1) is focused on increasing the productivity and competitiveness of selected livestock value chains. But there is little to indicate exactly what this means. Therefore, this segment of the evaluation report provides a brief explanation of the key issues related to livestock productivity (cattle). Additional information regarding sheep and goats is contained in Annex 8. The estimated cattle population in the sedentary area of Ethiopia (excluding 3 zones in Afar and 6 zones in the Somali regions) during 2013‐2014 is 55,027,280 head, as represented in the table below. Of that total, 20,545,625 head are females above the age of 3 years. This roughly constitutes the breeding population (excluding heifers which have not calved). Table 1: Ethiopia cattle population by age (2013‐2014) Age Total Male Female Number % Number % Number % TOTAL 55,027,280 100 24,555,570 44.6 30,471,710 55.4 - Under 6 months 5,228,666 9.5 2,514,073 4.6 2,714,593 4.9 - 6 months to 1 year 4,380,095 7.9 2,099,315 3.8 2,280,780 4.1 - 1 year to 3 years 9,026,584 16.4 4,095,873 7.4 4,930,712 8.9 - 3 years to 10 years 34,974,795 63.6 15,121,553 27.5 19,853.243 36.1 - Over 10 years 1,417,140 2.6 724,758 1.3 692,382 1.3 Source: CSA Agricultural Sample Survey 2013/2014 (E.C. 2006) With these numbers as a starting point, there are four ways to increase the productivity of Ethiopia’s cattle herd. They are: a. Increase fertility: The total estimated number of calves born during the year was 10,225,288 – which represents a crude fertility rate of 49.8%. Simply put, roughly half of the breeding age cows had a calf during the year. Increasing fertility to 55% would produce an additional 1,000,000 calves from the same number of cows – an increase in productivity. b. Decrease mortality: Estimated mortality during the year was 3,481,333 – which represents a crude mortality rate of 6.3%. Decreasing mortality to 5.3% would mean that an additional 550,000 cattle survive, thereby increasing animals available for breeding and sale (offtake) – increasing productivity. c. Increased milk production: An estimated 10,731,656 cows were milked at some time during the year. Average daily production of 1.371 liters (net) during a 6 month (180 day) average lactation period yielded roughly 2.65 billion liters of milk. Simply increasing average milk production to 1.5 liters per day would yield an additional 250 million liters of milk, or slightly more than 680,000 liters per day – increasing productivity. 6 d. Increased weight: An estimated 3.8 million head of cattle were slaughtered for domestic consumption during the year. The survey does not seek to quantify the amount of meat produced, but increasing average meat yield by 5 kilograms per head would yield an additional 19 million kilograms of meat available to the local market – an increase in productivity. In summary, because of the size of Ethiopia’s cattle herd even modest improvements in these factors will yield significant productivity gains. The basic elements of increasing livestock productivity are described as follows: a. Increasing fertility requires improving the physical condition (health) of breeding cows. That can be accomplished by improved feeding practices and access to feed (better nutrition) and increasing access to and use of veterinary services (health care). b. Decreasing mortality requires improving physical condition (health) of animals. As with increased fertility, that can be accomplished by improved feeding practices and access to feed (better nutrition) and increasing access to and use of veterinary services (health care). c. Increased milk production requires a combination of improving physical condition (health) by improved feeding practices and access to feed (better nutrition), increasing access to and use of veterinary services (health care), and genetic improvement. d. Increased weight requires improved feeding practices and access to feed (better nutrition) and increasing access to and use of veterinary services (health care). In summary, improved feeding practices and increased access to and use of veterinary services are essential to improving productivity. The catch is that there is a long lag time between improvements and increased production (offtake). For example, it takes 3 years for calves born due to increased fertility to wean their first calf (female) or be available for slaughter (male). Since most mortality occurs among calves, the same is true for decreasing mortality. Increased milk production and weight can be achieved in somewhat less time. But animals stunted early in life never fully recover, and for most cows increased milk production as a result of improved feeding will take place in the next lactation. Genetic improvements require a generation (3 years at least) to begin to materialize. To summarize, the lag time required to increase livestock productivity means that most of the impact from what the project is doing today will only begin to be realized over the next 3 years. Some impact may be realized in the short run, but most of it will take place after the end of the project. 2. EVALUATION FINDINGS The findings presented below are organized according to the seven evaluation questions posed by USAID in the SOW. AGP‐LMD has three IRs, each with related components and activities (Section 1.2). AGP‐LMD performance is examined with regard to the major activity indicators and targets contained in the PMP – first in general terms, and then with regard to each of the IRs. There are also four cross‐cutting issues (gender, ICT, environment, pro‐poor) but these issues are not linked to specific activities, and are without indicators or targets. 2.1 Program Performance The following are the findings of the evaluation team in response to question one: Are the selected program activities on track to reach the desired FtF results in Ethiopia? 2.1.1 General First of all, we found that CNFA and its consortium partners are making a credible effort to implement the project, with some very positive achievements to date. These include:  Capacity Building: The ambitious program of technical and business training across a wide range of subjects and a large number of participants received very positive comments from everyone we interviewed. There is no doubt that this is the most successful element 7 of the project to date in terms of direct impact with the beneficiary population, and most likely to have impacted productivity.  Women’s Participation: The specially designed entrepreneurship, governance and leadership training focused on women, and the substantive involvement of women in all project activities, has been very successful.  Business‐to‐Business: This activity is greatly appreciated for its contribution in introducing firms to one another, breaking down resistance to cooperating, establishing business linkages, and facilitating collaboration locally and internationally.  Dairy Value Chain: AGP‐LMD is doing a very good job of supporting dairy sector development, especially with regard to expanding processing capacity, supporting the establishment of collection centers to improve the supply network, and training.  Investment: The project is processing grants totaling more than $6 million to leverage private investment, and has successfully helped 9 borrowers obtain $2.2 million in DCA financing.  Regional Livestock Working Groups (LWGs): This is a highly complimented platform for bringing government interests, NGOs, business people, and producers together to discuss practical issues and problems impacting the livestock sector and recommend solutions. Second, based on our findings and conclusions, we believe that the project should concentrate its efforts during the remaining LOP on:  Greatly expanding for micro‐ and small enterprise (MSE) development in the rural areas adjacent to livestock producers.  Substantially increasing the role and responsibilities of the local Implementing Partners, especially in designing and implementing MSE development activities, and measuring the impact of project activities (with CNFA technical advice and support).  Facilitating the establishment of private Business Development Services (BDS) providers to ensure the availability of essential services for grantees and industrial value chain operations. The specific findings of the evaluation team which inform the above are discussed as follows: Project Design Issues: The goals of AGP and FtF appear complementary, but in practice make for an awkward implementation marriage. For example, the AGP goal is “to end poverty and enhance growth”, and implementation is focused on improving productivity and economic growth in the high potential areas of Ethiopia, with an emphasis on surplus producers. However, the goal of the FtF initiative (the primary source of funding for the project) is “to sustainably reduce poverty and hunger” with implementation focused on improving the incomes of poor rural households, and improving the nutritional status of children. There is certainly overlap between AGP and FtF beneficiary populations, but the extent of overlap and how it is considered during implementation is not clearly described. AGP‐LMD combines these goals under the umbrella objective of improving smallholder incomes and nutritional status, which aims to contribute to the achievement of both using a value chain development approach to improve productivity and competitiveness. However, the value chain approach does not necessarily engage with or impact poor rural households. AGP‐ LMD is not fully consistent with the AGP geographic focus areas either. It departs from woreda specific geographic targets to encompass naturally occurring dairy milk sheds and livestock growth corridors. Since livestock value chains transcend administrative boundaries and geography, this makes perfect technical sense. However, it makes for an awkward geographic marriage, with some project activities focused on target woredas while others emphasize growth corridors and milk sheds. The key development hypothesis of the project is that market‐driven enterprise development can generate increased producer incomes by pulling previously marginalized populations into commercial value chains, increasing their incomes. The assumption is that this will lead to improvements in nutrition, household food security and health in the target 8 population. However, project contributions to the achievement of AGP and FtF goals hinge upon exactly which households actually benefit from project interventions. Within this somewhat awkward framework, the component parts of AGP‐LMD resemble separate projects (dairy, meat, live animals, training, grants, policy, and nutrition), with very little complementarity or synergy between them. This dilutes oversight, making the management of project implementation complicated and difficult. 2.1.2 IR1: Increased Productivity and Competitiveness of Selected Livestock Value Chains CNFA is roughly on track to meet most of the performance indicator targets (see Tables 2, 3 and 4 below). The most notable exceptions are job creation, which methodology issues makes totally inadequate to capture the employment impact of increasing livestock value chain productivity and competitiveness, and the number of MSMEs receiving USG assistance to access loans. We note that performance cannot be expected to be linear, but increasing over time with the later years of the project producing the bulk of the anticipated results. There are significant issues regarding the relationship of the indicators to AGP (enhanced growth) or FtF (poverty reduction) goals. This is complicated by the fact that AGP is not attempting to measure impact, relying upon the final evaluation. The major PMP indicators, drawn primarily from mandatory FtF indicators, are simply inadequate to measure impact, or to articulate the “so what” of project activities. Therefore, a review of the indicators leaves the reader wondering what the project is actually achieving with regard to AGP and FtF goals. Table 2a overleaf presents project achievements as of the end of December 2014 against LOP Targets for selected PMP performance indicators. The following is a general discussion of these indicators, and what they represent. Further detail is provided in Annex 9. The primary indicator, value of incremental sales at household level attributed to project implementation is not a valid measure of impact. First of all the AGP‐LMD focus on value chains means it is not working at the producer level (with some exceptions in the dairy value chain), and must therefore use secondary data sources to which a set of assumptions are applied. Second, most results reported to date are from the live animal and meat value chain (specifically sheep and goats), with data derived almost entirely from total export sales based on a set of assumptions. The lack of detail on the component parts of the estimate (# of animals, sale price) makes it impossible to determine whether increased sales are a result of increased productivity, or market price increases. Third, in the short term there are no new animals being produced by the system (due to the lag in supply response), so any recorded increment may simply be the diversion of animals from other channels (market ready livestock do not go unsold). That means the total should be discounted to reflect only incremental price. Fourth, as increased dairy processing capacity comes on line, the contribution of increased milk sales will also need to be discounted because there is, as of yet, no new milk in the value chain. Rather, increased producer sales to processors mostly displace existing raw milk sales. In summary, the calculations for this indicator need serious review and reconsideration. We recommend reducing the LOP target for the indicator to the original $65,912,056. The indicator value of livestock and livestock product exports attributed to project implementation provides a reasonable source of information on project achievements. But the target is almost certainly overstated, because increased beef and live shoat exports are highly unlikely to materialize. In addition, the indicator is insufficient to measure the impact of increased exports on household income and nutrition. We recommend reducing the LOP target for the indicator to the original $135,906,300. 9 Table 2a: Selected AGP‐LMD PMP IR1 Performance Indicators Indicator Achieved to December 2014 LOP Target  Value of incremental sales attributed to project implementation - Milk - Cattle - Shoats $ 29,613,505 $ 524,230 $ 872,686 $ 25,582,734 $ 111,293,5291 $ 11,548,403 $ 2,104,501 $ 97,640,625  Value of livestock and livestock product exports attributed to project implementation - Beef - Live cattle - Shoat meat - Live shoat $ 32,315,261 ‐ $ 1,525,000 $ 30,790,261 ‐ $ 191,292,3212 $ 54,500,000 $ 21,104,021 $100,336,500 $ 15,351,800  Gross Margin per unit of land, kilogram or animal of selected product ‐ Cattle ‐ Shoats ‐ Milk ‐ ‐ $ 57.50 $ 10.60 $ 10.00  Number of jobs attributed to project implementation 220 5,025  Number of farmers and others who have applied new technologies or management practices as a result of USG assistance 7,158 53,099  Number of individuals who have received USG supported short‐term agricultural sector productivity or food security training 12,204 4,180 F 76,976 23,093 F  Number of food security private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community‐based organizations receiving USG assistance 499 700  Number of vulnerable households linked to markets as a result of the project 1,826 7,420  Value of new private investment in the livestock sector or food chain leveraged by project implementation $ 3,188,747 $35,475,000  Value of agricultural and rural loans facilitated by the project $ 7,556,862 $ 24,265,830  Number of MSMEs receiving USG assistance to access loans 18 193 The gross margin indicator is essentially useless. First of all, the methodology is not applicable to the livestock sector because output prices are not stable and are outside producer control. That makes the calculation unpredictably cyclical and variable. Second, it only looks at cash expense, and does not provide for recovery of the considerable upfront investment costs required to engage in livestock production. Third, it does not account for any changes in inventory. Those same reasons make in equally useless for perennial crops, and virtually useless for annual crops. Fourth, data collection based on farmer recall is notoriously inaccurate. Fifth, as a point of reference, standard operating procedure for agricultural production enterprises and professionals is to measure and manage production costs, using them as a reference point when considering production decisions based on price expectations. It needs to be eliminated, not just from the PMP, but from the pantheon of FtF indicators globally.                                                              1 Original LOP target: $65,398,306 2 Original LOP target: $138,906,300 10 As noted above, the methodology for measuring jobs created makes it totally inadequate to capture the employment impact of increasing productivity and competitiveness in livestock value chains. The next four indicators (number of farmers, individuals, enterprises and households benefitting from the project) are simple output figures. While useful for indicating the scale of activity, they do not measure the impact of the project on household incomes and nutrition. The next two indicators (value of new private investment leveraged, value of agricultural and rural loans facilitated) also provide a useful indication of the scale of activity. But again, actual impact of these investments on household incomes and nutrition is not being measured. Finally, the target for the number of MSMEs receiving USG assistance to access loans is far too ambitious, and the whole indicator merits further redefinition, review and adjustment. Table 2b (below) summarizes overall indicator achievement as a % of the LOP indicator. Table 2b: AGP‐LMD IR1 Gross Performance Measures Indicator Achieved to December 2014 LOP Target  Financial Expenditure - % of target $ 13,883,227 34% $ 41,173,362  Value of incremental sales attributed to project implementation - % of target Overstated – return to original level $ 29,613,505 26% $ 111,293,529  Value of livestock and livestock product exports attributed to project implementation - % of target $ 32,315,261 17% $ 191,292,321  Gross Margin per unit of land, kilogram or animal of selected product Useless No target established  Number of jobs attributed to project implementation - % of target 220 4% 5,025  Number of farmers and others who have applied new technologies or management practices as a result of USG assistance - % of target Problem Indicator – needs revision 7,158 13% 53,099  Number of individuals who have received USG supported short‐term agricultural sector productivity or food security training - % of target 12,204 16% 76,976  Number of food security private enterprises, producer organizations, water users associations, women’s groups, trade and business associations, and community‐based organizations receiving USG assistance - % of target 499 71% 700  Number of vulnerable households linked to markets as a result of the project - % of target 1,826 25% 7,420  Value of new private investment in the livestock sector or food chain leveraged by project implementation - % of target $ 3,188,747 9% $35,475,000  Value of agricultural and rural loans facilitated by the project - % of target $ 7,556,862 31% $ 24,265,830  Number of MSMEs receiving USG assistance to access loans - % of target Problem Indicator – needs revision 18 9% 193 Two cautions with regard to the levels of performance. First of all, it should be compared to the amount of funds expended as a % of project budget – 34% in the case of AGP‐ 11 LMD. Second, performance is not linear. Rather, one expects it to increase at an increasing rate over the LOP, with most impact taking place in years 4 and 5. For these two reasons, we believe that the project is roughly on track to meet performance targets, with exceptions to that statement noted. However, actually meeting the targets will depend upon the success of recommended actions. Beneficiaries: The pull philosophy of the project is to draw producers into the value chain by formal market participation. But other than capacity building, project activities are working too far distant from producers to have any direct and immediate impact on producer income. AGP‐LMD activities have emphasized facilitating and supporting industrial scale agriculture (milk processing plants, export abattoirs, large feedlots, and feed mills) investments, with the rather heroic assumption that this will exert the needed market “pull” on smallholder production. However, the link between prices paid by these enterprises, and any changes in smallholder producer/rural household income, is not being measured or recorded. The project incorporates key value chain actors, including animal feed processors, private and cooperative dairy producers/processors, live animal exporters, domestic and export abattoirs, input supply, and AI and animal health service providers. However, it has to date overlooked other key actors (traders, transporters and brokers) who play a significant role in the operation of livestock value chains. This needs to be remedied. Capacity Building: AGP‐LMD has provided a great deal of technical, business, leadership, and policy training to a wide range value chain actors. These have included, among others: skill training on animal husbandry management, feed formulation, sanitation and hygiene for meat and milk products; Standard Operational Procedures; marketing; entrepreneurship; business proposal development, data recording and bookkeeping; and policy formulation. All the individuals, firms, cooperatives and government service providers we interviewed complimented the relevance and effectiveness of the training provided. The project also offered special training to women in entrepreneurship and leadership development. Some women entrepreneurs in the livestock sector were also identified for additional entrepreneurship and business management training and mentoring. The project conducted needs assessments for each of these entrepreneurs, and developed a coaching strategy to support women entrepreneurs trained in the previous year. This involves long term coaching through the Women’s Bureau. Participants stated that training activities were very effective in improving beneficiary skills as reflected in the enhanced productivity and production of their businesses. For example, dairy producers stated that practices changed subsequent to the training increased milk yields, improved milk hygiene and quality, reduced the percentage of milk rejected by processors, increased the quality of processed milk, and increased income. Some dairy processors are also attempting to introduce quality‐based payment systems based on milk density measurements and alcohol tests. Experience sharing among different actors, within and across the regions, and visits to enterprises out of the country were also very much appreciated. Grants Program: The grants program is focused on leveraging private investment in the value chain. The objective is to enhance the capacity of key value chain actors, improve their efficiency and competitiveness, and strengthen the pull effect on small producers. The program has been carefully designed, and is being responsibly managed. But in practice it is proving to be slow and cumbersome, and in need of streamlining. Total grant funding budgeted under the project is $6,058,738, to be allocated to a combination of private enterprises, cooperative businesses, and women entrepreneurs (not including PLWHA credit and savings groups). However, only 19 grants with a total value of $1,773,983 have been signed to date, and only $255,394 actually disbursed. Another 56 proposals are being further refined and evaluated. The grants which have been signed, and those in process, seem reasonable. But the expected contribution of the program to the achievement of major indicator targets, while expected to be substantial, is yet to be realized. Furthermore, delays in process mean that most grants will not have any significant impact until after the end of the project. Individual grantees will also need continued TA throughout the remaining LOP to have the best chance of success. 12 Business Linkages: The Business‐to‐Business (B2B) meetings have been an effective and popular approach to building relationships between value chain businesses, and promoting the concept of collaboration. Facilitating the participation of abattoirs in the international business exhibition and study visits was also contributed to business linkage and experience sharing opportunities. AGP‐LMD also used these opportunities to introduce new concepts and ways of thinking. These activities are greatly appreciated as a process for introducing firms to one another, breaking down resistance, and facilitating collaboration locally and internationally. 2.1.2.1 Dairy Value Chain Dairy value chain activities are centered on larger urban centers, which provide the market outlet for pasteurized milk and milk products. The emphasis is on milk sheds, and urban and peri‐urban milk production. This does not entirely satisfy AGP targeting criteria, but is nonetheless making a significant contribution to the achievement of AGP goals. However, the impact of dairy value chain activities on the achievement of FtF goals is not being measured and recorded. A significant amount of technical and management training is going into the dairy sector, from producer level through small scale processors to new startups and larger scale processors. Training for input supply (feed, veterinary drugs) and services (animal health, AI) is supporting the production side. All of this provides an essential entrepreneurial and technical foundation for expanding dairy production, but much more needs to be done (collection centers, small‐ scale processing) to more effectively reach FtF beneficiaries. However, more effort is needed to strengthen the market link between producers and processors and band consumers. The grants program is leveraging new investment in dairy processing and supporting services. The emphasis on industrial milk processing, and the marketing of pasteurized milk and milk products, will undoubtedly have an impact on domestic milk consumption. But dairy development faces significant challenges. For example, at the beginning of the project total of 107,050 liters/day of milk was being processed, versus established processing capacity of 190,500 liters/day. With new capacity coming on line, all plants will be challenged to grow their product market while capturing enough milk to satisfy increasing demand. There is certainly potential at the smallholder level to fill this gap, but it will require greater emphasis on pulling the already available raw milk into the formal channel, and building the milk supply network. 2.1.2.2 Meat and Live Animal Value Chain Current meat and live animal value chain activities are focused at the end of the value chain as witnessed during our visits to export abattoirs, and large‐scale feed lots with an exclusive export orientation. There are also limited efforts underway with domestic abattoirs and the feed lots which provide animals to them. However, the domestic supply of meat, emanating from municipal abattoirs, raises hygiene and safety issues which need to be addressed to promote and support modern, commercially oriented private sector owned abattoirs slaughtering for the domestic market. Ethiopia’s comparative advantages make it solidly competitive in the export of live cattle and shoat meat. For beef there is solid export demand for fresh (hot) meat in the Gulf States and Egypt, and the cattle which Ethiopia exports (Boran type) are well known and preferred. But Ethiopia is not, and is not likely to ever be, competitive in the export of chilled or frozen beef. The reality is that Ethiopia cannot currently compete on either meat quality (tenderness, a combination of age and condition) or price (high cost of conditioning animals) with major exporters. To further complicate things, domestic prices for beef are often higher than export. As a result, further investment in export abattoir capacity for beef is not warranted. Live cattle export remains a profitable, but risky enterprise. With the growing domestic demand for beef, and demand for lowland animals of export quality, there is little room for further growth. There is also a solid market for chilled shoat carcasses of the size Ethiopia can produce in abundance. Ethiopia is currently competitive in this market, but to maintain its edge must continue efforts to increase productivity and decrease processing costs. 13 2.1.3 IR2: Improved Enabling Environment for Livestock Value Chains AGP‐LMD efforts to improve the enabling environment fall into two general areas. First of all, building upon some modest individual successes, AGP‐LMD is playing a key role by facilitating the establishment and organization of regional Livestock Working Groups (LWGs). Chaired by the relevant Livestock Authority, these are providing a platform for bringing together government representatives, business interests, producers, and other stakeholders to discuss practical issues and problems and recommend solutions. The outcomes and recommendations of these groups will be taken up for consideration by the Multi‐stakeholder Platforms (MSPs). This bottom‐up approach to raising policy and regulatory issues has great potential for influencing change. Second, AGP‐LMD is collaborating with and supporting efforts at the national level to pilot the Livestock Identification and Traceability System (LITS), and providing technical assistance to help address issues related to the livestock and livestock product exports and the national SPS system. Table 3: Selected AGP‐LMD PMP IR2 Performance Indicators Indicator Achieved to December 2014 LOP Target  Number of policies, regulations and administrative procedures developed, enacted or implemented as a result of the project - % target 5 44% 12  Reduction in the cost of doing business in the dairy, meat and live animal value chains ‐ 20% Performance indicators do not really capture what is taking place. Simply counting numbers of things done does not indicate their relative importance. For example, the Chief Veterinary Officer cited instances of important AGP‐LMD collaboration on several animal health and management issues – including LITS, and SPS, and the new Livestock Marketing proclamation. This support is critical to help Ethiopia reduce regulatory burdens and costs, and maintain its comparative advantage in live cattle and shoat meat exports. We have included include a new custom indicator for IR2 which captures in understandable terms the goal of these activities – to reduce the cost of doing business in the dairy, meat and live animal value chains. The intent is clear, but the methodology for this indicator needs to be developed, defined, and properly measured. In summary, AGP‐LMD efforts in IR2 are definitely contributing to improved livestock sector policy and regulations. But the impact of these efforts on value chain efficiency and productivity is yet to be realized. 2.1.4 IR3: Improved Quality and Diversity of Household Diet The IR3 program is being implemented in collaboration with the Ministry of Health (MOH) and the Ministry of Agriculture (MOA). We were told that this collaboration is helping DAs improve their understanding of health and nutritional issues. We visited nutrition training sessions, nutrition clubs, and cooking demonstrations promoting the use of animal source foods to improve maternal and child nutrition. The approach is to provide TOT training on maternal and child nutrition to IP staff and woreda focal persons as the basis for influencing behavioral change in the target communities. These are complemented by nutrition training and cooking demonstrations, which help mothers learn by involving them in the preparation of a nutritional porridge recipe using animal products, cereals, vegetables, fruits, oil and iodized salt. The project is also training DAs on dietary diversity, infant and young child feeding, and livestock product preservation and storage. The objective is to build their capacity to transfer nutrition education to beneficiaries at the grassroots level. The training includes an overview of 14 why nutrition matters, nutrition priority groups, dietary diversification and the importance of consuming animal products, as well as safe milk and meat consumption. The primary target is pregnant and lactating mothers, but also includes fathers as they play a major role in household decision‐making regarding what food appears on the plate. The project also supported School Milk Day and The Second World School Milk Day events in a number of towns. The objective was to initiate school milk feeding programs, and promote safe milk consumption among schoolchildren, their family members, and nutritional target groups in the community. These events emphasized the nutritional benefits of drinking safe milk as part of a regular diet, and the importance of boiling/pasteurizing milk. However, behavior change takes long periods of time to take effect, and is directly linked to access and control of household resources. Therefore, mothers need access and control of these resources in order to provide improved nutrition for their families. For these reasons it is not clear how effective these activities will be in contributing to improved nutrition. AGP‐LMD is roughly on track to meet the LOP Indicator targets related to the number of children under five reached by the project (see rationale above), and is definitely on track with regard to the number of people trained in child health and nutrition. Performance as measured by the indicators is quite good, but impact will be difficult to measure. Table 4: Selected AGP‐LMD PMP IR3 Performance Indicators Achieved to December 2014 LOP Target  Number of children under five reached by project supported nutrition programs - % of target 14,905 7,513 F 34% 44,283 22,359 F  Number of people trained in child health and nutrition through Project‐supported programs - % of target 17,635 5,706 F 49% 35,728 10,718 F Activities are also underway to improve the nutrition and economic status of HIV‐affected and infected households in project Deep Focus Woredas. We visited one group, and talked with the members. PLWHA are being helped to form savings and credit groups (SCG), which are receiving TA and capacity‐building, and are using their savings to invest in joint group enterprises or individual businesses. The activity is small, but is making a difference in the lives of the participants. This is really a stand‐alone activity, with little relationship to the rest of the project. But it, the nutrition program, and the wrap‐around PLWHA program are activities for which the Implementing Partners are responsible. From what we observed, what we were told, and the indicator information implementation is going quite well. But, impact is not being measured or recorded, and is yet to be realized. 2.2 Program Impact on Government Policy, Process and Administration The following are the findings of the evaluation team in response to question two: Did program activities lead to any significant changes in government policy, process, or administration that can be expected to contribute to increased production and investment in the sector and/or increased export competitiveness? AGP‐LMD is very judiciously working on government policy, process and administration issues related to livestock value chains. The scale of effort appears to be just about right. Specific activities which provide a setting for broad‐based identification and discussion of issues impacting the livestock sector include: Livestock Working Groups (LWGs): The project has facilitated the organization and establishment of these groups as instruments for developing livestock policy agendas in the regions, and to help tackle practical problems. Regional livestock authorities chair the LWGs, and participants include Regional government bureaus and agencies, private sector interests, and NGOs working in the livestock sector. LWGs have been established in all regions, an 15 executive body elected, and participants have identified priority policy issues and agendas to be presented and discussed at the next multi‐stakeholder platforms (MSPs) in their respective regions. Multi‐stakeholder Platforms (MSPs): The project plans to organize 4 MSPs per year in the regions, and 1 at the federal level. To date the project has hosted a total of 22 MSPs (19 regional and 3 federal. However, there is only anecdotal evidence of their impact. Live Animal Trading Proclamation: AGP‐LMD has conducted sensitization and awareness sessions on the implementation of the Live Animal Trading Proclamation for government officials, traders, feedlot operators and transporters. Unfortunately, the proclamation is seriously flawed. That said, project efforts are helping create a common understanding of the new proclamation, the substantial financial investment which will be required to implement it, and capacity building needs. It will also require changes in the mindsets of producers, market actors and law enforcement agencies – the need for which is debatable. Federal MOT representatives and regional Trade and Transport Bureaus are considering these challenges, and have promised to come up with possible solutions. In the absence of a repeal of the proclamation, this is probably the best that can be expected. It remains to be seen if the proclamation will have any positive impact on the sector. Livestock Identification Traceability System (LITS): In support of this pilot effort an international LITS Consultant has been contracted; a working group established to technically lead the pilot program; and draft data entry and animal health certification forms have been designed. Planning is underway to conduct the pilot. Government livestock personnel all cited the LWGs as being an important step in providing an open and transparent environment for discussing issues and possible solutions. However, the work on LWGs, and their relationship with the MSPs, is still being defined. As a result, the impact on value chain productivity competitiveness is yet to be realized. 2.3 Effectiveness of the Implementation Approach The following are the findings of the evaluation team in response to question three: What is the demonstrated effectiveness of the current implementation approach (e.g. working with local regional organizations like ORDA, REST, HUNDEE, Self‐Help – Africa, and regional consortiums)? The fragmented nature of project activities, their geographic dispersion, and the rather awkward marriage of AGP and FtF goals tends to dilute focus and make oversight and management difficult. Management is further complicated by the large number of federal and regional government entities, and private actors, involved in AGP and AGP‐LMD implementation. The burden of simply doing business in the current operational modality, with so many actors and in a setting with so many government interests and structures takes an inordinate amount of time and effort on the part of USAID, CNFA, and the other implementers. Simply managing the overall effort requires inordinate amount of oversight and communication with USAID, distracting senior staff, disrupting activities, and making implementation clumsy and inflexible. In addition, the modality of implementation currently being used does not take full advantage of implementing partner (IP) capabilities. In fact, IPs are not really partners. They are simply sub‐contractors, hired to carry out certain tasks, but without being substantively and continuously engaged in activity design, planning and implementation, or monitoring and evaluation. For example, the contractual agreement with the IPs consists of an annual purchase order for specific tasks as directed by CNFA. As a result these partners do not have complete information on the full range of work planned for the whole project. They only know what is being asked this year. This has two implications: (a) the IPs are activity oriented and not result oriented; (b) it is cumbersome for the IPs to effectively organize their staff and logistics over the project period. A prime example is the unexpected dropping of IR1 training activities this year. This caused serious problems for all IPs as they were forced to terminate staff who had been hired in anticipation of an ongoing training efforts. All the IPs stated that their unhappiness with the modality, indicating they are only continuing because of their commitment to USAID. 16 In summary, the management structure for AGP‐LMD is unreasonably complex and mitigates against effective implementation. Specifically, AGP‐LMD is not taking full advantage of IP institutional strengths, technical expertise, and local presence. It is not engaging them on a continuous basis in project planning or in measuring impact (M&E), and their capacity to provide effective BDS services is not being built by participating in the project. The same is true of local consulting firms, who are contracted for specific implementation tasks without being fully engaged in the management, oversight and planning of activities. As a result management is overly centralized in CNFA, leaving senior management staff inordinately burdened by coordinating with the large number of national and regional government entities, institutions, and stakeholders engaged in AGP and AGP‐LMD implementation. Coordinating with this array of actors and interests eats up huge amounts of senior staff time, with little apparent impact on implementation and the achievement of results. Effectively engaging the IPs as long‐term actors, and allowing them to represent the project and coordinate at the regional level, would improve communication and reduce the burden on CNFA to coordinate at all levels. This would give management staff more time to devote to implementation, instead of endless rounds of coordination. 2.4 Effectiveness of Partnerships with GoE AGP Implementers and Other Government Offices The following are the findings of the evaluation team in response to question four: How effective is the partnership with GoE‐AGP implementers and other government offices such as the Ministry of Agriculture, Ministry of Trade, Ministry of Industry, Federal Cooperative Agency, and ATA in terms of collaboration and coordination to implement AGP‐LMD? What factors contribute for success or challenges in partnership, particularly with the GoE‐AGP and what is recommended to resolve the challenges? We thoroughly discussed CNFA effectiveness in coordinating AGP‐LMD implementation with implementing partners, AGP coordination units, relevant sectors and relevant bureaus at federal and regional levels. Note that there are a large number of programs supporting livestock development in Ethiopia including ADLIP, RDPAS, GTP, AGP, and REDFS. In addition to these, AGP‐LMD is supporting specific platforms to strengthen partnerships with other AGP implementers and government offices. These include steering committees, technical committee, MSPM, LTC, JRIS, and others. The project is working particularly hard to sustain participation in regular quarterly monitoring and planning meetings with AGP at both the regional and federal level. The project also supports and participates in the crop livestock Technical Committee meetings at the federal level. In this context annual AGP‐LMD plans and activities are prepared in consultation with the regional stakeholders and implementing partners, and presented to respective Technical and Steering Committees. In spite of these efforts, there were complaints from several quarters regarding a lack of clarity and transparency. As discussed above, the implementation modality with the IPs means they cannot relieve the burden by representing AGP‐LMD or substantively participate in regional coordination bodies. The only role for the IPs is to facilitate and encourage private firms and cooperatives to compete for grant funds. For example, neither the regional stakeholders nor the IPs have complete information regarding how the grants program operates. To complicate things further, there are activities directly implemented by project staff at both federal and regional levels without any consultation with the regional IPs. This disorganized and confused arrangement for representing the project makes it difficult to effectively coordinate with government agencies. Most government offices confirmed that AGP‐LMD attempts to plan and implement activities in partnership with key government entities. For example, joint efforts to develop the pilot Livestock Identification and Traceability System (LITS) and the Rationalization Road Map document show indicate good relationships with the MOA, the Ministry of Livestock, the state minister’s office and the animal health directorate at the federal level. However, the effectiveness of partnerships at the regional level varies from very good to almost non‐existent. 17 However, we are astounded at the large number of government, donor, and institutional actors involved in livestock project implementation, including AGP, at the regional and federal level. These have spawned an inordinate number of working groups, committees, and coordinating bodies – all of which demand participation, but do not seem to impact implementation. In summary, the AGP‐LMD is really stretched to collaborate and coordinate with everyone who expects it. Staff expend an inordinate amount of time and effort talking about what they are doing, all of which takes time away from actual implementation. The need is for the whole apparatus to spend less time coordinating, and more time actually accomplishing something. It does not appear that the level of effort required to improve coordination would have any substantive impact on implementation or goal achievement. 2.4 Relevance of Project Activities The following are the findings of the evaluation team in response to question five: Based on sustainability and cost effectiveness, which activities should be continued and which activities are irrelevant or no more important to contribute to the project objectives in the future? The relevance of project activities is a function of how well they address national development objectives, their relationship with other on‐going GoE and USAID projects, and the needs and priorities of target beneficiaries. AGP‐LMD was specifically designed to contribute to the achievement of both AGP and FtF goals by increasing the productivity and competitiveness of selected livestock value chains: dairy products and meat and live animals. As a result, it functions as the livestock marketing component of AGP. The goal of AGP, which is “to end poverty and enhance growth,” aligns with FTF’s goal to, “sustainably reduce poverty and hunger.” The specific AGP‐LMD objective is to foster growth and reduce poverty by improving the productivity and competitiveness of selected livestock value chains. The overall AGP‐LMD goal is to improve Small Holder Incomes and Nutritional Status. AGP‐LMD attempts to link geographically and thematically with the prime AGP focus on the highly productive areas of Ethiopia, where smallholders have at least minimal commercial engagement. It also attempts to integrate activities with the USAID GRAD and PLI II/PRIME initiatives operating in selected food insecure woredas where the graduates of the Productive Safety‐Net Program (PSNP) reside. In these areas AGP‐LMD attempts to “pull” their products into the marketplace upon graduation from USAID "push" initiatives, assuming it will ultimately increase their incomes. In short, AGP‐LMD activities are expected to generate the market demand needed to expand commercial‐oriented input and service supply and form sustainable supplier and buyer relationships to link producers to formal markets. The focus is on producers graduating ("pushed") from subsistence agriculture, both in the selected high agro‐ecological potential areas targeted by target of AGP, and food insecure localities targeted by FtF. The AGP‐LMD implementation approach seeks to empower the private sector at all levels of the value‐chain as the most effective way to achieve sustainable livestock sector growth, and improve people’s nutrition and health. Although, private sector growth is decisive in market development and commercialization, it has been least addressed in GoE agricultural development strategies. Therefore, AGP‐LMD fills a key strategic gap by focusing on the private sector. It is within this context that we comment on the relevance of project activities. In our opinion there are significant design and implementation weaknesses which directly impact the relevance of project activities. They are: Intervention strategies: The AGP‐LMD IR1 intervention strategies under IR1 are numerous, not clearly defined and articulated, and overlap considerably. It would be far better to consolidate these strategies into: (a) capacity building for value chain actors (awareness creation, technical and business training, and experience sharing); (b) market expansion (domestic and global market linkages); (c) stimulating investment (grant funding, credit access); and (d) promoting access to and consumption of nutritious food. In addition, while gender and environment are mainstreamed in all categories, they need to be better articulated and given more prominence. 18 Indicators: The present performance indicators are inadequate to measure achievement and impact. For instance, the value of livestock/products exported is affected by a wide range of factors external to the production process. Most project indicators are simply recording the number of activities carried out and individuals and firms affected. It is nice to know the number of trainees, amount of grants distributed, total values of agricultural loans provided, number of children reached, etc. to get a sense of the scale of activity. But it would be far better to know the impact of project activities on household income and nutrition. Capacity Building: The extensive training program carried out during the first two years of the project was cited by everyone we interviewed as a major contribution to AGP. There is considerable anecdotal evidence that this is true, even though there is no quantitative data to measure impact. The inexplicable cancellation of much of the IR1 training agenda by USAID during year three interrupted one of the potentially major success areas of the project, ruptured the continuity of effort, and had no discernible impact on implementation of the grants program (the reason cited for cancellation). This needs to be immediately rectified. Grants: The specific objective of the grants program is to leverage private investment to improve the productivity and competitiveness of livestock value chains. The program is being carefully implemented, with the process guided by a project manual. We noted that, according to the manual, CNFA is expected to provide the approved grants efficiently and rapidly. However, the grant process has proven extremely cumbersome, and even approved candidates are impatient with the rate of disbursement. Disbursement delays mean that the program cannot be expected to contribute substantially to the achievement of project goals prior to the end of the project. As confirmed in the field, grants are being considered for projects that are obviously not feasible, and/or have serious management issues. (Abergelle Export Abattoir, Mekele Dairy Plant). In addition, the grants program almost totally overlooks micro and small enterprises in the value chain. And finally, the potential impact of the grants program on market linkages, and its subsequent effect on smallholder producers, is not clear. Beneficiary selection: In principle, value chain development needs to identify and mitigate critical problems along the chain. AGP‐LMD claims to be primarily focused on middle value‐ chain actors, important value chain actors in the meat and live animal value chain (livestock traders, transporters and brokers) have been largely ignored. These actors play a critical role in information, pricing, market linkages and the overall functioning of the live animal market. It will be particularly important to help them adapt to the new Live Animal Trading Proclamation. AGP‐LMD is more focused on live animal exporters and export abattoirs, which already create sufficient demand pull for existing production. A more critical problem is the domestic value chain, from producers through small traders and feed lots to domestic abattoirs and butcheries. In the dairy value chain AGP‐LMD activities are concentrated in the pre‐urban and urban areas processors, and existing dairy producers. These producers have already been commercialized, although the project support could be still relevant. The key question is, "how can the project support expansion of the milk shed collection areas to small rural milk producers?” Activities to increase producer access to AI services to improve genetics, and improve the management of dairy production, are underway. But it will take years for substantial increases in total milk availability and supply to materialize. In the meantime, more aggressive support to improve fertility (including bull services) and milk yield is needed. 2.5 Gender Issues The following are the findings of the evaluation team in response to question six: How has the program addressed gender issues? AGP‐LMD is successfully implementing gender mainstreaming activities in all project activities and components. The project has a gender equality advisor who facilitates the implementation of every component of the three IRs, and individual activities with a special focus on gender. This focus is reflected in indicator targets, and tailored training and mentoring activities for women. The project needs to maintain the present level of effort and expand it to 19 help close the gender gap, with even greater emphasis on capacity building in women’s entrepreneurship and leadership skills. Observations in the field confirm project report information regarding the concerted efforts being made to address gender issues. AGP‐LMD capacity building activities routinely record 30‐40% women participation, and has organized special training in business and leadership for women entrepreneurs. Experience sharing, exposure creation and business linkage activities all engage large numbers of women. The project also emphasizes women’s participation in the grants program, with funds set aside specifically for women entrepreneurs. It is generally acknowledged that women are more involved in livestock farming than men. The dairy value chain is dominated by women. We visited several women who are engaged as owner/operators in milk production, processing and retailing – with full access to and control of the resource and incomes. We also observed women working in the feedlots and abattoirs we visited, both as laborers and managers. Nutrition programs emphasize women, and the PLWHA efforts are reaching equal numbers of men and women. . In summary, AGP‐LMD is doing an excellent job of integrating gender, and is making very concerted and determined efforts to encourage high levels of women’s participation in project activities. However, the traditional roles of women in livestock production and marketing, and cultural norms regarding work responsibilities, merit further consideration in order to encourage rural women to participate in economic activities and leadership, with improved access to and control over resources. For example, the substantive participation of women at market points, livestock collection grounds, milk collection centers and cooperative operations is steadily expanding, but still relatively low. In addition, efforts are needed to specifically determine the impact of this emphasis on household income and nutrition. 2.6 Nutrition Activity Performance The following are the findings of the evaluation team in response to question seven: How was the performance of the nutrition sensitive activities implementation toward achieving expected outcomes? AGP‐LMD nutrition activities are on track to meet the LOP indicator targets related to the number of children under five reached by the project, and the number of people trained in child health and nutrition. However, changing food consumption and preparation customs and behavior is very difficult, especially when women do not have access to or control over resources. As such, the impact of nutrition activities on the target population – and their effectiveness in promoting behavior change – will only be realized over the long term. We visited nutrition training, nutrition clubs, and cooking demonstrations promoting the use of animal source foods. Trainers and extension agents explained the approach being used to encourage behavioral change in the target communities. They also explained the messages being promoted in nutrition education, and how cooking demonstrations are being used to show women how to incorporate animal source foods. The messages, and the information being communicated, is technically sound. The question is how effective it will be in encouraging behavior change. The project also supports school nutrition clubs, sharing nutrition best practices with students based on available resources. Teaching takes place in the morning before class. The idea is that young people are the most likely to change cultural and eating habits, and respond faster to new ideas and changes. For example, the popularity of pizza is having a great impact on diet by introducing young people to cheese consumption. We also met with a PLWHA group in Yetnora, Dejen of Amhara region. The group is broadly representative of the other 5 groups in the woreda. It has 12 active members, who have been trained on credit and saving schemes. They are saving money, and lending the funds to one another to finance micro‐enterprises. We visited with group members about how the activity was impacting their lives. They explained what they are doing and spoke of companionship, of encouragement, of hope, and of how it was helping them support their families. It is not reaching a lot of people, but it is helping. 20 In summary, nutrition activities (including PLWHA) are virtually stand‐alone efforts, having little relationship to the rest of the project. But they are directly impacting FtF target populations, and are giving the Implementing Partners the opportunity to demonstrate their capabilities. Both nutrition and PLWHA activities seem to be practically oriented, and we were not told of any issues. But they are still in the beginning stages, and will need to be expanded, evaluated, and further refined to improve and demonstrate the level effectiveness needed to achieve impact and merit replication. 3. SUMMARY AND CONCLUSIONS 3.1 Program Performance The following provides the conclusions of the evaluation team in response to question one: Are the selected program activities on track to reach the desired FtF results in Ethiopia? The goals of AGP and FtF appear complementary, but in practice make for an awkward implementation marriage. The AGP goal is “to end poverty and enhance growth”, and implementation is focused on improving productivity and economic growth in the high potential areas of Ethiopia, with an emphasis on surplus producers. Meanwhile, the goal of the USAID Feed‐the‐Future (FtF) initiative (the primary source of funding for the project) is “to sustainably reduce poverty and hunger” with implementation focused on improving the incomes of poor rural households, and the nutritional status of children. AGP‐LMD combines these goals with an objective of improving smallholder incomes and nutritional status. It aims to contribute to the achievement of both AGP and FtF goals using a value chain development approach to improve productivity and competitiveness. AGP and FtF target beneficiary populations definitely overlap, but the exact nature of their relationship to one another is not clearly articulated. Furthermore, it is not clear how the project focus on value chains engages poor rural households. In fact, AGP‐ LMD is not fully consistent with the AGP geographic focus areas. It departs from the woreda‐specific AGP geography to encompass naturally occurring milk sheds and livestock growth corridors. These livestock value chains transcend administrative boundaries and geography, which makes perfect technical sense. However, it makes for an awkward geographic marriage, with some activities focused on AGP target woredas, others on food‐ insecure non‐AGP woredas, and still others – especially dairy – on urban and peri‐urban agricultural areas. The key development hypothesis of the project is that market‐driven enterprise development can generate increased producer incomes by pulling previously marginalized populations into commercial value chains, increasing their incomes. The assumption is that this will lead to improvements in nutrition, household food security and health in the target population. However, project contribution to the achievement of AGP and FtF goals hinges upon exactly which households actually benefit from project interventions. In summary, the project is roughly on track to meet LOP indicator targets as articulated in the PMP, especially when compared to the percentage of the project budget expended to date. More importantly, the PMP indicators are inadequate to validate the key development hypothesis of the project. They are output focused, and are not designed to measure activity impact or the project contribution to the achievement of AGP, FtF and project goals. 3.1.1 General The conclusions of the evaluation team are that:  The link between AGP and FtF goals, and their link to the AGP‐LMD objective of increasing value chain productivity and competitiveness, is not clearly articulated in project documents. This results in a collection of activities with little synergy, an awkward geographic focus, and a dispersed beneficiary population.  It can be argued that AGP‐LMD supports rural economic growth, and that such growth is a necessary – but not sufficient – condition for rural poverty reduction. It can also be 21 argued that increasing value chain productivity and competitiveness can effectively contribute to the achievement of both AGP and FtF goals. However, project reports and documents do not make these arguments.  Individual project components, strategies and activities are not clearly stated or defined, especially with regard to the separate dairy and meat/live animal value chains. This creates confusion among implementing partners and stakeholders working on different value chain activities, and compromises effectiveness. The numerous AGP‐LMD activities (dairy value chain, meat and live animal value chain, capacity building, grants and financing, enabling environment, and nutrition) resemble separate projects, with little synergy, which dilutes impact and complicates project management.  CNFA is roughly on track to meet PMP targets, especially when compared with the percentage of the budget which has been expended. This is in spite of unrealistic implementation expectations; unreasonable expenditure targets during the first two years; and some ill‐considered decisions (e.g., suspending IR1 training in Year 3).  Major PMP indicators, especially the mandatory FtF indicators, are simply inadequate to measure project impact or contribution to the achievement of AGP and FtF goals. 3.1.2 IR1: Increased Productivity and Competitiveness of Selected Livestock Value Chains We believe that AGP‐LMD is making a substantial contribution to the achievement of the AGP goal. However, its contribution to the achievement of both the AGP and FtF goals is simply not being captured by the PMP indicators. There is therefore a pressing need to develop and incorporate custom indicators in the PMP to more directly measure impact and achievement. Given the complexity of measuring increased productivity in the livestock sector, AGP and AGP‐ LMD also need to develop a methodology for measuring achievements and impact. We believe that AGP‐LMD can increase its contribution to the achievement of the FtF goal by focusing efforts on the point of first sale/contact with producers in the value chain (small traders, dairy collection centers) and nearby actors (brokers, transporters). This will more directly impact target beneficiary household incomes. AGP‐LMD also needs to place greater emphasis on facilitating private input supply and service delivery providers to support increased productivity (increased fertility, decreased mortality, increased milk/meat production). The project has done enough to encourage large‐ scale investment at the processor level. Over the remaining LOP it needs to focus on: (a) training and support to develop micro and small scale enterprises closer to the producer (input supply, animal health and AI services, small scale fattening, brokers, transporters); and (b) providing BDS services to existing clients and grantees to help them succeed. (See Recommendation 4) Capacity Building: There is a great deal of anecdotal evidence the capacity building activities are generating positive impact. However, training is being treated more like an event, rather than a process leading to a desired result. Specifically, the impact of training on household income needs to be measured. Also, training is too often being provided free of charge to industrial enterprises (abattoirs, processors) who are well able to pay for them. The scale of training activities was drastically reduced during Year 3. It needs to be reinstated and expanded into a more comprehensive combination of technical and business training leading to tangible changes in the managerial skill and entrepreneurship of the actors to develop the selected value chains. (See Recommendation 4) Grants program: The grants program is leveraging new private investment in the livestock value chains. The total budget allocation of slightly more than $6 million to this activity makes it reasonable to expect that it will contribute substantially to the achievement of project goals. However, the grant process has proved to be terribly cumbersome. As a result, by the mid‐point of the project only $1,773,983 in grant funds has been committed (signed), and only $299,796 actually expended. It will take concerted effort to fully disburse the remaining grant funds 22 within the remaining life of the project. Unfortunately, there is little prospect of the grant funding producing significant measurable results before the end of the project. We recommend bringing the program to an orderly close (see Recommendation 6), finalizing the process that has been started, but not adding to it. Rather, during the remaining LOP the project should focus on an integrated micro and small enterprise development program, with technical and business training combined with small capital development grants (less than $1,000) to provide the capital needed for small value chain to establish and expand their activities. (This approach is already included for CAHWs, and PLWHAs). Specifically, it should be tied to enterprise development and management training, tied to the achievement of benchmarks for disbursement, and designed in collaboration with and fully managed by implementing partners. 3.1.2.1 Dairy Value Chain In the dairy value chain milk producers and processors are concentrated in urban and peri‐ urban areas, not rural areas, and are already part of the value chain. It is reasonable to support these actors, since the industry is still nascent and they are not yet strong enough to fully exploit the business opportunities available to them. However, it is not realistic to expect any pull effect on rural subsistence households within the project period. We believe that AGP‐LMD is doing a very good job of supporting dairy industry development, and facilitating the increased processing capacity which is needed to supply growing demand. But the initial impact will tend to be to displace milk from existing channels (raw milk) to processors. There will be some increase in milk production as feeding improves, and the collection area expands. But the immediate increase in total milk production will be limited to existing producers. A dependable market outlet for raw milk (processors) should eventually provide incentives and security for current producers to upgrade their herd (genetic improvement), try harder to get cows pregnant (improve fertility) and improve feeding practices (increase milk production per animal). However, the lag time for achieving significant results is likely to be a generation (three years). Producers are most likely to upgrade their herd and improve feeding rather than increase numbers. Some may buy pregnant dairy cows to increase their production, but this will not add to total milk production. However, increased processing capacity should eventually provide incentives for rural households at the fringe of the milk sheds to buy dairy animals and begin selling milk. The likely impact will therefore be an expansion in the size of the milk shed (collection area) to encompass more smallholder producers. Since most dairy producers do not produce all of their own feed, the increased demand for feed and forage will encourage new feed production, and eventually provide producers with the cash needed to enter dairy production themselves (buy cows). This will create a ripple effect, expanding the collection area over time, and involving more of the FtF target beneficiary population. There are excellent prospects for expanding milk production and consumption, but it will take time. Expanded milk processing capacity also creates a demand pull for input and service supply to increase productivity, as well as providing opportunity for new producers to enter the market. This growth enhancement is fully consistent with AGP goals. However, it does not readily contribute to FtF goals of reducing poverty and hunger. Milk producers are not poor, and pasteurized milk is a relatively high‐end product. However, with a sale price of Birr 7‐8 per ½ liter, it is very competitive with Coca‐Cola and consumption should increase. Milk sheds may eventually expand deeper into rural areas, impacting more producers. However, they are also unlikely to be poor. AGP‐LMD needs to do more to address weaknesses in the market link between producers and processors, and between processors and consumers. Specifically, it needs to do more to organize and/or strengthen collection centers as the gateway for raw milk entering the processed milk value chain. An expanded network of milk collection centers will expand the supply base and improve market efficiency. 23 However, AGP‐LMD should not seek to further expand large‐scale dairy processing capacity. It is time to consolidate efforts, and concentrate on ensuring the availability of BDS services to those already in existence. There is little or no practical and management experience with operating dairy processing plants in Ethiopia, and the availability of spare parts and repair/maintenance services is questionable. Therefore, technical and management support to processors over the remaining life of project will be essential to help safeguard investments already in process. We conclude that the following dairy chain activities would more directly impact the FtF target group: o Support the establishment of milk collection centers at the edge of expanding milk shed areas.3 This will provide additional incentives for increased dairy production (although not for poor farmers) and a ready market for input and service supply – including forage production (a smallholder crop alternative). o Expand entrepreneurship training in target woredas, combined with technical training in cottage‐level processing of raw milk into cheese and butter. This will provide women with expanded income opportunities, and improve the local availability of dairy products. o Add training on bull management to improve fertility, especially for high‐producing dairy animals, into technical training programs. The dairy value chains needs to ensure that fertility is maintained at a high level. AI contributes to genetic improvement, but after two attempts it is time for the bull. 3.1.2.2 Meat and Live Animal Value Chain We believe there is potential for a more efficient live animal and meat value chain to encourage increased livestock productivity in the highland areas. However, we are not certain just how much productivity can be increased in the lowland areas, which are already the primary source of cattle for live animal export, and of goats and sheep for carcass exports. It appears there is already excess demand for lowland animals (shoats and cattle), with traders reporting a highly competitive market in the Afar and Borena areas. In the highland areas AGP support for producers seems too weak to generate a substantial increase in the availability of sheep and goats. As a result, we believe that AGP‐LMD will be unable to meet its export goals for beef and live shoats. Furthermore, expanded export abattoir capacity, which the project is supporting, will not increase the number of animals available in the short run. It appears that lowland areas are already producing at or near their offtake limit, and it will take time to “pull” increased shoat production in the highlands. Increased processing capacity is more likely to increase demand for the existing supply lowland animals, creating supply constraints (limited ability to buy sufficient animals at the price processors are willing to pay) which will make it very difficult for Abergelle (Tigray), Abyssinia, and new facilities under construction to prosper. Ethiopian exports of live cattle, and goat and sheep carcasses, are solidly competitive in the regional market. But they are not likely to increase substantially without a rationalization and streamlining of taxes and regulations. However, a relaxing of current government restrictions (e.g., transport fees, FOREX repatriation, and minimum weight for live cattle export) would almost certainly increase exports by drawing animals into formal export channels. Beef exports are not presently, nor likely to be, competitive – at least in the near future. The major factors which keep Ethiopian beef from being competitive are quality (tenderness and taste, related to age and condition) and cost. Simply put, Ethiopia does not have the genetics, or the feed resources, to produce quality meat at a competitive price. In this setting AGP‐LMD meat and live animal activities may eventually result in increased livestock productivity, but only if improved in processing efficiency is translated into higher producer prices. Even then, it will depend upon whether producers have sufficient resources to improve nutrition, increase fertility, reduce mortality, and increase yield.                                                              3 We understand this is currently in the planning stage. 24 In summary meat and live animal activities are too concentrated at the top end of the value chain, working with abattoirs and large scale feedlots, too focused on exports, and too far distant from producers to exert effective “pull”. They should be refocused on the domestic market, including: encouraging investment in small‐scale input supply and animal health enterprises and fattening operations to increase productivity; increasing the efficiency of domestic abattoirs; and improving meat cutting and retailing (see Recommendation 4). 3.1.3 IR2: Improved Enabling Environment for Livestock Value Chains AGP‐LMD efforts to improve the enabling environment for livestock value chains are helping value chain actors adjust and adapt to new regulations (Live Animal Trading Proclamation), and laying the foundation for policy change. Project support for the establishment and organization of MSPs at the national and regional level, and regional LWGs) is proving an effective way of bringing government representatives, business interests, producers, and other stakeholders together to discuss practical issues and problems, and recommend solutions. In particular, the bottom‐up LWG approach to bringing policy and regulatory before government has great potential for influencing change. In addition, AGP‐LMD support for the pilot LITS, and technical assistance to help address issues related to the livestock and livestock product exports and the national SPS system, is helping Ethiopia reduce regulatory burdens and costs, and maintain its competitive advantage in live cattle and shoat meat exports. In conclusion, AGP‐LMD efforts to improve the enabling environment for livestock value chains are encouraging, but any substantive impact on rationalizing and streamlining government policy and regulation is yet to be realized. 3.1.4 IR3: Improved Quality and Diversity of Household Diet AGP‐LMD nutrition activities (including PLWHA) represent an important initial effort to practically address food preparation and consumption practices, and promote social and behavioral change to improve maternal and child nutrition. That said, they virtually stand‐alone efforts, having little relationship to the rest of the project. However, they directly impact FtF target populations, and are an opportunity for Implementing Partners to demonstrate their capacity. Both nutrition and PLWHA activities appear to be technically sound and practically oriented. But they are still in the beginning stages of implementation, and need to be scaled up, expanded, evaluated, and further refined to improve and demonstrate effectiveness. The one weakness we note is that the nutrition messages and cooking demonstrations incorporate animal source foods which are not readily available to poor rural families (e.g., pasteurized milk). Eggs and poultry are included, but not emphasized. We think that incorporating pilot poultry production and nutrition messages, promoting egg consumption for infants, toddlers, pregnant women, and lactating mothers would improve project impact by emphasizing the use of animal source foods which are most available to poor families. We believe it would lay the foundation for improved impact, as well as improve the linkage of these activities to the other elements of AGP‐LMD (see Recommendation 9). 3.2 Program Impact on Government Policy, Process and Administration The following are the conclusions of the evaluation team in response to question two: Did program activities lead to any significant changes in government policy, process, or administration that can be expected to contribute to increased production and investment in the sector and/or increased export competitiveness? AGP‐LMD is very carefully working on government policy, process and administrative issues related to livestock value chains. Activities are helping provide a setting and platform for broad‐based popular participation in discussing issues impacting the sector, for addressing systemic issues which constraint productivity and competitiveness, and putting bottom‐up pressure on the policy process. There are instances where the project has been able to help resolve a local issue. The important thing is for AGP‐LMD to be present in the discussions so as to influence action when the opportunity presents itself. However, nothing significant – such as 25 a major regulatory or tax change – has been achieved to date, and any significant impact on the productivity and competitiveness of livestock value chains is yet to be realized. We believe that AGP‐LMD needs to stay engaged, concentrating on facilitating and supporting MSP and LWG activities, and helping them raise issues and problems to higher levels. The project also needs to stay modestly engaged with the LITS and SPS issues. In addition, at every opportunity it should promote and support the development of a new unified policy and regulatory system for livestock to replace the present tangle (see Recommendation 8). 3.3 Effectiveness of Implementation Approach The following are the conclusions of the evaluation team in response to question three: What is the demonstrated effectiveness of the current implementation approach (e.g. working with local regional organizations like ORDA, REST, HUNDEE, Self‐Help – Africa, and regional consortiums)? The current implementation approach is not taking full advantage of local organizations and partners. It is being strangled by paperwork and process, with substantial delays in decision‐making that compromises the effectiveness of project implementation. We have no doubt that an implementation which fully engages the IPs in planning and implementation can be very effective, provided the operational issues are resolved. Specifically:  Subcontracting mechanism with IPs: The institutional strengths, technical and management capabilities of the regional Implementing Partners are simply not being fully engaged under the current sub‐contracting modality. Annual purchase orders do not provide an opportunity for long‐term planning or continuing involvement and participation in program planning, M&E, or program support. More importantly, there is no way that AGP‐LMD can expand the scale of their operations to match the existing capabilities of the IPs. In summary, substantive IP participation is essential for project success, and would be much better engaged using sub‐grants to finance program implementation, or a framework contract complemented by purchase orders for specific tasks (see Recommendation 5).  Death by a thousand cuts: All the stakeholders are doing the best they can, but project implementation is drowning in process and paperwork. Specifically, the number of individual actions for which specific USAID approval is required is daunting, and dealing with it saps AGP‐LMD of energy and momentum. USAID and the Contractor need to review the contract and streamline the process for approving STTA, travel, grants, etc. It simply does not have to be this complicated and clumsy.  Reporting: The templates for project reporting and documentation need serious attention. Specifically, project documents are ineffectual in articulating a clear rationale for the project and its component parts, and reporting on what the AGP‐LMD is achieving, where, and why it is important. In summary, AGP‐LMD is not projecting a clear and positive image. We believe the project has a story to tell, but it is just not being told (see Recommendation 1).  Indicators: The PMP indicators are entirely inadequate for measuring achievement and impact. We believe that a set of custom indicators are needed to better explain the expected contribution of the grants program (see Recommendation 2).  Personnel: Increasing the number of AGP‐LMD staff in the regions will not impact implementation unless they have someone to work with. It will be most effective if additional staff are used to provide technical assistance and advice to the Implementing Partners as their role and responsibility in implementation is expanded (see Recommendation 5). 26 3.4 Effectiveness of Partnerships with GoE AGP Implementers and Other Government Offices The following are the conclusions of the evaluation team in response to question four: How effective is the partnership with GoE‐AGP implementers and other government offices such as the Ministry of Agriculture, Ministry of Trade, Ministry of Industry, Federal Cooperative Agency, and ATA in terms of collaboration and coordination to implement AGP‐LMD? What factors contribute for success or challenges in partnership, particularly with the GoE‐AGP and what is recommended to resolve the challenges? We were astounded at the number of government, donor, and institutional actors engaged in implementing AGP at the regional and federal level. In combination with other ongoing programs and initiatives, this has spawned an inordinate number of working groups, committees, and coordinating bodies which demand participation. But that participation often has little apparent impact on implementation effectiveness. There is a lot of time and effort spent talking about cooperation and collaboration, all of which takes resources away from implementation. There is an urgent need is for the whole apparatus to spend less time in meetings and discussion, and more time actually implementing activities. In this setting, AGP‐LMD is working hard to sustain participation in regular quarterly monitoring and planning meetings with AGP at both the regional and federal level. The project also supports and participates in the crop and livestock Technical Committee meetings at the federal level. Annual plans and activities are prepared in consultation with the regional stakeholders and implementing partners, and presented to respective Technical and Steering Committees. However, nobody seems happy with the present set‐up. In particular, the failure to take full advantage of the IPs means they cannot relieve the burden on project management by representing AGP‐LMD, or substantively participating in, regional coordination bodies. To complicate things further, there are activities being directly implemented by project staff at both federal and regional levels without any consultation with the regional IPs. This disorganized and confused arrangement makes it hard to effectively represent what the project is doing and coordinate with government agencies. Most government offices confirmed that the AGP‐LMD project attempts to plan and implement activities in partnership with key government entities, especially at the federal level. However, the effectiveness of working relationships at the regional level varies from very good to almost non‐existent. In summary, there is certainly a need to maintain good working relationships with government counterparts. But there are so many coordinating structures that the project is really challenged by trying to cover them all. We believe that expanding the role and responsibility of the IPs, and empowering them with a greater role in implementation, would help resolve many of the existing issues at the regional level. But, AGP‐LMD needs to exercise caution when it comes to participating in all the things where it is requested. Simply put, it is not at all clear that collaboration and coordination is having any positive impact on implementation. 3.5 Relevance of Project Activities The following are the conclusions of the evaluation team in response to question five: Based on sustainability and cost effectiveness, which activities should be continued and which activities are irrelevant or no more important to contribute to the project objectives in the future? All the activities we visited and reviewed are designed to contribute to the achievement of the project objective, and hence are relevant. That said, we offer some comments on sustainability and cost effectiveness, and some specific recommendations to make the project more relevant, increase impact and better reach the target population – increasing effectiveness. First of all, nothing that the project is doing is sustainable – that is, likely to continue after the LOP. One expects that many of the beneficiary enterprises will continue operating. But 27 the BDS services and financing being provided by the project are not being implemented in way that ensures the future effectiveness as BDS providers. AGP‐LMD indicators measure outcomes, not impact. Second, the project is not reaching a scale of operations, having only expended 34% of its $41,173,362 budget to date. It must ramp up expenditures and the scale of operation to generate impact and be judged as cost effective. Third, cost effectiveness depends upon the scale of achievement and impact. And, as so often noted in this report, AGP‐LMD is measuring outcomes, not impact. As a result, we have no basis for judging the cost effectiveness of what is being done. That said, there are actions which can be taken to improve cost effectiveness and sustainability. They are as follows: Streamline Intervention Strategies: AGP‐LMD has ten interrelated intervention strategies under IR1. The structure is simply too complex to guide and focus activity. We recommend consolidating them into: (a) capacity building for value chain actors (awareness creation, training and experience sharing on business and entrepreneurship); (b) market expansion (establishing and strengthening domestic and global market linkages); (c) stimulating investment (provision of grant and creating access to finance; and (d) promoting access to and consumption of nutritious food. Cross‐cutting activities will remain mainstreamed in all categories. This will improve activity definition, focus, expected outcomes, and impact. Review and Revise Results Indicators: As previously emphasized, the indicators being used are simply inadequate to measure performance ‐ most measure outcomes, with impact being assumed. We recommend refocusing monitoring and evaluation efforts on revising the indicators, adding new ones to better represent what is being achieved, and focus on developing and implementing a methodology for measuring impact. Expand Capacity Building: The extensive program of capacity building carried out during the first two years of the project was extremely relevant to the achievement of project objectives. The challenge is to go beyond anecdote to measure that impact. However, the reduction in training during year three negatively impacted what promised to be one of the best performing parts of the project, interrupted continuity of effort, and did not improve implementation of the grants program. The program needs to be expanded into a more integrated MSE development effort (see Recommendation 4). Grants Program: The grants program is seeks to leverage private investment to improve the productivity and competitiveness of livestock value chains. There is no need for additional grant funds, or time to process new grants. Rather, AGP‐LMD needs to bring the program to an orderly end and focus on establishing a sustainable approach for providing BDS services to the grantees to help ensure their prospects for success. Beneficiary Selection: The principle of value chain development is identifying and mitigating the critical problems along the chain. However, important value chain actors in the meat and live animal value chain, such as livestock traders, transporters and brokers need to be incorporated into project training and B2B activities. In addition, more effort is needed to improve the efficiency and quality of domestic abattoirs and butchers. 3.6 Gender Issues The following are the findings of the evaluation team in response to question six: How has the program addressed gender issues? In our opinion AGP‐LMD has done a very good job of addressing gender issues. The project has a gender equality advisor who facilitates the implementation of every component of the three IRs, as well as individual activities, ensuring a special focus on gender. This focus is reflected in indicator targets, and tailored training and mentoring activities for women. We believe the project should maintain and expand its efforts, building on what has been done to help close the gender gap, with even greater emphasis on building in women’s entrepreneurship and leadership skills. It is generally acknowledged that women are more involved in livestock farming than men. In that light, AGP‐LMD capacity building activities routinely record 30‐40% women 28 participation, and the project has organized special training in business and leadership for women entrepreneurs. Experience sharing, exposure creation and business linkage activities all engage large numbers of women. The project also emphasizes women’s participation in the grants program, with funds set aside specifically for women entrepreneurs. Nutrition programs emphasize women, and the PLWHA efforts are reaching equal numbers of men and women. . In summary, AGP‐LMD is doing an excellent job of integrating gender, and is making very concerted and determined efforts to encourage high levels of women’s participation in project activities. However, the traditional roles of women in livestock production and marketing, and cultural norms regarding work responsibilities, merit further consideration in order to encourage rural women to participate in economic activities and leadership, with improved access to and control over resources. In particular, the project needs to focus on measuring the impact of gender activities on household income and nutrition. 3.7 Nutrition Activity Performance The following are the findings of the evaluation team in response to question seven: How was the performance of the nutrition sensitive activities implementation toward achieving expected outcomes? AGP‐LMD nutrition activities are on track to meet the LOP indicator targets related to the number of children under five reached by the project, and the number of people trained in child health and nutrition. However, changing food consumption and preparation customs and behavior is very difficult, especially when women do not have access to or control over resources. As such, the impact of nutrition activities on the target population – and their effectiveness in promoting behavior change – will only be realized over the long term. We believe that incorporating an emphasis on the consumption of eggs and chicken, the animal source food most accessible to the poor, will improve performance and help lay the foundation for future nutrition activity design. Specifically, we recommend a pilot effort to incorporate messages and training on poultry management, combined with efforts to provide access to improved birds (see Recommendation 9). In summary, nutrition activities (including PLWHA) are virtually stand‐alone efforts, having little relationship to the rest of the project. But they are directly impacting FtF target populations, and are giving the Implementing Partners the opportunity to demonstrate their capabilities. Both nutrition and PLWHA activities are technically sound and practically oriented. But they are still in the beginning stages, and will need to be expanded, evaluated, and further refined to improve and demonstrate the level effectiveness needed to achieve impact and merit replication. 4. RECOMMENDATIONS The Evaluation Team recommends the following actions to increase the contribution of AGP‐ LMD to both AGP and FtF objectives, and to achieve the project goal and objective: 1. Develop and incorporate in project documents and reporting a clear articulation of the relationship between the AGP and FtF goals, their linkage with the project goal and objectives, and exactly how project activities are expected to contribute to the achievement of those goals and the project objective.  At present the project rationale, and the expected contribution of activities to LOP targets, is not clearly articulated. 2. Develop custom PMP indicators to measure and record the outcome and impact of the training programs and the grants program; and their contribution to the achievement of project goal and objective.  We suggest an increased sales/income indicator for training. And, we suggest increased product sales (volume and value), increased input purchases (volume and value), and a re‐evaluation of the job creation indicator (person/days of employment). 29 3. Develop and implement a methodology to measure (rather than simply estimate) the impact of project activities on livestock productivity (fertility, mortality, and yield/animal) and rural household income.  Specifically, we recommend joint design and implementation with local Implementing Partners in line with Recommendation 5. This will improve the effectiveness of FtF reporting, and also help AGP measure the impact of its activities. 4. Greatly expanding support for micro‐ and small enterprise (MSE) development in the rural areas adjacent to the producer (first point of sale) as the focus of implementation during the remaining Life‐of‐Project.  Specifically, we recommend combining technical and business management training with small capital investment grants ($500 ‐ $1,000). This will promote value chain development, more directly impact the FtF target population, increase the number of beneficiaries, and contribute more clearly to the achievement of the AGP goal.  Note: Implementing this recommendation will most likely require an amendment to the Contract SOW.  Note: MSEs include feed and fodder suppliers, veterinary drugs suppliers, AI technicians, bull service providers, private veterinarians and animal health professionals, livestock traders, brokers, transporters, butchers, and small‐scale (cottage) dairy processors. 5. Substantially increasing the role and responsibilities of the local Implementing Partners, making them directly responsible for the design and implementation of the MSE development program recommended above.  Specifically, we recommend changing the current sub‐contracting modality to a sub‐ grant (preferred) or a multi‐year subcontract combining cost reimbursement for core operations with purchase orders for specific activities. This will dramatically increase impact, and take full advantage of IP institutional and technical capacity and operational presence, enhancing sustainability.  Note: Implementing this recommendation will most likely require an amendment to the Contract SOW. 6. Bring the large‐scale grants program to an orderly conclusion.  Specifically, we e recommend that LMD not consider any grant in excess of $100,000. AGP‐LMD has done enough to leverage new, larger‐scale value‐chain investments. Finalize what is in process, and bring it to a close. Focus during the remaining LOP on providing TA to help the grantees succeed. 7. Design and implement a professional BDS program for grantees and industrial scale value chain operations (milk processors, feedlots, feed processors, and abattoirs).  AGP‐LMD needs to develop a program of assistance with each client, defined in a written agreement which clearly states what LMD will do in terms of technical assistance and advice, client obligations, and results benchmarks. This will lay the foundation for private BDS service providers to emerge. 8. Keep IR2 activities modest and focused.  Facilitate MSP and LWG activities, helping t hem develop practical solutions to issues and problems. Continue support for LITS, and modest engagement with SPS issues. Steadfastly promote and support the development of a new unified policy and regulatory system for livestock to replace the present tangle. 9. Design and incorporate pilot poultry activities into IR3, emphasizing patio production and home consumption of eggs.  This will tie IR3 more directly to the program; emphasize the use of the animal source food (eggs and chicken) most accessible to the poor to improve nutrition; and lay the foundation for future nutrition activity design. The pilot poultry activity must be based on value chain concepts, perhaps engaging PLWHAs to raise chicks into mature chickens for program use. 30 Implementing the Recommendations We have crafted the recommendations to make implementation a fairly straightforward process. But, it will take commitment and decisiveness to get it done: the will to embrace change; and the decisiveness to press ahead and make it happen. AGP‐LMD cannot afford to have USAID and CNFA headquarters get bogged down in lengthy discussions, complicating matters and putting off what needs to be done. The changes recommended are in everyone’s best interest. They will stimulate creativity and action, promote achievement, and make everyone look better. With this in mind, we believe there are two steps to implementing the recommendations, as follows: 1. Annual Work Plan: We suggest implementing Recommendations 1, 2, 3, 6, 7, 8 and 9 in the context of developing the next Annual Work Plan. USAID will need to provide guidance to CNFA in order to carry this out, but they are not unduly complicated and do not appear to require any substantive changes to the project SOW or budget. The Contracting Office may wish to provide additional guidance, but there is no reason to delay getting started on these things. 2. Contract Amendment: We believe that implementing Recommendations 4 and 5 will require an amendment to the contract SOW to describe the MSE development program, establish the guidelines for expanding the role and responsibilities of the Implementing Partners, and amending the budget to accommodate these changes. USAID may also wish to consider a moderate extension of the project period (six months) to compensate for the time needed to make these changes. The amendment process starts with USAID preparing an amended SOW and Budget for the contract. The Contracting Office will provide the necessary guidance to ensure that everything is done in order. This may take some time, but need not be complicated. In fact, with broad general agreement between the parties the next Annual Work Plan can include activities in anticipation of a Contract amendment during the planning period, and provide for amendment of the Annual Work Plan once the Contract is amended. As stated earlier, this need not be a long, drawn‐out, complicated process. But it will require will and decisiveness to get it done, setting the stage for implementation during the remaining LOP. Summary CNFA is doing a credible job of implementing AGP‐LMD, and the project is clearly contributing to the achievement of the AGP goal. But the project contribution to the achievement of the FtF goal is not clear, project activities are fragmented; and impact and results are not easily measurable. Therefore, we recommend that implementation during the remaining LOP focus on the development and implementation of a rural MSE development in rural areas adjacent to the producer to increase results and impact. Implementation at the scale recommended will require dramatically expanding the role and responsibilities of Implementing Partners to ensure continuity through the LOP.