July 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by a Technical Team under the USAID/Nigeria Monitoring, Evaluation & Learning Activity. EVALUATION NIGERIA AGRICULTURAL POLICY PROJECT MID-TERM PERFORMANCE EVALUATION ii Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation NIGERIA AGRICULTURAL POLICY PROJECT (Contract No: AID-620-LA-15-00001) MID-TERM PERFORMANCE EVALUATION Prepared by: Professor Aderibigbe Olomola, Team Leader/Agriculture Policy Specialist Professor Olajide Ajao, Agriculture Economist/Policy Specialist Dr. Ayodele Fashogbon, Agriculture Economist/Survey Specialist Mr. Olufemi Gisanrin, Sr. Monitoring, Evaluation & Learning Specialist, MEL Activity Submitted by: Paul DeLucco Chief of Party, USAID/Nigeria MEL Activity DevTech Systems, Inc. Submitted to: Dr. Chidimma Anyanwu Contracting Officer Representative USAID/Nigeria July 2019 Cover Photo: Drivers of malnutrition - Field work in Kebbi State. Photo by Feed the Future Agricultural Policy Project. Disclaimer: The views expressed in this report are those of the Evaluation Team and do not necessarily reflect the views of USAID, the Government of Nigeria, or any other organization or person associated with this project. iii Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation TABLE OF CONTENTS ACRONYMS …………………………………………………………………………….iv 0.0 EXECUTIVE SUMMARY ………………………………………………………… vi 1.0 EVALUATION OVERVIEW..................................................................... 1 2.0 BACKGROUND .................................................................................. 3 3.0 ACTIVITY OVERVIEW……………………………………………………..4 3.1 Results Framework …………………………………………………………………...4 3.2 Agricultural Policy Activity’s Strategy ………………………………………………...6 3.3 Main Activities ………………………………………………………………………...6 3.4 Critical Challenges/Assumptions ……………………………………………………..8 3.5 Performance Indicators …………………………………………………………….....8 4.0 EVALUATION PURPOSE AND METHODOLOGY......................................11 4.1 Evaluation Questions ……………………………………………………………11 4.2 Evaluation Methodology……………………………………………………………...12 4.2.1 Methodology for the Sample Selection ................................................. 13 4.2.2 Sample Selection of Partners for Key Informant Interviews.................... 13 4.3 Desk Review ……………………………………………………………………17 4.4 Qualitative Interview Guides ……………………………………………………17 4.5 Key Informant Interviews………………………………………………………..........17 4.6 Mapping of Evaluation Questions………………………………………………..........19 4.6.1 Distribution of Key Informants across Selected States........................... 20 4.7 Sample Size for Training Beneficiaries of Activity…………………………………...21 4.8 Qualitative Data Analysis……………………………………………………………22 4.9 Data Gathering, Analysis and Oversight………………………………………..........22 4.10 Guiding Principles and Values ……………………………………………………...22 4.11 Study Limitations …………………………………………………………………...22 5.0 FINDINGS, CONCLUSIONS, RECOMMENDATIONS ................................24 5.1 Evaluation Question 1 (EQ1) ………………………………………………………..24 5.1.1 Findings for EQ1 ................................................................................ 24 5.1.2 Conclusions for EQ1 .......................................................................... 28 5.1.2.1 Conclusions Related to the Activity’s Performance Indicators ............. 28 5.1.3 Recommendations for EQ1................................................................. 29 5.1.3.1 Recommendations Related to the Activity’s Performance Indicators .... 29 5.2 Evaluation Question 2 (EQ2) ……………………………………………………30 5.2.1 Findings for EQ2 ................................................................................ 30 iv Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 5.2.2 Conclusions for EQ2 .......................................................................... 31 5.2.3 Recommendations for EQ2................................................................. 32 5.3 Evaluation Question 3 (EQ3) ……………………………………………………32 5.3.1 Findings for EQ3 ................................................................................ 32 5.3.2 Conclusions for EQ3 .......................................................................... 33 5.3.3 Recommendations for EQ3................................................................. 33 5.4 Evaluation Question 4 (EQ4) 33 5.4.1 Findings for EQ4 ................................................................................ 33 5.4.2 Overall Findings for EQ4 .................................................................... 35 5.4.7 Conclusions for EQ4 .......................................................................... 35 5.4.8 Recommendations for EQ4................................................................. 36 5.5 Evaluation Question 5 (EQ5) ……………………………………………………36 5.5.1 Findings for EQ5 ................................................................................ 36 5.5.2 Conclusions for EQ5 .......................................................................... 36 5.5.3 Recommendations for EQ5................................................................. 37 ANNEXES ANNEX 1: Statement Of Work ………………………………………………………...1 ANNEX 2: NAPP Using AMEL Plan Indicators …………………………………………18 ANNEX 3: Breakdown of Capacity Building Respondents ……………………………..25 ANNEX 4: List of Key Stakeholders Contacted ……………………………………….30 ANNEX 5: List of Key Informants Contacted …………………………………………32 ANNEX 6: Literature Reference …………………………………………………….....43 ANNEX 7: Evaluation Team Biographical Info .………………………………………...46 ANNEX 8: Conflict of Interest Statements (COI) ….………………………………….48 ANNEX 9: Statement of Differences by IPs ……………………………………………91 v Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation TABLES Table 1: Agricultural Policy Project’s Planned Interventions..................................................................2 Table 2: Agricultural Policy Activity Performance Indicators.................................................................9 Table 3: Population Mix of Capacity Building Participants ................................................................... 13 Table 4: Distribution of Respondents by Geographic Zone................................................................ 14 Table 5: Analytical Approach and Data Sources to Answer Evaluation Question ......................... 16 Table 6: Organizations Included in the Field Work Sample................................................................. 18 Table 7: Evaluation Questions and Target Respondents...................................................................... 19 Table 8: Distribution of Respondents Interviewed by State & Sector............................................... 20 Table 9: Participants’ Gain from the Capacity Building Workshops.................................................. 26 FIGURES Figure 1: USAID/Nigeria Economic Growth Results Framework (2015–2019 CDCS) ...................4 Figure 2: FTF Nigeria Agricultural Policy Project Results Framework.................................................5 Figure 3: Three-stage Evaluation Methodology....................................................................................... 12 Figure 4: Breakdown of the Capacity Building Respondents, by Category ...................................... 15 Figure 5: Distribution of Respondents Interviewed during Mid-term Evaluation, by State .......... 21 Figure 6: Stakeholder Participation in Training Workshops, by Type and Topic ........................... 26 Figure 7: Usefulness of the Training Workshops................................................................................... 26 vi Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation ACRONYMS ABU Ahmadu Bello University AGRA Alliance for a Green Revolution in Africa ANR Michigan State University College of Agriculture and Natural Resources APP Agricultural Promotion Policy of Nigeria (Federal-level) APRNet Agricultural Policy Research Network ARCN Agricultural Research Council of Nigeria ATA Agricultural Transformation Agenda ATLAS.ti Software for Analyzing Qualitative Data BSU Benue State University CDCS Country Development Cooperation Strategy CNFA Consortium for New Frontiers in Agriculture CSEA Centre for the Study of the Economies of Africa CSO Civil Society Organizations DO Development Objectives DQA Data Quality Assessment ECOWAS Economic Community of West African States EGRP Economic Growth and Recovery Plan EQ Evaluation Question FCT Federal Capital Territory FGD Focus Group Discussion FMARD Federal Ministry of Agriculture and Rural Development FSP Food Security Policy FTF Feed the Future FTFMS Feed the Future Monitoring System FUAM Federal University of Agriculture, Makurdi FUNAI Federal University Ndufu-Alike GDP Gross Domestic Product GIS Geographic Information System GON Federal Government of Nigeria IFPRI International Food Policy Research Institute IR Intermediate Result KII Key Informant Interview KSU Kebbi State University KSUSTA Kebbi State University of Science and Technology M&E Monitoring and Evaluation MANR Ministry of Agriculture and Natural Resources vii Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation MEL Monitoring, Evaluation and Learning MOU Memorandum of Understanding MSU Michigan State University NAERLS National Agriculture and Extension Research Liaison Services NASS National Assembly NBS National Bureau of Statistics NGO Non-governmental Organization FY Fiscal Year SOD Statement of Differences SIR Sub-intermediate Result SR Strategic Results SMART Specific, Measurable, Achievable and Attributable, Realistic and Time￾bound/Timely STATA Statistical Analysis Software USAID United States Agency for International Development USG United States Government UST University of Science and Technology UT University of Technology WEF World Economic Forum WTO World Trade Organization YISA Youth Initiative for Sustainable Agriculture viii Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 0.0 EXECUTIVE SUMMARY Background From November 2018 through March 2019, the Monitoring, Learning and Evaluation (MEL) Activity conducted a mid-term performance evaluation of the Feed the Future (FTF) Nigeria Agricultural Policy Project, hereinafter referred to as the “Agricultural Policy Activity” or the “Activity,” covering the Activity’s performance from 2015 through September 2018. The goal of the Agricultural Policy Activity is to; a) Increase national capacity to generate and analyze information, and to formulate evidence-based policy options for considerations by decision makers and stakeholders; and, b) Improve policy processes driven by empirical evidence and active dialogue among relevant stakeholders. The evaluation team (the “Team”) conducted key informant interviews (KIIs) in the seven FTF focus states of Benue, Cross River, Delta, Ebonyi, Kaduna, Kebbi, and Niger, as well as in the Federal Capital Territory (FCT) and with researchers in Ibadan, Oyo State. Methodology The evaluation methodology involved KIIs comprising post-graduate students participating in the Activity’s Visiting Scholars’ program at Michigan State University (MSU), Nigerian visiting professors, members of the National Advisory Committee, media, government officials, academic researchers, private sector, and civil society organizations (CSOs) who have participated in the Agricultural Policy Activity. The team also contacted 282 out of 309 sampled participants that benefited from the Activity’s capacity building trainings (particularly on the statistical program STATA), CSO representatives, and media representatives, either face-to-face or by telephone. Findings, Conclusions and Recommendations Overall, the greatest influence of the Activity has been exerted in reaching out broadly to all stakeholders in the agricultural policy space and, in some respects, unifying the agricultural research community. The most successful in-country activities identified by the evaluation team were: ▪ The Activity’s work with the Cross-River State government in 2018 to review the 2014 State￾level Comprehensive Agricultural Policy. Embarking on and completing the review process is a major achievement which should be identified among the Activity’s success stories. ▪ Analyses of fish and rice subsectors in Kebbi State, rice in Niger State, and rice in Ebonyi State. The policy analyses focusing on such key commodities and resulting in the production, printing, and distribution of a commodity-specific policy brief are important Activity successes. ▪ To encourage buy-in, the Activity took advantage of a convened meeting of the directors for budget and planning, Ministries of Agriculture, and the monitoring and evaluation (M&E) units from 33 of Nigeria’s 36 states and the FCT in order to introduce the Activity’s scope and objectives to the participating states. ▪ The capacity building activities of the Activity included 23.4 percent youth and 30 percent women, which signifies a conscious effort for gender and youth inclusion. ix Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Less-harmonious outcomes include: ▪ Several of the stakeholders reported being improperly informed about training content, duration, and follow-ons. ▪ Participants were critical of the short notice with respect to capacity building events or meetings expected with key stakeholders. ▪ Actual influence on policy-making was limited. Overall, the Activity has generally met, been close to meeting, or in several cases exceeded its performance indicators, producing high-quality research based on collaboration with Nigerian stakeholders and including Nigerian universities and many students. All seven FTF states have benefited, as well as key stakeholders and beneficiaries, particularly in Abuja, the FCT where the Government of Nigeria (GON) is located, and in the university town of Ibadan in Oyo State. The method of delivery, particularly the capacity building activities, has contributed to ensuring that the Activity is on track to achieving its annual and life-of-project targets. However, some questions arose regarding the usefulness of the indicators, which are discussed under EQ1 (Section 5.1) of the main report. EQ1 (a) and (b) – Have the methods of delivery contributed to ensuring the Activity is on track to achieving its annual and life-of-project targets? Are the intended results per the Contract agreement occurring? The methods of delivery, especially capacity building activities, have contributed to ensuring that the Activity is on track to achieving its annual and life-of-project targets. Conclusions for EQ 1 (a) and (b). There are areas of concern about the overall effectiveness and benefits of the Activity monitoring including: • Negative deviation of some performance indicators in year three compared with years one and two of the Activity; and, • Lack of data to track some indicators, especially in the first and second years of the Activity. Recommendation for EQ 1 (a) and (b) Remove redundant indicators, such as; “Number of formal, informal, or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed.” EQ1 (c) – Is the Activity working on any policy issues? If yes, how critical and necessary are the policy issues chosen by the Activity in transforming the agriculture sector in Nigeria? The Activity has undertaken research and analysis on sensitive and delicate topics. These include the effect of wide-spread smuggling of rice and poultry (permanent bans in violation of their commitments under the Economic Community of Western African States (ECOWAS) and the World Trade Organization (WTO)) and on the impact of climate change on Nigeria’s poultry sector. x Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Conclusions for EQ 1(c) The design of capacity building activities in some cases failed to recognize the existing policy research needs of the various sub-sectors of agriculture in Nigeria and did not sufficiently emphasize dissemination of the research reports and policy analyses, limiting their influence. Recommendation for EQ The Activity should propose a revised work plan leading up to the termination date, and work on fixing dates and agendas with the partner institutions well in advance. EQ1 (d) – How appropriate is the Activity’s capacity building of policy actors and other stakeholders? To what extent is the Activity engaged in capacity building taking into consideration the audience, methods, and effects of the capacity building? The results show that the training programs are appropriate. A majority, 79 percent, of those who participated in communication workshops are stakeholders from the media, while 83 percent of those participating in the policy workshops are from the think tank/NGO category. The workshops on the use of software are largely, 80 percent, attended by academia and the private sector, 73 percent. Conclusions for EQ 1(d) The Activity has over-emphasized academic-style research that may be seen as advancing the careers of the IFPRI and MSU researchers, rather than on bringing Nigerian collaborators to the forefront. Recommendation for EQ 1(d) The Activity should carry out careful selection of eligible participants, in collaboration with knowledgeable local stakeholders, with the aim of encouraging broad participation by women’s groups, youth, and socially disadvantaged persons. EQ1 (e) – How does the Activity intend the capacity building to influence policy via policy actors and other stakeholders? The way the Activity expects capacity building to influence policy varies according to the type of training and its participants. For instance, the media is expected to apply the knowledge gained to improving their understanding of agricultural policy issues and doing a better job of communicating those issues. Conclusions for EQ 1(e) The Activity has not sufficiently emphasized dissemination of the research reports and policy analysis, limiting its influence. Recommendations for EQ 1 (e) To ensure that the Activity exerts its intended influence, research reports and policy analyses will be disseminated more widely among stake holders. EQ1 (f) – On what evidence are the policies based? Policy gaps or areas of emphasis in the implementation of the Activity components are based on experience from previous research, interactions with the literature, and areas of interest of Activity participants. xi Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Conclusion for EQ 1(f) The links between the Activity and policy authorities are weak, especially at the federal level, in establishing which policy priorities are to be entrenched in the Activity’s work plan. Recommendation for EQ 1(f) Roles and responsibilities of partners and coordinators should be well-defined, with enough time for collaborative planning, and these roles should be made known to all stakeholders. EQ1 (g) – What should the Activity put in place to ensure that the end-line intended impact of the Activity is on track and achieved? With the complexity of the analytic methods taught during the capacity-building sessions, respondents suggested that the Activity should allow a longer duration for training programs, i.e., four days instead of two days. The Activity interacted with stakeholders and partners on an informal basis, with no memorandum of understanding (MOU) with any groups other than U.S.-based implementing partner, MSU. Conclusion for EQ 1(g) Mapping of the audience, training curriculum, and expected outcomes was not adequately considered in the design of the Activity. Additionally, directors undergoing training in STATA for three to four days may not achieve the desired result, due to distractions from their everyday tasks. Recommendations for EQ 1(g) • Potential beneficiaries of the capacity building activities should have a proper understanding of existing policies, including implementation strategies and expected results. • The training requirements of bureaucrats must be distinguished from those of policy analysts. EQ2 (a) and (b) – How well domesticated is the Agricultural Policy Activity in the states of Nigeria? To what extent are the state-level policy actors aware of national/federal initiatives and aligning state policies accordingly. State-level policy actors, including analysts, reported a low level of awareness of federal-level agricultural policies, limiting alignment of state and federal policies. With only a few exceptions, the research agenda and approach were determined by the Activity, rather than mutually with local research groups, ministries, and civil society. Conclusion for EQ 2(a) and (b) The design of the capacity building activities in some cases failed to recognize the existing policy research needs of the various sub-sectors of agriculture in Nigeria. Recommendations for EQ 2(a) and (b) A detailed proposed work plan should be shared with identified stakeholders and their input solicited. xii Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ2 (c) – To what extent has the Activity engaged multiple stakeholder groups (value-chain actors, farmers, private sector, policy actors, think tanks, CSOs, etc.) to achieve its intended outcomes? A predictable calendar of activities has not been shared in advance with local partners. Over half of the key stakeholders feel that neither scheduling nor process has been clear in the course of Activity implementation. Conclusion for EQ 2(c) The Activity would have had greater “local buy-in” if its initiatives had been oriented to the local research agenda, rather than priorities determined in Washington, DC or East Lansing, MI. Recommendations for EQ 2 (c) The Activity must include federal-level policy initiatives in its training programs to promote awareness and understanding among policy actors at the sub-national level and among the research community. EQ3 (a) – What are the most critical constraints in the formulation and implementation of agricultural policies in Nigeria, and how is the Activity responding to these constraints? The most critical constraints, both at the federal and state levels, include the low capacity of government officials and inadequate funding arising from the low level of government revenue, delays in budget approvals, and delays in the release of approved funds. Conclusion for EQ 3(a) The challenge of limited funding from ministries, departments, and agencies worsens the weak absorptive and executive capacity critical to the formulation and implementation of agricultural policies, both at the federal and sub-national levels. Recommendations for EQ 3(a) The Team recommends timely approval of budgets, with prompt release of appropriated funds and rigorous efforts to generate internal revenue. The Activity can consider making funds available to execute collaborative policy works with the focus government. EQ3 (b) – How responsive has the Activity been to the changing needs of the policy environment and seizing opportunities for impact? The Activity embarked on some policy dialogues at the height of the conflict between herders and farmers with a view to understanding the causes and drawing up lessons to minimize or prevent the occurrence of the conflict in some other states. Conclusion for EQ 3(b) The Activity has been responsive to the changing needs of the policy environment by the type of policy issues it has attended to in the focal states Recommendations for EQ 3 (b) The Activity should continue its partnership with local ministries, departments, and agencies, as exemplified by its promotion of policy dialogues in Niger and Kebbi states at the height of conflict between herders and farmers. This type of initiative is not easily discernible at the federal level. xiii Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ3 (c) – To what extent has the Activity fostered policy dialogue with and/or among USAID/Nigeria’s FTF activities and other donor programs? Collaboration of the Activity with other ongoing USAID-sponsored FTF activities is at a low level. For example, at the state level, Activity partners expressed a lack of awareness of USAID-sponsored projects and other ongoing Nigeria FTF activities. Conclusion for EQ 3(c) Coordination with other USAID/Nigeria FTF activities and other donor programs to attract private-sector investment and increase productivity and growth in the sector is weak Recommendations for EQ 3 (c) The Activity should coordinate with USAID to identify existing and new FTF activities in the states where the Activity is being implemented, and design appropriate mechanisms for collaboration in each component of the Activity. EQ4 (a) – What are the benefits of the Activity’s research to policy actors and stakeholders? Is it reaching them, if yes, how? If not, why not? Policy makers and other stakeholders, based on the KIIs, reported very low usage of the Activity’s research. Part of the reason for this is the Activity’s emphasis on research rather than policy analysis. Conclusion for EQ 4(a) The policy research activities are not clearly demand-driven. Recommendations for EQ 4 (a) There should be a transparent process of identifying research topics which should be sufficiently inclusive of key policy actors and stakeholders. EQ4 (b) – How do policy actors and stakeholders use the Activity’s research? How do policy actors and stakeholders assess the utility of the Activity’s research? • At the federal level, policy makers and stakeholders informed the Team that they did not make much use of the Activity’s research due to a disconnect between the Activity’s research priorities and the work plans and policy priorities of FMARD. • At the state level, the Activity has assisted the state agriculture ministries in Cross River State with a sector-wide plan. In Kebbi State, the Activity supported the development of commodity￾specific plans for rice and fish. Conclusion for EQ 4(b) • The policy research activities are not clearly demand-driven. • One of the Activity’s greatest successes has been assisting the agriculture ministries of Cross River and Kebbi in applied policy analysis. Recommendations for EQ 4 (b) There should be a transparent process of identifying research topics that is sufficiently inclusive of key policy actors and stakeholders. xiv Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ4 (c) – How knowledgeable are policy actors and stakeholders about the Activity’s different types of research products? Not a single Nigerian stakeholder mentioned accessing the Activity website. The extent to which stakeholders have access to available online research products depends on the availability of the requisite infrastructure in the partner institutions or among individual stakeholders. Conclusion for EQ 4(c) Researchers in several research institutes in Nigeria are not prioritized as target groups. A mapping of potential research institutions that could participate in the Activity would yield over 50 institutions in Nigeria, and yet the research reports and policy briefs have been carried out only by two institutions: IFPRI and MSU. Recommendations for EQ 4 (c) The issue of weak demand for research products must be addressed through strengthening research infrastructure among local partners and stakeholders. EQ4 (d) – How is the Activity’s research influencing and/or informing policy actors, private sector and other value￾chain actors? The academic nature of the research is not well-suited to policy analysis at either the federal or state levels which require quicker responses to pressing questions. Again, some research studies are based on the interests of researchers, rather than being based on a consensus of policy actors, as evidenced by the experience in the National Agriculture and Extension Research Liaison Services (NAERLS). Conclusion for EQ 4(c) The prioritization of research topics is restrictive; this has tended to make the value of the research less evident than would have been the case if research priorities had been determined by Nigerian policy actors and key stakeholders Recommendations for EQ 4(c) There should be a transparent process of identifying research topics that are sufficiently inclusive of key policy actors and stakeholders. EQ4 (e) – What are the dissemination plans for the different types of research products that the Activity produces? One high-level FMARD official noted that he had a box full of reports by the Activity (and the National Assembly (NASS)) that he did not have sufficient time to read. Several of the MSU Scholars reported that the MSU professors had provided them with numerous pertinent studies and materials in both paper copies and electronic files. Conclusion for EQ 4(e) The most effective approach by the Activity would be in-person dissemination of research products to government officials, while making paper copies and electronic files available to the MSU scholars. Recommendations for EQ 4(e) The Activity’s dissemination should be driven by partners’ needs and what works best for the partners. xv Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ (5) What measures is the activity putting in place to ensure sustainability of the interventions? The leadership of research teams, restricted as it is to IFPRI and MSU researchers, limits sustainability. Conclusion for EQ 5 Sustainability measures are weak and unlikely to ensure regular conduct of policy research to enhance growth in the sector. The lack of local implementing partners, such as the APRNet or YISA, limits the transfer of functions performed by the Activity. Recommendations for EQ 5 Nigerian policy researchers should be included in the conduct of policy research under the Activity. This should be achieved through an entrenched and systematic process of calling for research proposals from Nigerian researchers in already identified thematic policy areas. The main body of this report contains additional detail on the Team’s responses to each EQ and a more in-depth discussion of the findings, conclusions, and recommendations. Sustainable actions and a sustainability plan to bring together Nigeria’s agricultural research community are recommended for the Activity as it completes the remainder of Year Four and prepares for Year Five. The dissemination and explanation of the research already completed, and continued training of Nigerian actors in carrying out research and producing papers and policy briefs is advisable. Statement of Differences from IPs It is USAID policy that any USAID evaluation report can include a Statement of Differences (SOD) from the implementing partners, funders, or evaluation team members as an annex. The SOD presents any significant unresolved differences of opinion regarding the findings, conclusions, and recommendations in the evaluation report. The IPs of the Activity - Michigan State University (MSU) and the International Food Policy Research Institute (IFPRI), have submitted a SOD that is attached to this report as Annex 9. 1 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 1.0 EVALUATION OVERVIEW This report presents findings, conclusions, and recommendations to support the mid-term performance evaluation of the Feed the Future (FTF) Nigeria Agricultural Policy Project, hereinafter referred to as the “Agricultural Policy Activity” or the “Activity,” a five-year United States Agency for International Development (USAID)/Nigeria-funded capacity building, policy research, and dissemination activity. Michigan State University (MSU), in partnership with the International Food Policy Research Institute (IFPRI), began implementing the Activity on July 1, 2015 and will close on June 30, 2020. The goal of the Activity is to: i) increase the national capacity to generate and analyze information and formulate evidence￾based policy options for consideration by decision-makers and stakeholders; and, ii) improve policy processes driven by empirical evidence and active dialogue among decision-makers and stakeholders. This two-part goal will be achieved through: a) strengthening the regional and national capacity for greater evidence-based agriculture policy processes through collaborative research, with due recognition of gender dimensions, to inform and improve the agricultural policy debate and formulation and, ultimately, its impact; b) strengthening the interaction between the various stakeholders in the agricultural policy process to better promote and foster informed policy dialogue among them, in order to promote a “think tank” culture; and, c) assisting the federal and state governments to improve their capacity to plan and implement effective policy analyses and programs, and demand and apply policy research in their policy processes. Activities (see Table 1 below) vary based on the different training locations with which the Agricultural Policy Activity engages among the state governments. The purpose of this mid-term performance evaluation is to determine whether the agricultural policy intervention funded by USAID/Nigeria is meeting the stated development objectives and to understand the lessons learned through implementation. The evaluation provides an independent examination of the overall progress and accomplishments of the Activity. 2 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Table 1: Agricultural Policy Project’s Planned Interventions STATE TRAINING LOCATIONS ENGAGED WITH STATE GOVERNMENTS INTERACTIONS WITH STATE GOVERNMENTS INITIATED STATE LEVEL RESEARCH Akwa Ibom Bauchi Benue X Cross River X X X Delta X X Ebonyi X FCT X Jigawa Kaduna Zaria X X Kano X Katsina Kebbi X Kogi Kwara Nassarawa Niger X X Ogun Abeokuta X Ondo Akure X X Osun Oyo Ibadan and Igbo￾Ora X X Plateau Rivers 3 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 2.0 BACKGROUND With an abundant and diversified agricultural base, Nigeria’s agricultural sector generates about 22 percent of gross domestic product (GDP) and employs about 70 percent of the labor force. Until the early 1970s, Nigeria was self-sufficient in food production, with a small surplus for export, and agriculture was the main foreign exchange earner. Post-1970, however, the sector stagnated for a number of reasons—chief among them the discovery, exploitation, and export of oil, as well as a deliberate policy to shift resources from agriculture to industry (Oyejide:1986). Because agriculture employs an overwhelming majority of Nigerians, stagnation of that sector resulted in increased poverty. The poverty rate (measured at US $1.25/day per capita consumption) increased from 28 percent in 1980 to 68 percent in 2012. Since 2015, however, the performance of the agricultural sector is believed to have improved and it now contributes 25 percent to the country’s GDP (National Bureau of Statistics [NBS], 2017; Central Bank of Nigeria, 2018). Nigeria is, by sub-Saharan Africa standards, relatively developed but it is not as far along as other parts of the developing world, especially the fast-growing countries of South Asia. Nigeria, the largest country by population and size of the economy in Africa, has abundant and rich agricultural resources and human capital, and a diversified and rich natural resource base. Until recently, economic performance had been slow and driven primarily by the oil sector. However, since 2005, the agricultural sector has been growing rapidly, with value-added averaging about seven percent annually, nevertheless, the above 2017 figure of 25 percent to the GDP of Nigeria suggests that productivity and incomes in the sector are low (NBS: 2018). In spite of recent improvements in performance in the agricultural sector, labor productivity is low and slow-growing. Unemployment and poverty in rural areas, especially among youth, tends to be high. According to NBS, poverty incidence (using US $1.25/day per capita as the measure) in 2011 was 52 percent in urban areas and 66.1 percent in rural areas. This may be due partly to high unemployment rates combined with low productivity in the agricultural sector. Although sectoral poverty data are not available, rural unemployment rates are higher (37.2 percent) than urban unemployment rates (33.5 percent). Recognizing the need to revitalize and transform the agriculture sector to increase rural incomes and grow the country’s economy, in 2011 the Federal Government of Nigeria (GON) embarked on a visionary strategy for the sector by launching the Agricultural Transformation Agenda (ATA). The goal of the ATA was to increase rural incomes and grow the economy by providing an enabling environment for the private sector to lead the agricultural transformation process. More recently, in 2013 Nigeria joined the New Alliance for Food Security and Nutrition led by the G-8 and the Grow Africa initiative led jointly by the African Union, the New Partnership for African Development, and the World Economic Forum to share knowledge about its own policy and investment commitments through the ATA and promote greater private sector investments. The Agricultural Policy Activity is intended to ensure that all partners meet the growing expectations of the role that national and international research can play in supporting the capacity, knowledge, and information needs of the Federal Ministry of Agriculture and Rural Development (FMARD). 4 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 3.0 ACTIVITY OVERVIEW The goal of the Agricultural Policy Activity is to; a) increase national capacity to generate and analyze information and to formulate evidence-based policy options for considerations by decision-makers and stakeholders; and, b) improve policy processes driven by empirical evidence and active dialogue among relevant stakeholders. The Activity has three principal objectives: 1. To strengthen national capacity for greater evidence-based policy processes in agriculture by increasing the capacity of Nigerian analysts to undertake and make widely available relevant evidence-based policy analysis; 2. To promote and foster informed policy dialogue among all stakeholders in the agricultural sector through an inclusive, transparent, and sustainable process at the country level, building blocks for a well-integrated and developed national policy system; and 3. To help federal and state governments improve their capacities to plan and implement effective policy analyses and programs, and demand and absorb policy research in their policy process. 3.1 Results Framework To achieve this goal and the principal objectives, the overall results framework (Figure 1) for the FTF Innovation Lab for Food Security Policy (FSP) project and the Country Development Cooperation Strategy (CDCS) provide the model for the results framework for the Activity. The FSP results framework is itself consistent with that of FTF, the United States Government’s (USG) global hunger and food security initiative. The results framework adapted for this Activity shows the pathways by which the Activity will achieve its goals of promoting more inclusive, private-sector-led agricultural growth (which is the FTF strategy’s First Level Objective) and reducing poverty (which is the goal of the CDCS). This framework is central to the management and monitoring and evaluation (M&E) of this Activity and to this mid-term performance evaluation. Within the USAID CDCS, the FTF Agricultural Policy Project is located under Development Objective (DO) 1, Broadened and inclusive growth, Intermediate Result (IR) 1.2: Business Environment Improved, Sub-IR 1.2.1: Legal and Regulatory Environment Improved (Figure 1). Figure 1: USAID/Nigeria Economic Growth Results Framework (2015–2019 CDCS) 5 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation The overall results framework for the FTF Innovation Lab for FSP and the USAID Nigeria’s CDCS provide the model for the results framework for the Agricultural Policy Activity. The FSP results framework is itself consistent with that of FTF, the USG’s global hunger and food security initiative. The results framework adapted for the Activity is illustrated in Figure 2. It shows the pathways by which the Activity will achieve its goal of promoting more inclusive private-sector led agricultural growth (which is FTF strategy’s First Level Objective) and poverty reduction (which is the goal of CDCS). Figure 2: FTF Nigeria Agricultural Policy Project Results Framework 6 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation The Agricultural Policy Activity’s results framework also includes one custom sub-sub IR and four strategic results (SR) across the activity components that will contribute to the sub-IRs. These tiers are arranged to illustrate the causal relationship between planned efforts under the Activity and their expected effects, and to identify the intermediate results that are critical to achieving activity objectives. Specifically, the framework demonstrates how planned activities and deliverables will lead to expected outputs, outcomes, results, and eventual impact. Together, the DOs, IRs, sub-IRs, and SRs identified in Figure 2 above provide the framework for identification and implementation of activities designed to achieve the required results. 3.2 Agricultural Policy Activity’s Strategy The Activity uses a robust approach to enhance skills, training, and institutional capacities to meet the demands for policy analysis by the FMARD and the national food security and agricultural policy processes. In order to achieve this, the Activity engages in training collaborations with FMARD, academic institutions, and other key stakeholders. It strives to strengthen the links between various actors and to foster the development of a network of institutions that can independently interact during the policy process. The policy-driven collaborative research and analysis component directly supports the knowledge needs of the policy process and is sub-divided into two sub-components: research and policy analysis. The research component involves research teams comprising Activity and Nigerian researchers with priority topics identified in consultation with FMARD, the states’ Ministries of Agriculture, development partners, and other key stakeholders. These researchers and their networks provide their expertise to inform policy debates with empirical evidence, as they increase interaction between the supply of information (Nigerian researchers) and the demand for information (FMARD and other stakeholders). The policy analysis component, on the other hand, is more focused on demand-driven and shorter-term policy analyses that originate from requests by FMARD, the states’ Ministries of Agriculture, and their development partners. Strengthening the evidence‐based policy process and promoting impact will ensure that the Activity addresses policy gaps through increased and targeted policy communications and capacity building at universities and FMARD. Strategies to strengthen the evidence-based policy process include: directly contributing to the donor-government policy dialogue on agricultural policy; launching a policy brief series for the Activity; promoting a “think tank” culture within the agricultural policy process by organizing seminars and events targeted at all agricultural policy actors; providing training courses for FMARD and scholars; and providing for selected Nigerian graduate students to take advanced courses at MSU through the Scholars Exchange. 3.3 Main Activities The Agricultural Policy Activity planned to accomplish the SRs, IRs, and activity objectives through a work program organized around three themes—capacity building, research, and outreach. Examples of activities funded by the Activity include the following. 1. Support skill-enhancing training of staff from Nigerian partner institutions on methods and applications for research and policy analysis. The content of this training was developed in consultation with FMARD, Ahmadu Bello University (ABU), the University of Ibadan, and other key stakeholders to ensure their appropriateness in addressing the most serious gaps in knowledge, analytical and research approaches, and application tools in Nigeria. 7 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 2. Devise and institute a direct-training collaboration between researchers at MSU and IFPRI with graduate student researchers and their supervisors in Nigerian universities. This is to include a competitive annual selection of two or three Nigerian post-graduate students (one at the Master’s level and one or two at the Ph.D. level). These students and their research advisors form a research team with MSU and IFPRI researchers on the proposed research topics. Where possible (and to be encouraged), the research project will constitute part of the students’ thesis or dissertation. The Nigerian students attend technical courses (e.g., econometrics, research design, and/or modelling techniques) at MSU for one semester for Master’s-level students and one year for Ph.D. students. Periodic team meetings are scheduled to discuss team findings and progress and facilitate more learning by junior researchers. 3. Nigerian research supervisors visit the U.S. to participate in various activities, broaden their outlook, and facilitate the provision of feedback for their research students. The visits also facilitate interaction of the Nigerian professors with faculty at the department of Agricultural, Food and Resource Economics, as well as faculty at the College of Agriculture and Natural Resources, to broaden their outlook as well. The Nigerian professors collaborate with senior IFPRI researchers based in Washington, DC. 4. Based on demand, periodic training courses at Nigerian universities on various research methods and tools are organized to reach a broader set of promising graduate students and young research professionals. Lecturers and students in Nigerian universities have access to all course materials developed. Additionally, as part of this periodic training, lectures by visiting MSU and IFPRI staff are encouraged at the Activity’s collaborating institutions. This is a key long-term effort to facilitate the training of the next generation of Nigerian policy researchers and analysts. The training courses, which cover policy analysis methods and tools, target a broad audience including the research community, practitioners in government (such as those at FMARD and other ministries related to agriculture), CSOs, non-governmental organizations (NGOs), the private sector, and think tanks. 5. Major conferences are held to promote the visibility of the policy research and analysis being undertaken, including Activity workshops and other activities. MSU and IFPRI have proposed organizing three major conferences during the tenure of the Activity (2015–2020), which will bring together experts on selected issues in agriculture and economic development to provide guidance to Nigerian graduate students and young research professionals working in these areas. In addition, the Activity funds select graduate students to attend a national/regional conference and present their work. Such conference participation will broaden the reach of these conferences and increase interaction between actors at various institutions interested in agricultural policy. The interactions are intended to strengthen the links between such actors and foster the development of a network of institutions that can independently interact during the policy process. 6. Formal ties are to be established with two or more Nigerian institutions in order to develop appropriate skills, knowledge, and tools to strengthen the credibility and relevance of evidence generated by policy analysis and research for informing policy. In addition to two already identified institutions (ABU and University of Ibadan), additional institutions are to be selected in line with FTF focus states and their interest and/or relevance to Activity research topics; the purpose is to broaden the Activity’s interaction with institutions across Nigeria’s diverse agroecological and/or geopolitical systems. 8 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 3.4 Critical Challenges/Assumptions There were several critical assumptions made in order for the Agricultural Policy Activity to achieve its targets and objectives. The first assumption is that the policy-making environment in Nigeria remains relatively stable and is not subjected to major upheavals, often associated with shifts in political power resulting from undemocratic forces. The second is that the government, policy makers, and stakeholders targeted by the Activity are receptive to new ideas, responsive to presented evidence, and committed to implementing and enforcing the outcomes of policy research and analysis. In other words, the results framework relies on there being political will and support for the policy change agenda and that the Activity receives the cooperation and participation of targeted stakeholders in program activities and its outputs. These factors pose challenges; Nigeria is new to broad policy reform as, until recently, much of Nigeria’s policies were concerned with changing the urban landscape, the fortunes of urban dwellers.1 The Agricultural Policy Activity will, thus, provide timely evidence to improve the local capacity to conduct evidence-based policy analysis and advocacy to overcome these challenges. 3.5 Performance Indicators The Agricultural Policy Activity’s performance indicators for monitoring and reporting purposes are listed in Table 2 below. These 12 indicators were identified by reviewing the performance indicators developed for FTF as used in the Feed the Future Monitoring System (FTFMS), and those developed for the global food security policy. Most of these indicators are custom indicators; only indicators five (5) and eight (8) are standard indicators from the FTF Indicator Handbook.2 The indicators track the delivery of outputs and outcomes related to the Activity’s efforts to more broadly strengthen evidence-based policy processes for agriculture and food security in Nigeria. 1 Ering, Simon; Odey, Judith; Eteng, Out; and Esther Patrick Archibong. (2014). Rural Development Policies in Nigeria: A Critical Appraisal. International Journal of Education and Research. 2:9, Sept. 2014. 2 Feed the Future. (2014). Feed the Future Indicator Handbook – Definition Sheets. U.S. Government working document, October 2014 version. Washington, DC: US Government. Feed the Future Innovation Lab for Food Security Policy. (2014). Performance Monitoring Plan. Approved version of 17 November 2014. East Lansing, MI: Michigan State University. 9 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Table 2: Agricultural Policy Activity Performance Indicators S/N INDICATOR DEFINITION LEVEL ID IN FTFMS TYPE CATEGORY UNIT OF MEASURE DISAGG. BY DATA SOURCE FREQ. 1 Number of policy research and best practice papers generated SR Custom Output New Information, knowledge, and practices Papers Policy area Activity records Annual 2 Number of stakeholder learning forums (national or global) held where findings/best practices are presented SR Custom Output Knowledge dissemination and learning events Stakeholder forums Policy area Activity records Annual 3 Number of participants attending Activity organized events SR Custom Output Knowledge dissemination and learning events People participating Type of organization, gender Activity records Annual 4 Number of formal, informal or ad hoc meetings held with key government or private sector partners, where project outputs and findings are discussed SR Custom Output Knowledge dissemination and learning events Meetings Policy area Personal logs Annual 5 Number of agricultural and nutritional enabling environment policies completing the following processes/steps of development as a result of USG assistance in each case: SR 4.5.1(24) Output Policies, programs, and regulations reviewed and analyzed Policy (Only for 1. Analysis) Policy area Activity records Annual ▪ Analysis ▪ Stakeholder consultation/ public debate ▪ Drafting or revisions ▪ Approval legislative or regulatory ▪ Full and effective implementation ▪ (S) 6 Number of collaborative research teams involving local partners formed and undertaking policy research and analysis SR Custom Output Policies, programs, and regulations reviewed and analyzed Research teams Policy area Activity records Annual 7 Number of technical training courses offered to build technical skills and capacity for policy analysis SR Custom Output Human resource and organizational capacity building Training courses Policy area Activity records Annual 10 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation S/N INDICATOR DEFINITION LEVEL ID IN FTFMS TYPE CATEGORY UNIT OF MEASURE DISAGG. BY DATA SOURCE FREQ. 8 Number of individuals who have received USG supported short-term agricultural sector productivity or food security training (RiA) (WOG) SR 4.5.2(7) Output Human resource and organizational capacity building People trained Gender and type of individual Activity records Annual 9 Number of individuals who have received USG supported short-term agricultural sector productivity or food security training SR Custom Output Human resource and organizational capacity building People trained Gender and type of individual Activity records Annual 10 Number of organizations/entities assisted to participate in various roundtable and stakeholder meetings, seminars and conferences focused on agricultural policy issues Sub-IR Custom Output Institutional capacity building Organizations Type of organization, policy area Activity records Annual 11 Index (or scorecard) of quality of agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation to capture level of satisfaction and confidence Sub-IR Custom Outcom e Better policy formulation process Stakeholder perception and level of satisfaction and confidence None Stakeholder survey Baseline, midterm and end-line 12 Index (or scorecard) of quality of the institutional architecture for agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation survey to capture level of satisfaction and confidence Sub-IR Custom Outcom e Better policy environment Stakeholder perceptions and level of satisfaction and confidence Country Stakeholder survey Baseline, midterm and end-line 11 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 4.0 EVALUATION PURPOSE AND METHODOLOGY The primary purpose of this mid-term performance evaluation is to determine whether the assistance provided by USAID/Nigeria through the Agricultural Policy Activity is meeting the stated development objectives and to identify the lessons learned. The analysis below provides a detailed and independent picture of the major accomplishments and weaknesses of the Activity since its inception and determines how its successes can be sustained by providing responses to the evaluation questions. This evaluation provides USAID/Nigeria, its implementing partners, and agriculture sector stakeholders with data on outcomes and progress toward achieving Activity objectives. It also answers fully all of the evaluation questions and provides specific recommendations to USAID/Nigeria about how the Activity interventions can be adjusted during the remaining two years of its duration, and how achievements may be sustained and scaled by other FTF activities, partners, private sector firms, or by the GON. The evaluation also highlights the lessons that USAID should learn in terms of contract agreement and management from this Activity. 4.1 Evaluation Questions The following evaluation questions were posed by USAID/Nigeria: 1. Is the Agricultural Policy Activity implemented as intended according to the Contract agreement? a. Have the methods of delivery contributed to ensuring the Activity is on track to achieving its annual and life of project targets? b. Are the intended results per the Contract agreement occurring? c. Is the Activity working on any policy issues? If yes, how critical and necessary are the policy issues chosen by the Activity in transforming the agriculture sector in Nigeria? d. How appropriate is the Activity’s capacity building of policy actors and other stakeholders? To what extent is the Activity engaged in capacity building taking into consideration the audience, methods, and effects of the capacity building? e. How does the Activity intend the capacity building to influence policy via policy actors and other stakeholders? 2. On what evidence are the policies based? a. What should the Activity put in place to ensure that the end-line intended impact of the Activity is on track and achieved? b. Are key stakeholders (GON, think tanks, CSOs, community-based organizations, associations and media) satisfied with the capacity building activities implemented by the Activity? Why or why not? c. How well domesticated is the Agricultural Policy Activity in the states of Nigeria? d. To what extent are the state-level policy actors aware of national/federal initiatives and aligning state policies accordingly? e. To what extent has the Activity engaged multiple stakeholder groups (value-chain actors, farmers, private sector, policy actors, think tanks, CSOs, etc.) to achieve its intended outcomes? 12 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 3. To what extent has the Activity coordinated with other USAID/Nigeria FTF activities and other donor programs to achieve the overall strategic objectives of improving an enabling environment in the agricultural sector for private sector investment? a. What are the most critical constraints in the formulation and implementation of agricultural policies in Nigeria, and how is the Activity responding to these constraints? b. How responsive has the Activity been to the changing needs of the policy environment and seizing opportunities for impact? c. To what extent has the Activity fostered policy dialogue with and/or among USAID/Nigeria’s FTF activities and other donor programs? 4. What methodologies does the policy research component of the Activity employ and what is the value of the research, conducted by the Activity, to the stakeholders? a. What are the benefits of the Activity’s research to policy actors and stakeholders? Are the benefits reaching them and if yes, how? If not, why not? b. How do policy actors and stakeholders use the Activity’s research? How do policy actors and stakeholders assess the utility of the Activity’s research? c. How knowledgeable are policy actors and stakeholders about the Activity’s different types of research products? d. How is the Activity’s research influencing and/or informing policy actors, private sector, and other value-chain actors? e. What are the plans for dissemination of the different types of research products that the Activity produces? 5. What measures is the Activity putting in place to ensure sustainability of the interventions? 4.2 Evaluation Methodology Given the diverse range of stakeholders benefiting from the Agricultural Policy Activity’s support on agricultural policy development and capacity building, the evaluation team (the “Team”) used a mixed methods approach to address the evaluation questions. The methodology included engaging 309 respondents out of more than 1,500 of the Activity’s capacity building participants. Focusing on the FTF focus states, a near-comprehensive census was undertaken. On the qualitative path, primary data was elicited via key informant interviews (KIIs), document review, and Team knowledge of actual conditions related to agricultural research and applied implementation in Nigeria. A content analysis of the primary data was then performed, identifying similarities and differences in the data, as well as recurrent themes. The KIIs were conducted either in person, by telephone, or via email. Figure 3: Three-stage Evaluation Methodology Reviewed Project documents and developed draft data collection protocols Conducted interviews with Project implementation team and stakeholders/beneficiaries— in person (personal and roundtable) and by telephone Analysis of data collected and development of report 13 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 4.2.1 Methodology for the Sample Selection The Team used a mixed-methods approach for determining the sample frame. Lists of participants were first stratified by level of participation. • Key stakeholders: All five categories of key stakeholders were contacted (47) • Beneficiaries of capacity building: Three-hundred and nine (309) beneficiaries of the capacity building activities were planned to be contacted out of over 1,500 beneficiaries who participated in the Activity’s capacity building efforts. 4.2.2 Sample Selection and Composition of Partners for Key Informant Interviews The database was stratified by region (South/North), by gender (Female/Male) and based on type of Activity undertaken (software training, policy dialogue, training in how to communicate policy and statistical analysis, monitoring and evaluation training, and youth empowerment). This led to 20 strata from which the respondent list was drawn. Capacity building participants were randomly selected from the identified strata based on the breakdown in Table 3 below. Table 3: Population Mix of Capacity Building Participants The sample size (using a 95 percent confidence interval and a 5 percent margin of error) was 309 out of 1,561 trainees/beneficiaries of Activity’s capacity building activities. In addition to these 309 respondents, the following stakeholders were included in the respondent pool: • 26 key stakeholders out of 48, which includes academia/research institutes, GON at all levels (including the federal legislature), CSOs, media, and private sector (agro-input dealers and agro processors) (Table 4) • Six (current and completed) out of 13 graduate students supported on exchange at MSU • Six out of 13 Nigerian visiting professors accompanying sampled students to MSU • All six National Advisory Committee members A review and assessment of monitoring data corresponding to the Activity’s performance indicators year￾by-year for the first three years of the Activity was conducted and is detailed in Annex 1. The Activity staff at IFPRI and MSU shared contacts, reports, and information. All of the persons contacted among the identified key stakeholders (44 out of 44) and beneficiaries of the capacity building (282 out of GENDER POPULATION SAMPLE % OF THE POPULATION Female 465 64 14 Male 1,096 245 22 14 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 309) voluntarily responded to the evaluation questions. Table 4 shows the distribution of KIIs by geographic zone, while Figure 4 below indicates the breakdown of the capacity building respondents by organization. Table 4: Distribution of Respondents by Geographic Zone Zone/state Stakeholder Group Respondents Selected (no) Respondents Received (no) Completion Rate (%) Total Respondents (no) North-West/ Kaduna Public Sector 2 2 100 4 Academia 3 2 67 North-West/ Kebbi Public Sector 2 2 100 4 Academia 3 2 67 North-Central/ Benue Public Sector 2 2 100 3 Academia 3 1 33 North-Central/ Niger Public Sector 2 2 100 Academia 2 2 100 5 Media 1 1 100 South-West/Oyo Academia 5 5 100 6 NAC 1 1 100 South-East/Ebonyi Public Sector 2 2 100 4 Academia 2 2 100 South-South/Cross River Public Sector 1 Academia 1 1 100 South-South/Delta Public Sector 2 2 100 2 Academia 2 0 0 FCT Public Sector 10 6 60 15 NAC 5 4 80 Media 7 1 14 Private Sector 1 1 100 NGO/CSO 4 2 50 IFPRI/MSU 1 1 100 Source: Compilation by Evaluation Team (January 2019) 15 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Figure 4: Breakdown of the Capacity Building Respondents, by Category Data generated from the KIIs were coded and analyzed using the ATLAS.ti software tool for qualitative analysis. Data collection tools were linked to evaluation questions in order to aggregate and synthesize results. Themes from the respondents’ answers were coded and analyzed, and afterward from KIIs the Team identified respondent quotations to include in this report. Using STATA, cross-tabulations and bar charts were generated to show the percentage distribution of responses to quantitative data from the KIIs with the sample of capacity building training beneficiaries. Thus, the Team arrived at both qualitative and quantitative results. 6 11 28 82 155 282 2 4 10 29 55 100 Think Tank/NGO Private Sector Media Government Officials Academia Total Number in Category Percent Contribution Some respondents were reluctant to participate in the evaluation. One visiting Nigerian professor said, “I already provided my trip report to MSU and IFPRI. Why should I repeat it?” However, once she realized this was an independent evaluation, she cheerfully gave the evaluation team a quarter of an hour by telephone. 16 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Table 5: Analytical Approach and Data Collection Sources to Answer Each Evaluation Question EVALUATION QUESTION ANALYTICAL APPROACH DATA COLLECTION SOURCE 1. Is the Agricultural Policy Project implemented as intended according to the Contract agreement? Gather information from Activity work plan and report on the activities’ theory of change (TOC) and implementation plans. Verify with direct beneficiaries if the intended activities are implemented. Confirm with main government counterparts, direct beneficiaries, and independent experts’ quality of the delivered activities. Describe effect of the training activities on participants. Gather expert opinions on what activities are needed in the future and compare with the plan. Gather information on perceived effect of the Activity from experts. Document review; KIIs with USAID, government (federal and state), private sector, media, academia (universities and research institutions), farmer associations/NGOs/CSOs/think tanks, program scholars, capacity building participants and Activity Advisory Committee members 2. Are key stakeholders (government, think tanks/CSOs/community-based organizations, associations, private sector, academia, and media) satisfied with the capacity building activities implemented by the Activity? Why or why not? Compile list of key stakeholders and the list of capacity building activities (implemented and planned); gather opinion about relevance of the activities; and measure satisfaction with the activities from main stakeholders. Document review; KIIs with selected stakeholders, capacity building participants 3. To what extent has the Activity coordinated with other USAID/Nigeria FTF activities and other donor programs to achieve the overall strategic objectives of improving the enabling environment in the agricultural sector for private sector investment? Compile list of other FTF activities and other donor programs; review documents for evidence of cooperation; obtain information about cooperation with other donors from the Activity; confirm Activity cooperation efforts; enquire about potential for cooperation from other projects/donors; compare with the Activity plans. Document review; interview with Activity staff, donors, and USAID 4. What methodologies does the policy research component of the Project employ and what is the value of the research, conducted by the Activity, to the stakeholders? Compile list of research supported by the Activity and summarize the methods used. Confirm with authors. Obtain stakeholders opinion about the value and quality of the research. Confirm value of the research with independent experts. Document review; KIIs with scholars/researchers and the Activity staff 5. What measures is the Activity putting in place to ensure sustainability of the interventions? Compile/review the sustainability measures/strategy developed by the Activity and compare to what is implemented to date. Discuss with Activity staff if remaining planned activities will be implemented on time; confirm with stakeholders that sustainability activities are relevant and will lead to sustainability of the results. Find out from the stakeholders what works well, what needs to change and how the desired change will be achieved and sustained. Document review; Interviews with Activity staff; KIIs with government and other stakeholders on sustainability of Activity results 17 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 4.3 Desk Review A desk review of Activity’s documents was conducted to extract relevant information, including the technical proposal, original contract, Activity MEL Plan, annual work plans, quarterly and annual progress reports, data quality assessment (DQAs) reports, contract amendments, memorandums of understanding, and other relevant materials documenting the management, implementation process, and results for the Activity. 4.4 Qualitative Interview Guides KIIs and telephone interviews of beneficiaries were held with a variety of stakeholders, Activity and USAID staff, including GON staff (federal and state including the national legislature), think tanks, CSOs, community-based organizations (CBOs), farmer associations, academia, the private sector, and the media (see Table 6 below and Figure 4 above). 4.5 Key Informant Interviews Semi-structured KIIs typically reveal rich insights and views about an activity's performance. By using a semi-structured format to balance the requirement that respondents speak to all areas of importance to the evaluation, the respondents were given leeway to make broader observations about the Activity’s performance that might not be anticipated. Furthermore, whenever possible, the KIIs were taken one step further by including closed-response, survey-type questions. This permitted descriptive quantification of KII responses, such as “73 percent of KII respondents said the Activity was effective,” rather than “most” or “some” said. It is important, though, to consider that in qualitative data, the outliers can sometimes convey the most valuable information, as their concepts lie “outside the box” and do not mirror the status quo. There are seven main groups of key informants for this evaluation: 1. Activity staff 2. Universities involved in the Activity in USAID/Nigeria’s focus states (academia) 3. Federal and state ministries, including the National Legislature 4. Representatives of NGOs/CSOs/associations/think tanks 5. Media 6. Private sector partners, including agro-input dealers and agro processors 7. Activity Advisory Committee members 18 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Table 6: Organizations Included in the Field Work Sample STAKEHOLDERS NORTH SOUTH SAMPLE Federal Universities ABU Federal University of Technology (UT), Minna Federal University of Agriculture, Makurdi Federal University, Birnin Kebbi Federal University Ndufu-Alike (FUNAI), Ebonyi University of Calabar Federal College of Education (Tech) Asaba North = 3 South = 2 State Universities Kebbi State University of Science and Technology (KSUSTA), Aliero Kaduna State University (KSU) Ibrahim Babangida University Benue State University, Gboko University of Mkar, Mkar Ebonyi State University Delta State University, Abraka Cross River State University, Calabar North = 2 South = 2 Private Sector Nigeria Agro Input Dealers’ Association Nigeria Agri-Business Group North = 1 South = 1 People in Government FMARD Agricultural Research Council of Nigeria (ARCN) Kaduna State Ministry of Agriculture and Forests Benue State MANR Kebbi State MANR Niger State MANR Ebonyi State Ministry of Agriculture and Natural Resources (MANR) Cross River State MANR Delta State MANR North = 4 South = 3 NGOs, Think Tanks and Associations Youth Initiative for Sustainable Agriculture (FCT) Nigerian Association of Agricultural Economists International Institute of Tropical Agriculture (IITA) (Manson-FCT) Centre for the Study of the Economies of Africa (CSEA) Association of Deans of Agriculture Universities Agriculture Policy Research Network (APRNet), Enugu North = 2 South = 2 Media News Agency of Nigeria New Telegraph - Niger Daily Trust – Cross River state Business News, (Ebonyi) North = 2 South = 2 19 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 4.6 Mapping of Evaluation Questions with the Category of Stakeholders The five evaluation questions were addressed in accordance with the relevance of stakeholders to a particular aspect of the Activity. This is described in Table 7 below. On the basis of correspondence between the evaluation question and category of stakeholders, research instruments were prepared to drill the interview exercise down to necessary sub-questions, as will be shown shortly in the data collection protocols identified for the evaluation. Table 7: Evaluation Questions and Target Respondents S/N EVALUATION QUESTIONS RELEVANT STAKEHOLDER/ RESPONDENT/ANNOTATIONS 1 Is the FTF Activity implemented as intended, according to the Contract agreement? a. Have the methods of delivery contributed to ensuring the Activity is on track to achieving its annual and life-of-project targets? b. Are the intended results per the Contract agreement occurring? c. Is the Activity working on any policy issues? If yes, how critical and necessary are the policy issues chosen by the Activity in transforming the agriculture sector in Nigeria? d. How appropriate is the Activity’s capacity building of policy actors and other stakeholders? To what extent is the Activity taking into consideration the audience, methods, and effects of the capacity building? e. How does the Activity intend the capacity building to influence policy via policy actors and other stakeholders? f. On what evidence are the policies based? g. What should the Activity put in place to ensure that the end-line intended impact of the Activity is on track and achieved? MSU IFPRI USAID FMARD and state ministries Farmer associations/NGOs/CSOs Media Academia 2 Are key stakeholders (GON, think tanks, /CSOs, community-based organizations, associations, and media) satisfied with the capacity building activities implemented by the Activity? Why or why not? a. How well domesticated is the Agricultural Policy Activity in the states of Nigeria? b. To what extent are the state-level policy actors aware of national/federal initiatives and aligning state policies accordingly? c. To what extent has the Activity engaged multiple stakeholder groups (value-chain actors, farmers, private sector, policy actors, think tanks, CSOs, etc.) to achieve its intended outcomes? Satisfaction with capacity building activities Satisfaction with capacity building activities These stakeholders are listed in Table 5 above Question (c) is to be addressed to the IFPRI/MSU 3 To what extent has the Activity coordinated with other USAID/Nigeria FTF activities and other donor programs to achieve the overall strategic objectives of improving the enabling environment in the agricultural sector for private sector investment? a. What are the most critical constraints in the formulation and implementation of agricultural policies in Nigeria, and how is the Activity responding to these constraints? b. How responsive has the Activity been to the changing needs of the policy environment and seizing opportunities for impact? c. To what extent has the Activity fostered policy dialogue with and/or among USAID/Nigeria’s FTF activities and other donor programs? MSU IFPRI USAID FMARD Other relevant FTF activities to be provided by USAID Donors implementing related projects also identified and targeted for the interview 20 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation S/N EVALUATION QUESTIONS RELEVANT STAKEHOLDER/ RESPONDENT/ANNOTATIONS 4 What methodologies does the research component of the project employ and what is the value of the research, conducted by the Activity, to the stakeholders? a. What are the benefits of the Activity’s research to policy actors and stakeholders? Is it reaching them, if yes, how? If not, why not? b. How do policy actors and stakeholders use the Activity’s research? How do policy actors and stakeholders assess the utility of the Activity’s research? c. How knowledgeable are policy actors and stakeholders about the Activity’s different types of research products? d. How is the Activity’s research influencing and/or informing policy actors, private sector and other value-chain actors? e. What are the plans for dissemination of the different types of research products that the Activity produces? MSU IFPRI USAID FMARD and state ministries Farmer associations/NGOs/CSOs Media Academia 5 What measures is the Activity putting in place to ensure sustainability of the interventions? MSU IFPRI USAID FMARD 4.6.1 Distribution of Key Informants across Selected States One of the primary sources of data for this evaluation is KIIs involving 44 stakeholders spread across states selected on the basis of the nature of participation in the Activity. The Team conducted field work in Benue, Cross River, Delta, Ebonyi, FCT, Kaduna, Kebbi, Niger, and Oyo States, where various categories of stakeholders are located. The specific location of the interviewees is identified using GIS in each of the states (see Figure 5 below). Respondents are located not only at the state capital, but also in locations where project beneficiaries or participants are located. For example, in Kaduna State, the Team interviewed stakeholders both in Kaduna and Zaria while in Kebbi State, stakeholders were interviewed in Birnin Kebbi and Aliero. Table 8: Distribution of Respondents Interviewed during Mid-term Evaluation by State & Sector S/N State Academia Public Sector Private Sector Media Advisory Committee Members Total 1 Benue 1 2 0 0 0 3 2 Cross River 1 0 0 0 0 1 3 Delta 0 2 0 0 0 2 4 Ebonyi 2 2 0 0 0 4 5 FCT 0 6 1 1 7 15 6 Kaduna 2 2 0 0 0 4 7 Kebbi 2 2 0 0 0 4 8 Niger 2 2 0 1 0 5 9 Oyo 5 0 0 0 1 6 Total 15 18 1 2 8 44 21 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Figure 5: Distribution of Respondents Interviewed during Mid-term Evaluation by State 4.7 Sample Size for Training Beneficiaries of Activity The sample size for training beneficiaries was derived from the database of 1,565, cleaned to 1,561 beneficiaries, provided by the Agricultural Policy Project. A confidence level of 95 percent and 5 percent margin of error were used to select a sample size of 309 beneficiaries from the list provided. The sample size was calculated using RaoSoft Software.3 This statistical approach was used to ensure a representative distribution of the sample frame across all the states covered by the Activity and allow the Team to generalize results to the population. The sample frame contains information about the training beneficiaries, including some details on what interventions they have received and what organization the beneficiaries came from, when they have participated in the Agricultural Policy project, and type of training. A random sampling procedure in STATA was used to select a representative sample of beneficiaries from the total population across states and FCT covered under the Activity. The organization and gender distribution of the randomly selected representative sample of 309 beneficiaries was 64 of the sample sizes (21 percent) female and 245 (79 percent) male from the states and FCT based on the population (see Table 3 above). 3 http://www.raosoft.com/samplesize.html 22 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 4.8 Qualitative Data Analysis The data collected were analyzed qualitatively. This qualitative analysis was guided by the main evaluation questions and addressed project implementation performance and the extent to which project objectives have been achieved. Some of the data were subjected to descriptive analysis using suitable descriptive statistics (e.g., mean, percentages) focusing on performance indicators as variables and the proportion of respondents attesting to the relevance of the activities, the level of awareness and satisfaction, appropriateness of methodologies, outreach procedures, and other achievements based on the activity’s objectives. 4.9 Quality Control Mechanism for Data Gathering, Analysis and Oversight Both primary and secondary data were collected for this evaluation. The secondary data were obtained from the project units, project beneficiaries, and periodic reports (e.g., quarterly and annual reporting). For ease of access, formal letters requesting such data were obtained from USAID/Nigeria and presented by the Team to appropriate authorities concerned with the release. These letters were critical passports to the information required for successfully carrying out this evaluation. The primary data were obtained through KIIs. As a quality control mechanism, Team members conducted all KIIs. To minimize waste of time during interviews, all responses were recorded via a digital recording device with respondent consent. This made programming of appointments efficient and fact-checking possible during subsequent stages of the evaluation. Team members that conducted the interviews took handwritten notes on emerging key issues relating specifically to the evaluation questions and sub￾questions. This was necessary for better interpretation of the data. 4.10 Guiding Principles and Values Ethical considerations are of the utmost importance and the evaluation was conducted with integrity and sensitivity, showing the highest level of respect to all participants. Informed consent was obtained from all respondents prior to their involvement in the data collection process. Respondents were referred to by role/job title and location, if necessary, but only where this information would not reveal the identity of the respondent(s). The Team adhered to the “do no harm” principle, placing the protection of human subjects as our highest priority. The Team followed DevTech and USAID’s security advice. The Team ensured the confidentiality of data, respected the privacy of all individuals concerned, and made all data collected available to DevTech and USAID in a usable format. 4.11 Study Limitations • The team used qualitative approach, which is widely accepted in conducting performance evaluations; however, there are concerns of biases that may be due to the variations in memory recall, especially from beneficiaries of the capacity building component. Attitudes and emotions may have interfered with the assessment of the activity’s implementation. • The time frame for data collection is short and the stakeholders/beneficiaries of capacity building are widely spread across the country. Hence, samples were drawn from the population based on a range of specific criteria. 23 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation • Behavior and body language of respondents could not be observed during telephone interviews, a limitation particularly for the capacity building beneficiaries. Nevertheless, the data obtained through the KIIs can still be validly applied to meet the objectives of this evaluation. The Team has relevant knowledge of the policy environment and subject matter, which is often brought to bear on the interpretation of the results. Triangulation was accomplished through studying a wide range of Agriculture Policy Project documents and reports to further validate emerging findings. Moreover, in addition to the KIIs, secondary data obtained from project documents were used to examine some of the evaluation questions. The evaluation benefited from comments of experts during debriefing sessions and views of different commentators on preliminary drafts of this evaluation report. 24 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 5.0 FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 5.1 Evaluation Question 1 (EQ1) Is the Feed the Future Nigeria Agricultural Policy Project implemented as intended according to the contract agreement? 5.1.1 Findings for EQ1 EQ1 (a) and (b) – Have the methods of delivery contributed to ensuring the Activity is on track to achieving its annual and life-of-project targets? Are the intended results per the Contract agreement occurring? To answer these questions, the Team conducted an analysis of the performance indicators identified in the project document to determine whether the Activity is on track to achieve the intended results. In this way, the Team seeks to determine whether what should happen is actually happening. Details of the indicator analysis are presented in Annex 1. • There is a varying definition of research output. From year one to year three, the indicator used is “number of policy research and best practice papers generated.” However, in year four this has changed to “number of high-quality research reports published.” • There is a lack of specificity in the indicators. In year three, the Activity used “Number of formal, informal, or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed (custom).” A stronger indicator to capture output and findings already exists in “Number of agriculture policy communications developed and/or written for stakeholder consumption.” • There is instability among indicators. An indicator that was used at the start of the Activity was retained for one or two years—and then archived (e.g., indicator number two). For example, “Number of stakeholder-learning forums (national or global) held where findings/best practices are presented” was archived prior to the start of year four. • There is questionable relevance of indicators. Indicators such as “Number of for-profit private enterprises, producers’ organizations, water users’ associations, women's groups, trade and business associations, and CBOs receiving USG food security related organizational development assistance” has only tangential relevance. By and large, the Agricultural Policy Activity is being implemented according to the Contract agreement, actively working in the three areas of research, capacity building and outreach. All seven FTF states have benefited, as well as key stakeholders and beneficiaries—particularly in the FCT, where the GON is located in Abuja, and in the university town Ibadan, Oyo State. The method of delivery, especially capacity building activities, has contributed to ensuring that the Activity is on track to achieving its annual and life￾of-project targets. Whether the intended results per the Contract agreement are occurring are discussed in this section and in Annex 1. Simply put however, the Activity is meeting its targets in majority of the indicators; the indicators themselves, however, leave some room for improvement. 25 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ1 (c) – Is the Activity working on any policy issues? If yes, how critical and necessary are the policy issues chosen by the Activity in transforming the agriculture sector in Nigeria? The Activity has undertaken research and analysis on sensitive and delicate topics. These include the significance of the wide-spread smuggling of rice and poultry (permanent bans in violation of their commitments under the Economic Community of Western African States (ECOWAS) and the World Trade Organization (WTO)) and on the effect of climate change on Nigeria’s poultry sector. EQ1 (d) – How appropriate is the Activity’s capacity building of policy actors and other stakeholders? To what extent is the Activity engaged in capacity building taking into consideration the audience, methods, and effects of the capacity building? The Team examined each category of stakeholders and the type of training programs in which they participated to discover participants’ perceptions on what they gained from the trainings and their relevance and usefulness. • The results show that the training programs are appropriate. A majority (79 percent) of those who participated in communication workshops are stakeholders from the media, while 83 percent of those participating in in the policy workshop are from the think tank/NGO category. The workshops on the use of software are largely attended by academia (80 percent) and the private sector (73 percent) (Figure 6). • The blend of participants in the capacity-building activities leans heavily towards post-graduate scholars and university professors. For example, the Activity did not target experts from the National Agricultural Research Institutes. The National Agricultural Research Council of Nigeria (ARCN) is the exception, in that two dozen ARCN experts have participated in the Activity. • As shown in Figure 7, although percentages vary by stakeholder type the capacity building workshops were reportedly very useful to the participants and many of them confirmed that they gained from their participation in the workshops. Even on the lowest end, 55 percent of those attending from the private sector considered the workshops to be highly useful. Respondents from the government (80 percent), academia (77 percent), the media (75 percent) and think tanks/NGOs (83 percent) all indicated that the workshops have been highly useful. The majority of the stakeholders have also gained in terms of improvement in technical skills and increased access to new analytical tools and approaches (see Table 8). Notably, KIIs with persons active in the media showed satisfaction in terms of “helping them tell the story right.” • The point must be stressed however, that the effects of the training activities depend on the type and quality of training. Where there is wide-spread satisfaction with the quality of training, questions arose regarding the attention devoted to advance planning, the content and areas of emphasis, and how the knowledge derived can be put to gainful practice professionally. These are areas which should be strengthened as the Activity strives to meet specified targets and achieve end-line objectives. 26 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Figure 6: Stakeholder Participation in Training Workshops, by Type and Topic4 Figure 7: Usefulness of the Training Workshops (“How useful were the trainings?”) Table 9: Participants’ Gain from the Capacity Building Workshops Respondent Type (n) New Technical Skill5 New Analytical Tools New Approach Others Academia (155) 72% 51% 59% 3% Government (82) 70% 66% 73% 1% Media (28) 64% 68% 61% 0% Private sector (11) 64% 45% 36% 9% Think tank/NGOs (6) 33% 33% 17% 17% All (282) 69% 56% 62% 2% 4 The Sample Average provides the aggregate participation rates from the sampled trainees of capacity building trainings. Participation in trainings are not mutually exclusive. 5 Based on the interview guide and the context of this survey, Technical Skills refers to technical writing skills, New Analytical Tools are new tools of analysis, and New Approach refers to a new process of doing things gained as a result of participating in Activity’s organized capacity building activities. 27 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Source: Computation by Evaluation Team EQ1 (e) – How does the Activity intend the capacity building to influence policy via policy actors and other stakeholders? • The contract agreement called for the Activity to be making a tangible influence on Nigerian agricultural policies in order to increase the usage of research and policy analysis within FMARD and state ministries. This occurred when the Activity assisted Cross River State in a 2018 review of the state’s 2014 Agricultural Plan, as well as with commodity-specific state-level plans for development of the rice and fish sector in Kebbi State. • The way the Activity expects capacity building to influence policy varies according to the type of training and its participants. For instance, the media should apply the knowledge gained to improve their understanding of agricultural policy issues and do a better job of communicating those issues. In this case, policy may be influenced through better dissemination of information. With those participants involved in software training, they are in turn taking on the training of those on their duty post. In addition, they should use the knowledge gained in doing better analyses, and thus provide better reporting that then can be widely disseminated. In general, however, participants are expected to apply the knowledge gained to their jobs so that they can perform better— thus making their contribution to the improvement of the policy process. • The Activity is expected to manage these varying expectations and to design mechanisms to ensure that they are met. This aspect is further addressed in Section 5.1.3, which makes recommendations to make the Activity truly “influential” through the three components being implemented. EQ1 (f) – On what evidence are the policies based? Policy gaps or areas of emphasis in the implementation of the Activity components are based on experiences from previous research, interactions with the literature, and areas of interest of project participants. The link between the Activity and policy authorities is weak, especially at the federal level, in establishing which policy priorities are to be entrenched in the Activity’s work plan. EQ1 (g) – What should the Activity put in place to ensure that the end-line intended impact of the Activity is on track and achieved? The findings of this evaluation include areas that need to be improved upon based on the evaluation of the various components so far implemented. Areas of weakness are as follows. • Short duration. With the complexity of the analytic methods taught during the capacity-building sessions, respondents suggested that the Activity should allow a longer duration for training programs, i.e., four days instead of two days. • Limited access to software after training. The trial version of STATA used for training expires within a month and thereafter, students, professors, and researchers with limited means have no way to put their new knowledge into practice. • Low perceived value. Local respondents involved in the Activity’s research efforts complained that the activity does not aim high enough in terms of capacity building for local researchers. More than one university respondent said, “They just used us to collect data.” • Weak activity coordination at the state level. Activity staff stated that “working at the state level requires a lot of effort,” which is certainly true. 28 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation • Limited responses. There is no feedback mechanism to strengthen interaction among IFPRI, MSU, FMARD, and the state-level focal points/coordinators. • Lack of formal work planning. The Activity interacted with stakeholders and partners on an informal basis, with no memorandum of understanding (MOU) with any groups other than its other U.S.-based implementing partner, MSU. Respondents repeatedly complained that “there was no work plan,” which came as a surprise to the Team. In the light of the foregoing, the Activity should urgently take necessary steps to strengthen the above areas of weakness in order to achieve the objectives of this project. Specific measures that may be helpful in this regard are presented in Section 5.1.3. 5.1.2 Conclusions for EQ1 • The Agricultural Policy Activity has over-emphasized academic-style research that may be seen as advancing the careers of the IFPRI and MSU researchers, rather than as bringing Nigerian collaborators to the forefront. • The Agricultural Policy Activity has not sufficiently emphasized dissemination of the research reports and policy analysis, limiting its effect. • Even though it has largely met its performance indicators, the Activity has not achieved the intended results. This may reflect faults in the indicators. • Mapping of the audience, training curriculum, and expected outcomes were not adequately considered in the design of the Activity. For instance, both Nigerian and U.S.-employed bureaucrats must be conversant with existing policies—such as the Agricultural Promotion Policy of Nigeria document pertaining to the federal level, the Economic Recovery and Growth Plan, the National Agriculture Investment Plan, the Comprehensive Africa Agriculture Development Program—and their respective roles, including monitoring and evaluation to verify the realization of desired results. Additionally, directors undergoing training in STATA for three to four days may not achieve the desired result, due to distractions in their everyday tasks. • The Activity did not properly establish partnerships with local groups and interactions remained on an informal level. Many participants have no idea whether an MOU was ever signed with their institutions and some researchers are involved with no extant contract. Involvement of researchers is often piece-meal and payment follows no schedule. Sometimes, the extent of involvement and the financial implications of their participation was not understood by the researchers. • While the Activity offers trainings in how to analyze data, it does not ask local researchers to conduct the actual analysis on its research efforts, reserving those roles for the IFPRI and MSU staff. 5.1.2.1 Conclusions Related to the Activity’s Performance Indicators • Performance indicators do not comply with the SMART approach to indicator development. • The quarterly assessment of activities is not based upon a strict definition of indicators. It is difficult to track progress using the indicators specified in the initial Activity MEL Plan. This is evidenced by the proliferation of indicators (in the form of varying definitions). Although 12 indicators are often mentioned, the number of indicators rose to a high of 17 indicators during year three, as shown in Annex I, Table 4.2. 29 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation • The Activity met its targets during the first two years. • The Activity needs to pay particular attention to three areas of concern in order to improve the overall effectiveness and maximize the benefits of project monitoring:  Untimely availability of the FY18 Quarter Four Report covering July 1, 2018 to September 30, 2018 to the Team. This report was not available to the Team by November and throughout the end of December 2018. It was only made available in January 2019 by special request.  Negative deviation of some performance indicators in year three compared with years one and two of the activity.  Lack of data to track some indicators, especially in the first and second years of the activity. 5.1.3 Recommendations for EQ1 • To ensure that the Agricultural Policy Activity accomplishes its intended impact, a greater emphasis on dissemination is advisable from here forward. • To heighten the impact on actual policy-making, the Activity should take a more “hands-on” approach by working to explain the research to individuals and groups, ensuring they understand both how the research was conducted, and working with them on drawing out the policy implications. • The Activity delivery process should be more organized, more coordinated, and more transparent. Activity staff should be more active in that arena, rather than in new research. • Potential beneficiaries of the capacity building activities should have a proper understanding of existing policies, including implementation strategies and expected results. The training requirements of bureaucrats must be distinguished from those of policy analysts. • Roles and responsibilities of partners and coordinators should be well-defined, with time for collaborative planning, and these roles made known to all stakeholders. • The capacity building process should include:  Local participation in the process of preparation of the training needs and curriculum within each thematic area;  Preparation of a training calendar;  Printing of brochures, research reports, and policy briefs produced by the Activity and their wide circulation to all stakeholders well ahead of time;  Careful selection of eligible participants, in collaboration with knowledgeable local stakeholders, with the aim of broad participation including by women’s groups, youth, and socially disadvantaged persons. 5.1.3.1 Recommendations Related to the Activity’s Performance Indicators • Retain only SMART indicators. • Stop undue proliferation of indicators to avoid shifting of goal posts. • Be consistent in the use and definition of indicators. • Discontinue indicators that are not relevant to the Activity (e.g., “Number of for-profit private enterprises, producers’ organizations, water users’ associations, women's groups, trade and business associations, and CBOs receiving USG food security-related organizational development assistance”). 30 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation • Remove redundant indicators, such as the “Number of formal, informal, or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed.” 5.2 Evaluation Question 2 (EQ2) Are Key Stakeholders (GON, Think Tanks, Civil Society Organizations, Community-Based Organizations, Associations, and Media) satisfied with the capacity building activities implemented by the activity? Why or Why not? 5.2.1 Findings for EQ2 Particularly for the students and professors, the Activity has had a real and positive influence on their career path in giving them new skills. Those interviewed in the public and private sectors, as well as CSOs and the media, were less enthusiastic—with a handful of respondents downright critical of the approach undertaken. Overall, 77 percent of the capacity building respondents said they found the trainings “highly useful” and 79 percent found them “very relevant.” These findings, however, vary widely according to category of stakeholder. In terms of usefulness of the training, the proportion of those who claimed that the trainings were highly useful vary from NGOs (83 percent), government (80 percent), academia (77 percent), media (75 percent), to the private sector (55 percent). The results show that majority of the participants in each of the categories of stakeholders found the trainings highly useful. Those who found the trainings not useful are generally less than 5 percent in each category - with the exception of the private sector, where 18 percent of participants found the trainings not useful (Annex 2). A similar pattern was observed regarding relevance of the training to the agricultural policy process. The proportion of the respondents that found the trainings highly relevant were NGOs, followed by the government (80 percent), academia (79 percent), media (71 percent), and the private sector (64 percent). The proportion of the respondents who found the trainings not relevant to the agricultural policy process was generally less than 10 percent—with the exception of government ministries, departments, and agencies, which showed a proportion of 15 percent (Annex 2). EQ2 (a) and (b) – How well domesticated is the Agricultural Policy Activity in the states of Nigeria? To what extent are the state-level policy actors aware of national/federal initiatives and aligning state policies accordingly? • State-level policy actors (including analysts) reported a low level of awareness of federal-level agricultural policies, limiting alignment of state and federal policies. • Respondents noted that, with only a few exceptions, the research agenda and approach were determined by the Activity, rather than mutually with local research groups, ministries, and civil society. Several respondents related that the Activity staff appeared to have little interest in the local research agenda. At the University of Calabar, for example, and at the Cross-River State Ministry of Agriculture, the Team was told that the Activity’s personnel “was not even aware of any research activities going on in Cross River State.” Fifty-two percent of the capacity building beneficiaries interviewed through mid-January 2019 did not feel that they had enough advance input on Activity’s activities. Senior officials at FMARD volunteered that the Nigeria Agricultural Policy Activity has not adequately aligned its work program with that of FMARD. As one said, “Where is the collaboration with FMARD on research?” 31 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ2 (c) – To what extent has the Activity engaged multiple stakeholder groups (value-chain actors, farmers, private sector, policy actors, think tanks, CSOs, etc.) to achieve its intended outcomes? • Research and capacity building activities have emphasized post-graduate university students, with many others also benefiting from training and interaction with MSU and IFPRI experts. In a country with several research institutes covering the various sub-sectors of agriculture, however, this is rather restrictive. Many young and experienced researchers stated that there is no opportunity to exploit Nigeria-based opportunities, which could enhance their capacity to conduct policy research. • In many instances, a predictable calendar of activities has not been shared ex ante or ex post with local partners. Over half of the key stakeholders feel that scheduling and process has not been clear to them in the course of the project implementation. • 158 out of the 282 students, professors, and others (56 percent) learned the utility of the statistical software STATA and its pull-down menu features. No respondent was happy to learn that the license for the distributed software would expire a month later. • Many respondents stated that they participated in the 2018 Agricultural Policy Activity conference, but otherwise had had little interaction with the Activity and were not kept aware of activities on an ongoing basis. One respondent from the academic/research world made these comments: “The organization of the conference was okay. We were lodged in a very good hotel, but local transport was a bit of a problem. That was one of the downsides.” He recommended that “there be a conference proceeding, with information about the presenters and the papers all in one place. Like a book of abstracts that would then be distributed to people.” • One of the aspects of the training workshops that was liked best was the facilitation by young people. “The young presenters were full of energy. There was great diversity of experience. Older people were assigned to give critiques.” 5.2.2 Conclusions for EQ2 • The design of the capacity building activities in some cases failed to recognize the existing policy research needs of the various sub-sectors of agriculture in Nigeria. • The Activity would have had greater “local buy-in” if its activities were oriented to the local research agenda, rather than priorities determined in Washington, DC or East Lansing, MI. • The Activity has not adequately coordinated with FMARD’s evolving project coordination unit, nor with those coordination units in the states’ Ministries of Agriculture. Researchers tend to seek out the people with knowledge of a particular topic, such as agricultural extension services, and in every state or local government area there is a certain protocol to follow in terms of administrative hierarchy. This is an area that requires a lot of time and effort by the implementing partner, best done at the beginning of the Activity. • While USAID is likely not going to fund dozens of licenses for STATA, there may be other ways for the Activity to make STATA available, particularly on a “site license” basis for universities or groups like the ARCN. These opportunities should be further explored. • Success at the state level is often because the latter bodies lack in-house research capability. 32 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 5.2.3 Recommendations for EQ2 • A comprehensive list should be constructed of research institutes in the country, or at least in the seven FTF focus states, identifying their capability in conducting policy research in agriculture, existing skill gaps, and areas of needed improvement or future emphasis. • The Activity should emphasize outreach to those organizations identified, including presentation of the research and policy analysis conducted to date. • A detailed proposed work plan should be shared with these stakeholders and their input solicited. Inputs received from stakeholders on the proposed work plan would also help inform any potential FTF activity dedicated to the policy domain. • The Activity should then propose a revised work plan for the remainder of the Activity and work on fixing dates and agendas with the partner institutions well in advance. • The Activity must include federal-level policy initiatives in its training programs to promote awareness and understanding among policy actors at the sub-national level and among the research community. Including modules on ECOWAS policies is advisable as well. • For each of the seven FTF focus states, the Activity should develop a mechanism for each state ministry of agriculture to formulate a “wish list” of research and policy analysis needs for the remainder of year four and for year five and beyond. 5.3 Evaluation Question 3 (EQ3) To what extent has the Activity coordinated with other USAID/Nigeria FTF activities and other donor programs to achieve the overall strategic objectives of improving the enabling environment in the agricultural sector for private sector investment? 5.3.1 Findings for EQ3 EQ3 (a) – What are the most critical constraints in the formulation and implementation of agricultural policies in Nigeria, and how is the Activity responding to these constraints? There are many critical constraints in the formulation and implementation of agricultural policies in Nigeria. Regarding the Activity, however, several constraints were identified to be crucial. These are the weak capacity of government officials and inadequate funding arising from low level of government revenue, delays in budget approvals, and delays in the release of approved funds. This was found to be the case both at the federal and state levels. EQ3 (b) – How responsive has the Activity been to the changing needs of the policy environment and seizing opportunities for impact? The Activity has been responsive to the changing needs of the policy environment by the type of policy issues it has attended to in the focal states. For example, the Activity embarked on some policy dialogues at the height of the conflict between herders and farmers, with a view to understanding the causes and drawing on lessons to minimize or prevent its occurrence in some other states. This was carried out in Niger and Kebbi States in partnership with local ministries, departments, and agencies. This type of initiative, however, is not easily discernible at the federal level. 33 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation EQ3 (c) – To what extent has the Activity fostered policy dialogue with and/or among USAID/Nigeria’s FTF activities and other donor programs? The Activity has participated in the Agriculture Donor Working Group, although this is not sufficient evidence of fostering dialogue with or among USAID/Nigeria’s FTF activities, or other donor programs for that matter. Collaboration of the Activity with other ongoing USAID-sponsored FTF activities is at a low level. For example, at the state level, Activity partners expressed a lack of awareness of USAID￾sponsored projects and other ongoing Nigeria FTF activities. In Kaduna State, local partners are unaware of the cultivating new frontiers for agriculture (CNFA) activities; thus, they could not extend an invitation to them to attend the dissemination activities relating to the policy briefs that have been prepared under the Activity. 5.3.2 Conclusions for EQ3 • The challenge of limited funding by ministries, departments, and agencies exacerbates the weak absorptive and executive capacity critical both at the federal and sub-national levels. • Coordination with other USAID/Nigeria FTF activities and other donor programs to attract private-sector investment and increase productivity and growth in the sector is weak, as there are few others working in the policy-formulation domain. USAID is funding The Alliance for a Green Revolution in Africa to implement Partnership for Inclusive Agricultural Transformation in Africa (PIATA) in Kaduna and Niger States. Yet there is no evidence that the Activity is coordinating with these initiatives to strengthen efforts aimed at achieving the project objectives. • There was a lack of personal financial incentive to key stakeholders, in particular the National Advisory Committee members, who are senior experts and essentially participate on a pro bono basis. 5.3.3 Recommendations for EQ3 • Timely approval of budgets is urged, with prompt release of appropriated funds and rigorous efforts to generate internal revenue. The Activity, on its part, can consider making funds available to execute collaborative policy works with the focus government. • The Activity should coordinate with USAID to identify existing and new FTF activities in the states where the Activity is being implemented, and design appropriate mechanisms for collaboration in each component of the Activity. • CNFA is being funded by USAID to implement the FTF Nigeria and Nestle Maize Quality Improvement Partnership activity in Kaduna State. The Activity can leverage the efforts under this partnership, as well as the PIATA initiatives, to ensure that the policy-development actions being undertaken in Kaduna State, for example, are leveraged for greater influence on productivity and food security. 5.4 Evaluation Question 4 (EQ4) What methodologies does the research component of the project employ and what is the value of the research, conducted by the activity, to the stakeholders? 5.4.1 Findings for EQ4 The Team found that the Activity uses sound academic-style methodologies in its research reports, starting with research design, data collection, data analysis, and formulation of conclusions. There is a 34 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation strong emphasis on teaching STATA. Few, if any, of the stakeholders outside of those collaborating on the research could recall actually using the research reports, although the policy briefs were more widely read. EQ4 (a) – What are the benefits of the Activity’s research to policy actors and stakeholders? Is it reaching them, if yes, how? If not, why not? The Activity’s research has primarily benefited the post-graduate students selected for the MSU Scholars Exchange and other Nigerian stakeholders collaborating on specific research reports. Policy makers and other stakeholders, based on the KIIs, reported very low usage of the Activity’s research. Part of the reason for this is the Activity’s emphasis on research rather than policy analysis. EQ4 (b) – How do policy actors and stakeholders use the Activity’s research? How do policy actors and stakeholders assess the utility of the Activity’s research? • At the federal level, policy makers and stakeholders expressed that they did not make much use of the Activity’s research due to a disconnect between the Activity’s research priorities and the work plans and policy priorities of FMARD. • At the state level, the Activity has assisted the state agriculture ministries in Cross River State with a sector-wide plan. In Kebbi State, the Activity supported the development of commodity￾specific plans for rice and fish. EQ4 (c) – How knowledgeable are policy actors and stakeholders about the Activity’s different types of research products? The two major conferences held to date have provided the opportunity for policy actors and stakeholders to learn about the Activity’s research activities through presented papers and panel discussions. Several stakeholders discussed how paper copies of the research reports and policy briefs were easier to digest than electronic copies. Not a single Nigerian stakeholder mentioned accessing the Activity website. The extent to which stakeholders have access to available online research products depends on the availability of the requisite infrastructure in the partner institutions or among individual stakeholders. The issue of weak demand for such products may have to be addressed through strengthening research infrastructure among the local partners and stakeholders. EQ4 (d) – How is the Activity’s research influencing and/or informing policy actors, private sector and other value￾chain actors? There was little evidence that the Activity is actually influencing policy actors, the private sector, and other value-chain actors. The academic nature of the research is not well-suited to policy analysis needs at either the federal or state level, which require quicker responses to pressing questions—questions which FMARD, State Commissioners of Agriculture, and department heads grapple on any given day or week. Through the two major conferences, the Activity has successfully informed the audience about ongoing and completed research. Further, the MSU scholars and other experts participating in the collaborative research have learned about the IFPRI and MSU research methods. EQ4 (e) – What are the dissemination plans for the different types of research products that the Activity produces? Based on the KIIs, due to limited bandwidth and electricity in Nigeria, the most effective approach to dissemination by the Activity has been in-person dissemination of research products. One high-level FMARD official noted that he had a box full of the reports by the Activity (and the National Assembly (NASS)) that he did not have sufficient time to read. Several of the MSU Scholars reported that the MSU professors had provided them with numerous pertinent studies and materials, in both paper copies and electronic files. One capacity building participant noted that the database on poverty statistics used for the STATA training at her university was “precisely what I needed for my thesis.” 35 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 5.4.2 Overall Findings for EQ4 • The methodology for identifying research priorities is neither clear-cut nor as inclusive as expected. According to an interviewee from FMARD, “The project is supply-driven and less by demand, which is not too good for a policy project like this. No joint development of framework for the implementation of the project. The project is solely conceived by USAID and MSU. The GON through the FMARD was not involved in the concept of the project that borders on policy development and support. Someone just called and informed me about the project.” • Targeting with regard to policy research is restrictive. Researchers in several research institutes in Nigeria are not prioritized as target groups. A mapping of potential research institutions that could participate in the Activity would yield over 50 institutions in Nigeria, and yet the research reports and policy briefs have been carried out only by two institutions: IFPRI and MSU. • Some research studies are based on the interest of researchers, rather than a consensus of policy actors, as evidenced by the experience in the National Agriculture and Extension Research Liaison Services (NAERLS). When asked how the research topics were determined, an interviewee from NAERLS explained that “research can be an off-shoot of a previous research. It is also the case that researchers determine their own focus.” • Decision-making in the research process between the Activity and collaborators is one-sided and led by the Activity. Fifty-two percent of the initial round of KIIs with capacity building beneficiaries expressed that they were not adequately consulted a priori on the training needs, priorities, and length of training. • Partners expressed frustration about not knowing the extent of their participation in the research process. Some collaborators (e.g., NAERLS) expressed that their staff are solely asked to collect data. “We are not involved in data analysis, which is where we need capacity building. We are also not involved in report writing.” • Respondents expressed confusion regarding which research papers, policy briefs, or training activities were sponsored under the Activity, versus those attributable to the IFPRI/MSU project, the NASS, the FTF Innovation Lab located at MSU, and the series of IFPRI working papers. In one office after another, respondents did not recognize the name “Nigeria Agricultural Policy Project,” often simply referring to it as a “Feed the Future Project.” • State government officials and academics in Cross River and Kebbi States expressed satisfaction that the Activity had been able to help them develop state-specific agricultural development plans, including for specific commodities. • Nearly every FMARD official interviewed saw the lack of linkages between the Activity and FMARD’s policy coordination unit as a weakness in the Activity’s approach. One high-level female official said, “Which department do they respond to?”. 5.4.7 Conclusions for EQ4 • The policy research activities are not clearly demand-driven. • The prioritization of research topics is restrictive; this has tended to make the value of the research less visible than would have been the case if research priorities had been determined by Nigerian policy actors and key stakeholders. • While the Agricultural Policy Activity very well may be in conformity with USAID rules, the Team concludes that the Activity has not done sufficient branding, dissemination, and publicity- 36 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation generating about its specific products and USAID’s support. Considerable confusion exists among the very stakeholders who have been benefiting, in part due to the series of name changes. • One of the Activity’s greatest successes has been assisting the agriculture ministries of Cross River and Kebbi in applied policy analysis. While the Activity should not be trying to set agricultural policy in Nigeria, it is a highly useful service to help policy actors to have as much information as possible during the policy development process. 5.4.8 Recommendations for EQ4 • There should be a transparent process of identifying research topics which should be sufficiently inclusive of key policy actors and stakeholders. • For effective capacity building, researchers from partner institutions should be fully involved in the research studies from conceptualization through report writing and publication, in that way most effectively “moving up in the research food chain.” 5.5 Evaluation Question 5 (EQ5) What measures is the activity putting in place to ensure sustainability of the interventions? 5.5.1 Findings for EQ5 • Whereas all of the policy research was collaborative in nature, the leadership of the research teams is restricted to IFPRI and MSU researchers. This limits sustainability. • There was no support to acquire the research tools (especially STATA) for use by those who are trained to enable them, in order to practice what they learned and continue to deploy their knowledge to implement policy research. • The most convincing evidence for sustainability is the knowledge gained by the MSU scholars, research collaborators, and capacity building beneficiaries. As one MSU scholar said, “The knowledge gained carries on.” • Two groups expressed interest in becoming local partners. The first is the Agricultural Policy Research Network (APRNet). The current head of APRNet stated “We would like to broaden the participation of APRNet in the Project.” APRNet’s roughly 125 members may be the biggest “consumers” of the Agricultural Policy Activity’s research. The second is the Youth Initiative for Sustainable Agriculture (YISA), which works with youth in the fields, an aspect the Agricultural Policy Activity does not do. YISA would offer a complementary linkage. 5.5.2 Conclusions for EQ5 • The sustainability measures are weak and unlikely to ensure regular conduct of policy research to enhance growth in the sector. • The lack of a local implementing partner such as the APRNet or YISA limits the transfer of functions performed by the Activity to local actors. One FMARD official noted, “Development partners come and try to sell their idea and it’s easy for locals to say ‘yes.’ […] Whatever investment they have made, five years down the line, they won’t have if there’s nothing on the ground to say, ‘this is what we have achieved.’” 37 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation • It is not clear how the Activity intends to transfer its functions to local stakeholders. 5.5.3 Recommendations for EQ5 • Nigerian policy researchers should be included in the conduct of policy research under the Activity. This should be achieved through an entrenched and systematic process of calling for research proposals from Nigerian researchers in already identified thematic policy areas. For instance, the Activity can issue requests for proposals from the agricultural policy research community and select the best proposals for funding under the identified research themes. • The Activity should consider the possibility of supporting at least one institution in each focal state to acquire a STATA site license for use in policy research. Reaching out to other donors is recommended, with the Activity facilitating contacts and installation. • The Activity should support learning and linking partnerships between local institutions, such as between two universities (Federal University of Agriculture, Makurdi [FUAM] and Benue State University [BSU]), (ABU and Kebbi State University [KSU]), (ABU and Kebbi State University of Science and Technology, Aliero [KSUSTA]), as well as between universities and Planning Research and Statistics Departments of Ministries of Agriculture in the focal states. These partners should participate in each of the components of the Activity: research, capacity building, and outreach. Rather than limiting the activities to IFPRI and MSU, funds should be provided to the local institutions to build partnerships and jointly carry out specific research, training, and dissemination activities. This is an important built-in mechanism to foster the sustainability of the Activity. END 1 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation ANNEX 1: STATEMENT OF WORK USAID/Nigeria Feed the Future Nigeria Agricultural Policy Activity Mid-Term Performance Evaluation Scope of Work I. BACKGROUND INFORMATION Activity Identification Data II. DEVELOPMENT CONTEXT Background USAID’s Feed the Future (FTF) activities in Nigeria facilitate the attainment of the goals under the agenda for agriculture expressed by the Federal Government of Nigeria. In order to achieve the outcomes outlined in these strategies, there is a need for the government, Nigerian institutions, and other key stakeholders in the policy process to strengthen information and knowledge management systems, identify and fill key knowledge gaps, strengthen strategic planning through promoting participation, and build networks. In fact, since the introduction of the Agricultural Transformation Agenda (ATA) in 2010/2011, there has been increased demand from the Federal Ministry of Agriculture and Rural Development (FMARD) and its stakeholders for greater policy research and capacity training as Nigeria seeks to accelerate its agricultural transformation. The Nigeria Agricultural Policy Activity is designed to address this need for policy research and capacity building, and work with various stakeholders to ensure that Nigerian institutions are equipped to respond effectively and in a timely manner to the increasing capacity, knowledge and information needs of FMARD. The Nigerian Agricultural Policy Activity is an associate award to the USAID FTF Innovation Lab for Food Security Policy Leader with Associate award implemented by Michigan State Development Objective Activity Title Broadened and Inclusive Growth Feed the Future Nigeria Agricultural Policy Activity Award Number Project Dates AID-620-LA-15-00001 July 1, 2015 to June 30, 2020 Type of Contract Project Funding Cooperative Agreement $12,499,999 Agreement Officer’s Representative (AOR) Implementing Partner (IP) Dr. Osagie Aimiuwu Michigan State University and International Food Policy Research Institute 2 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation University (MSU) with International Food Policy Research Institute (IFPRI) as a consortium member. The five-year Activity runs from July 1, 2015 to June 30, 2020. The budget over the life of the project is US$ 12.5 million. The Nigerian Agricultural Policy Activity is implemented by MSU and IFPRI in collaboration with national partners in Nigeria. The goal of the Nigerian Agricultural Policy Activity is to promote inclusive agricultural productivity growth, improved nutritional outcomes, and enhanced livelihood resilience for men and women through an improved policy enabling environment. This goal will be achieved through increased national capacity of Nigerians to generate and analyze information and to formulate evidence-based policy options for considerations by decision makers and stakeholders, and to improve policy processes that are driven by empirical evidence and based on active dialogue among relevant stakeholders. The Nigerian Agricultural Policy Activity has three principal objectives: 1. To strengthen the national capacity for greater evidence based policy processes in agriculture by increasing the capacity of Nigerian analysts to undertake and make widely available relevant evidence-based policy analysis. 2. To promote and foster informed policy dialogue among all stakeholders in the agricultural sector through an inclusive, transparent, and sustainable process at the country level, building blocks for a well-integrated and developed national policy system. 3. To help federal and state governments improve their capacities to plan and implement effective policy analyses and programs, and demand and absorb policy research in their policy process. At the end of this Activity, USAID/Nigeria expects the MSU/IFPRI to achieve the following: ▪ Increased national capacity to generate and analyze information, and to formulate evidence-based policy options for considerations by decision makers and stakeholders; and ▪ Improved policy processes driven by strong empirical evidence and active dialogue among relevant stakeholders. Intended Results Within the USAID Country Development Cooperation Strategy, FTF Agricultural Policy Activity is located under Development Objective (DO) 1, Broadened and inclusive growth, Intermediate Result (IR) 1.2: Business Environment Improved, Sub-IRs 1.2.1: Legal and Regulatory Environment Improved. Please see Figure 1 below. 3 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Figure 1: USAID/Nigeria Economic Growth Results Framework (2015 -2019 CDCS) III. RESULTS FRAMEWORK The overall results framework for the FTF Innovation Lab for Food Security Policy (FSP) project and the USAID Nigeria’s CDCS provide the model for the results framework for the Nigerian Agricultural Policy Activity. The FSP results framework is itself consistent with that of FtF, the U.S. government’s global hunger and food security initiative. The results framework adapted for the Activity is illustrated in Figure 2. It shows the pathways by which the project will achieve its goal of promoting more inclusive private-sector led agricultural growth (which is FTF strategy’s First Level Objective) and poverty reduction (which is the goal of CDCS). This framework will be central to the management, monitoring, and evaluation of this project. The various tiers of the results framework are aligned with the overall objectives of the FTF initiative as well as that of CDCS. The goal: DO and the IRs that appear in the red font are linked with the CDCS of USAID/Nigeria. The two sub-intermediate results (Sub-IR 1.3 and 1.1, represented by boxes with solid black lines) are aligned with the FTF initiative. These Sub-IRs are also linked with CDCS’s Sub-IRs 1.1.1 (increased agricultural productivity), Sub-IR 1.1.2 (improved trade and transportation), Sub-IR 1.1.3 (increased resiliency of vulnerable households and communities), Sub-IR 1.2.1 (improved legal and regulatory environment), Sub-IR 1.2.2 (improved access to finance/credit by farmers), Sub-IR 3.2.1 (strengthened management capacity and transparency of government authorities) and Sub-IR 3.2.2 (strengthened public financial management and transparency) of CDCS. The Nigerian Agricultural Policy Activity results framework also includes one custom Sub-sub IR and four strategic results (SR) across the project components that will contribute to the Sub￾IRs. These tiers are arranged to illustrate the causal relationship between planned efforts under the Activity and their expected impact and to identify the IRs critical to achieving the objectives IR 1.3: Energy Access Increased SIR 1.3.1: Power generation increased SIR 1.3.2 Reliability and reach of transmission network improved IR 1.2: Business Environment Improved SIR 1.2.1: Legal & Regulatory Environment Improved SIR 1.2.2: Access to Finance Improved DO1: Broadened and Inclusive Growth IR 1.1: Agricultural Competitiveness Increased SIR 1.1.1: Agriculture Productivity Increased SIR 1.1.2: Trade & Transport Improved SIR 1.1.3: Resiliency of Vulnerable Households and Communities Increased 4 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation of the Activity. Specifically, the framework demonstrates how planned activities and deliverables will lead to expected outputs, outcomes, results, and eventual impact. Together the DOs, IRs, Sub-IRs and SRs identified in Figure 2 provide the framework for identification and implementation of activities designed to achieve the required results. 5 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Figure 2: Results Framework for the Project 6 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation IV. PROJECT ACTIVITIES AND DEVELOPMENT CONTEXT The Nigerian Agricultural Policy Activity plans to accomplish the SRs, IRs and activity objectives through an activity work program organized around three themes – Capacity building, Research and Outreach. Examples of types of activities to be funded by the project include: 1) Skill enhancing training of staff from Nigerian partner institutions on methods and applications for research and policy analysis. The content of this training was developed in consultations with FMARD, Ahmadu Bello University (ABU), the University of Ibadan, and other key stakeholders to ensure their appropriateness in addressing the most serious gaps in knowledge, analytical and research approaches, and application tools in Nigeria. 2) Organizing and instituting a direct training collaboration between researchers at MSU and IFPRI with graduate student researchers and their supervisors in Nigerian Universities. A competitive annual selection of two or three Nigerian Post Graduate Students (one at the Masters Level and one or two at the PhD Level) was used to select graduate students for this program. These students and their research advisors form a research team with MSU and IFPRI researchers on the proposed research topics. Where possible (and to be encouraged) the research project will constitute part of the students’ thesis or dissertation. The Nigerian students attend technical courses (e.g. econometrics, research design and/or modelling) at MSU for periods of a semester for Masters and one year for PhD students. Periodic team meetings were scheduled to discuss team findings and progress and facilitate more learning by junior researchers. 3) Short term visits to the U.S by the Nigerian research supervisors to enable them to participate in various activities to broaden their outlook and to facilitate the provision of feedback for their research students. The visits also facilitate the interaction of the Nigerian professors with faculty at the department of Agricultural, Food and Resource Economics as well as faculty at the College of Agriculture and Natural Resources to broaden their outlook as well. The Nigerian professors will also be encouraged to collaborate with senior IFPRI researchers based in Washington, D.C. 4) Based on demand, organize periodic training courses at Nigerian universities on various research methods and tools to reach a broader set of promising graduate students and young research professionals. Lecturers and students in Nigerian universities will be able to access all course materials developed. Additionally, as part of this, periodic training, lectures by visiting MSU and IFPRI staff will be encouraged at the program’s collaborating institutions. This is a key long-term oriented effort at institutional capacity building to facilitate the training of the next generation of policy researchers and analysts in Nigeria. The training courses pay attention to policy analysis methods and tools, targeting a broader audience that involves not only the researcher community, but also includes practitioners in government (such as at FMARD and other ministries related to agriculture), Civil Society Organizations (CSOs), Non-governmental Organizations (NGOs), private sector, and think tanks. 7 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation 5) Organize major conferences to promote the visibility of the policy research and analysis being undertaken, based on project workshops and other activities. MSU and IFPRI propose to organize three major conferences during the period (2015-2020) which will bring together experts on selected issues in agriculture and economic development to provide guidance to Nigerian graduate students and young research professionals working in these areas. In addition, selected graduate students will be funded by the program to attend a national/regional conference and present their work. Such conference participation will broaden the reach of these conferences and increase interaction between actors at various institutions interested in agricultural policy. The interactions are intended to strengthen the links between such actors and foster the development of a network of institutions that can independently interact during the policy process. 6) Establishment of formal ties with two or more Nigerian institutions in order to develop appropriate skills, knowledge and tools needed to strengthen the credibility and relevance of evidence generated by policy analysis and research for informing policy. In addition to two already identified institutions (ABU and University of Ibadan) additional institutions have been selected in line with FTF focus States and their interest and/or relevance to project research topics and in order to broaden the project’s interaction with institutions across Nigeria’s diverse agro-ecological and/or geopolitical systems. Specific activities being undertaken by the Activity under the themes outlined above are necessary to create the appropriate enabling environment to achieve the intermediate and sub-IRs as well as the overall goal of the project. The Nigerian Agricultural Policy Activity was developed based on the following development challenges: • Need and support for evidence-based policy and program actions to address the challenges of agricultural transformation and forge long-term solutions to reducing poverty in rural areas and chronic food insecurity. • Need for improving policies and increasing public and private investments to achieve food security goal through agricultural and economic growth. • Need to strengthen policy institutions and processes through coordinated actions that support greater engagement and participation by civil society and private sector participation in policy systems. • The need to have impact at scale. There were a few critical assumptions for the Nigerian Agricultural Policy Activity to achieve its targets and objectives. One is that the policy making environment in Nigeria remains relatively stable and is not subjected to major upheavals often associated with shifts in political power resulting from undemocratic forces. Second assumption is that the government, policy makers and stakeholders targeted by the project are receptive to new ideas and responsive to evidences presented and committed to implementing and enforcing the outcomes of policy research and analysis. In other words, the Results Framework relies on there being political will and support for the policy change agenda and that the project will receive cooperation and participation of targeted stakeholders in program activities and its outputs. 8 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation While the above factors pose serious challenges, every effort will be made to achieve the mission of the project, which is to inform the process through rigorous and timely evidence building and help improve local capacity to use this evidence for policy analysis and advocacy. Existing Data The Evaluation Team will have access to the vital documents relevant to conducting this evaluation. These documents will include the technical proposal, original contract, monitoring and evaluation plan, annual work plans, quarterly and annual reports, data quality reports, contract amendments (if any), memorandums of understanding, and any other relevant materials documenting the management, implementation process and results for the Nigeria Agricultural Policy Activity permitted by the Office of Acquisition and Assistance. V. EVALUATION RATIONALE Evaluation Purpose The primary purpose of this mid-term performance evaluation is to determine whether the assistance provided by USAID/Nigeria through the Nigeria Agricultural Policy Activity is meeting the stated development objectives, and to understand the lessons learned from this particular Activity in Nigeria. It is intended to provide an independent examination of the overall progress and accomplishments of the Nigeria Agricultural Policy Activity. The evaluation should provide a detailed picture of the major accomplishments and weaknesses of the Activity since its inception and determine how its successes can be sustained. This mid-term performance evaluation will provide USAID/Nigeria, its implementing partners, and agriculture sector stakeholders with data on outcomes and progress toward achieving its objectives alongside answering fully all of the evaluation questions, and provide recommendations on how to adjust project interventions during the remaining two and half years of the Activity in order to accomplish its objectives. The evaluation should also elucidate the lessons that USAID should learn from this Activity in terms of activity design and management. The evaluation should include specific recommendations to USAID/Nigeria about how the Activity interventions can be sustained and scaled by other FTF activities, partners, private sector firms, or by the Government of Nigeria (GON). Audience and Intended Users The primary users of the evaluation findings are the USAID Economic Growth and Environment (EGE) Office, other Mission colleagues, and other U.S. Government (USG) officials. Other audiences include non-USG donor organizations, the GON, and private sector firms, all of whom can use the findings to improve and build knowledge. The evaluation report will also serve as a reference material for the future program design (see Table 1 below). 9 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Table 1: Audience and Intended Uses and Users # Intended Use Target Audience USAID/ Nigeria FtF Feedback/ Other USG Implementing Partners GON /Other Dev. Partners Research Communities 1 Inform Policy ✓ ✓ ✓ 2 Inform Project design ✓ ✓ ✓ ✓ ✓ 3 Improve project monitoring ✓ ✓ ✓ ✓ ✓ 4 Improve operational policy and planning ✓ ✓ ✓ ✓ 5 Improve resource management ✓ ✓ ✓ ✓ 6 Enhance professional growth ✓ ✓ ✓ ✓ ✓ 7 Add scientific Knowledge ✓ Evaluation Questions The evaluation will respond to the following critical questions posed by USAID/Nigeria. 1. Is the Nigeria Agricultural Policy Activity implemented as intended according to the Activity design? a. Have the methods of delivery contributed to ensuring the Activity is on track to achieving its annual and life of project targets? b. Are the intended results per the Activity design occurring? c. Is the Activity working on any policy issues? If yes, how critical and necessary are the policy issues chosen by the Activity in transforming the agriculture sector in Nigeria? d. How appropriate is the Activity’s capacity building of policy actors and other stakeholders? To what extent is the Activity engaged in capacity building taking into consideration the audience, methods, and effects of the capacity building? e. How does the Activity intend the capacity building to influence policy via policy actors and other stakeholders? f. On what evidence are the policies based? g. What should the Activity put in place to ensure that the end-line intended impact of the Activity is on track and achieved? 2. Are key stakeholders (GON, think tanks, civil society organizations, community-based organizations, associations and media) satisfied with the capacity building activities implemented by the Activity? Why or why not? a. How well domesticated is the Agricultural Policy Activity in the states of Nigeria? 10 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation b. To what extent are the state-level policy actors aware of national/federal initiatives and aligning state policies accordingly? c. To what extent has the Activity engaged multiple stakeholder groups (value chain actors, farmers, private sector, policy actors, think tanks, CSOs, etc.) to achieve its intended outcomes? 3. To what extent has the Activity coordinated with other USAID/Nigeria FTF activities and other donor programs to achieve the overall strategic objectives of improving the enabling environment in the agricultural sector for private sector investment? a. What are the most critical constraints in the formulation and implementation of agricultural policies in Nigeria, and how is the Activity responding to these constraints? b. How responsive has the Activity been to the changing needs of the policy environment and seizing opportunities for impact? c. To what extent has the Activity fostered policy dialogue with and/or among USAID/Nigeria’s FTF activities and other donor programs? 4. What methodologies does the research employ and what is the value of the research, conducted by the Activity, to the stakeholders? a. What are the benefits of the Activity’s research to policy actors and stakeholders? Is it reaching them, if yes, how? If not, why not? b. How do policy actors and stakeholders use the Activity’s research? How do policy actors and stakeholders assess the utility of the Activity’s research? c. How knowledgeable are policy actors and stakeholders about the Activity’s different types of research products? d. How is the Activity’s research influencing and/or informing policy actors, private sector and other value chain actors? e. What are the dissemination plans for the different types of research products that the Activity produces? 5. What measures is the Activity putting in place to ensure sustainability of the interventions? VI. EVALUATION METHOD AND METHODLOGY Evaluation Design The Evaluation Team will use a mixed methods approach to complete a thorough and effective mid-term performance evaluation within the timeline and budget parameters set in the contract. The evaluation team should consider a range of possible methods and approaches for collecting and analyzing the information required to answer the evaluation questions. The methodology should include, but not be limited to, the following techniques to conduct the evaluation. Prior to arriving in country and conducting field work, the team should submit to USAID/Nigeria three key deliverables: 1) Document Review Summary, 2) Methodology / Detailed Work Plan, 3) Site 11 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Visit Schedule and plan. USAID/Nigeria will review these three deliverables, turning them around within two work‐weeks to the evaluation team. • Document Review/Data Analysis. Prior to arriving in country and conducting field work, team members will review various documents and reports, including but not limited to the Livelihoods original agreement and amendments; USAID/Nigeria strategy document; FtF Project Appraisal Document; activity quarterly and annual reports; activity technical studies; and other relevant documents such as the specification of the activity deliverables. USAID/Nigeria and IFPRI/MSU will provide the relevant documents. • Key Informant Interviews (KIIs). The team will conduct interviews and focus groups with a variety of stakeholders including USAID staff, project beneficiaries, partner banks, government staff, implementing partner staff, and other key donor partners. A list of stakeholders and contacts will be provided by USAID/Nigeria and IFPRI/MSU, and additional individuals may be identified by the evaluation team at any point during the evaluation prior to the drafting of the final report. • Site Visits. In addition to the many KIIs that will take place in Abuja, the Evaluation Team will visit activity sites to interview beneficiary communities, service providers, private sector entities engaging with beneficiaries, and federal, state, and local government authorities as indicated. The Evaluation Team will apply robust data analysis techniques including triangulation to draw conclusions and recommendations linked to key findings that are based on strong evidence. The Evaluation Team will compare data collected from KIIs with results from the Activity documents and reports. USAID staff from the EGE Office and from other Mission offices may accompany the Evaluation Team as needed. VII. EVALUATION PRODUCTS Deliverables The Evaluation Team will be responsible for producing the following deliverables: • Document Review Summary and proposed outline of draft evaluation report (either prior to the field or at the end of first week of interviewing). • Detailed Work Plan detailing evaluation methodological approach and draft schedule of field activities and draft and final questionnaire (s) to be used during interviews (prior to travel to the field). • Summary of initial findings, including a PowerPoint Briefing (at the end of follow‐up and synthesis effort and prior to Team’s departure from Nigeria). • Draft Evaluation Report. • A summary of "key steps in the evaluation, key findings, recommendations and next steps" and a PowerPoint presentation of these. 12 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation • Final Evaluation Report (within two weeks of receiving Mission comments on draft report). All reports are to be submitted in English in both electronic and hard copies. The Team will provide five printed copies of the Final Evaluation Reports. The consultants will be responsible for report production. The Final Evaluation Report should not exceed 30 pages in length in its body, not including title page; Table of Contents; List of Acronyms; usage of space for tables, graphs, charts, or pictures; and/ or any material deemed important and included as Annexes. The Final Evaluation Report and PowerPoint addressing the Mission's comments should be submitted in both Word and PDF formats. Once the PDF format has been approved by the Mission, the Team will submit the Final Evaluation Report to the Development Experience Clearinghouse for archiving. Reports should be submitted consistent with the Automated Directives System (ADS) 579. Reporting Guidelines USAID’s evaluation policy requires that all evaluation SOWs include USAID’s Criteria to Ensure the Quality of the Evaluation Report (USAID Evaluation Policy, 2011). The policy also indicates that the report will outline in detail, any additional expectations USAID has regarding a report’s structure, format, and length. The format for the evaluation report is as follows (number of pages is illustrative): • Executive Summary (2 pp.) • Table of Contents (1 pp.) • Introduction (1 pp.) • Background (2-3 pp.) • Methodology (1 pp.) • Findings/Conclusions (17- 20 pp.) • Issues and Challenges (1-2 pp.) • Recommendations/Future Directions (5-10 pp.) • References • Annexes The draft report will be submitted electronically. The final report will be submitted in five hard copies and one electronic copy. All reports will be in English. The report must: • Distinguish clearly between findings, conclusions (based strictly on findings) and recommendations (based clearly on the evaluation findings and conclusions); • Comply with USAID’s Evaluation Policy • Ensure submission to the DEC after finalization. Dissemination Plan 13 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation It is expected that USAID/Nigeria will plan a dissemination session with other technical units within the mission, implementing partners, donor community as well other stakeholders. VIII. TEAM COMPOSITION The Evaluation Team will consist of four key members, including at least one (1) international consultant (who will lead the team) and local consultants. The team members should represent a balance of several types of knowledge and expertise related to agricultural policy programming and agriculture-led development. USAID/Nigeria recommends the following staffing structure for the evaluation: Team Leader/Senior Evaluation Specialist (International): The Team Leader will be responsible for the overall management of the evaluation, including coordinating and packaging the deliverables. The Team Leader will develop the outline for the draft report, present the report, and after incorporating USAID/Nigeria staff comments, submit the final report to USAID/Nigeria through DevTech Systems Inc within the prescribed timeline. In addition to the responsibilities outlined above, the Team Leader will: Preparations 1. Finalize and negotiate with USAID/Nigeria the team’s work plan. 2. Establish assignment roles, responsibilities, and tasks for each team member. 3. Ensure that the logistics arrangements in the field are complete. Management 1. Take the lead on preparing, coordinating team member input, submitting, revising and finalizing the assignment report. 2. Manage the process of report writing. 3. Manage team coordination meetings in the field. 4. Coordinate the workflow and tasks and ensure that team members are adhering to the schedule. Communication 1. Ensure harmony within the team. 2. Serve as primary interface with USAID/Nigeria and serve as the spokesperson for the team. 3. Debrief USAID/Nigeria as the evaluation progresses and organize a final debriefing. 4. Keep USAID/Nigeria apprised of progress challenges, work changes, team travel plans in the field, and report preparation via phone conversation or email at least once a week. 5. Serve as primary interface with USAID/Nigeria for the submission of draft and final reports, and deliverables to USAID/Nigeria. 6. Make decisions in conjunction with USAID/Nigeria about the safety and security of the team. 14 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Direction Assume technical direction lead, as required, in order to ensure quality and appropriateness of assignment and report content. Qualifications for the Team Leader: • A graduate degree in Development Economics, Public Policy, Economics, Business Administration, Agriculture, or a related field. • At least ten years of experience assessing or evaluating USAID-supported agricultural policy and agricultural value chain activities. • Previous experience serving as an evaluation Team Leader on a USAID-supported agricultural policy activity. • Previous experience working in Africa. • Experience facilitating and providing leadership in collaborative and participatory evaluations with multiple stakeholders. • Excellent writing, communication and presentation skills. • The ability to produce preliminary and final reports on time. Other team members will include: 1. A host-country national or international Senior Evaluation Specialist: The Senior Evaluation Specialist must have at least a Master's Degree or equivalent in Development Economics, Public Policy, Public Administration, Business Administration, Social Sciences, or related fields. S/he must have a minimum of seven (7) years conducting both qualitative and quantitative evaluations/assessments around improving capacity for service delivery and policy formulation in developing countries. S/he must demonstrate good interpersonal and diplomatic skills as well as strong report writing and analytical skills and excellent oral communication skills in English 2. A host country mid-level Agriculture Economist: The Agriculture Economist must possess technical competence in the field of Agriculture, Agricultural Policy, Agribusiness and Agriculture value chains, Economics, or related fields. S/he must have a minimum of five (5) years conducting both qualitative and quantitative evaluations with strong report writing and analytical skills. S/he must demonstrate strong knowledge of the Nigerian agricultural policy environment. 3. A national Evaluation Specialist with at least five (5) years relevant experience and strong logistics and planning skills. S/he must demonstrate experience in organizational capacity building/assessment required. S/he must demonstrate strong experience with data collection procedures, surveys, and analysis of data using statistical analysis tools. 15 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation USAID leaves to DevTech’s discretion other necessary team members/staff for the evaluation (e.g. logistics, scheduling and translation, data analysis). Aside from the above mentioned key personnel, the offeror must decide how the evaluation team should be structured in order to successfully address the evaluation questions. All attempts should be made for the team to be gender balanced and to include local (Nigerian) experts. A statement of potential bias or conflict of interest (or lack thereof) is required from each team member. USAID may propose internal staff from USAID/Nigeria or from Washington to accompany the team in this evaluation as observers. As observers, their role will be to provide guidance and background information, and to reply to the external evaluators’ questions. They will review and comment on the report for accuracy, but evaluators may accept or reject comments. The final report should reflect the opinions of the external evaluators and is the sole responsibility of the selected Evaluation Team. Conflicts of Interest All evaluation team members will provide a signed statement attesting to a lack of conflicts of interest or describing an existing conflict of interest relative to the activity being evaluated. USAID/Nigeria will provide the conflict of interest forms. IX. SCHEDULING AND LOGISTICS Responsibilities The funding source will be through the budget of the USAID/Nigeria Monitoring, Evaluation, and Learning (MEL) Activity. The EGE Office of USAID/Nigeria contributes to the budget of the MEL Activity. The MEL Activity /DevTech will be responsible for all offshore and in￾country administrative and logistical support, including identification and fielding appropriate consultants. DevTech will arrange and schedule meetings, international and local travel, hotel bookings, working/office spaces, computers, printing and photocopying. DevTech will make all logistical arrangements, including the vehicles for travel throughout Nigeria, and should not expect any logistical support from the Mission. DevTech will also make their own arrangements about space for team meetings and equipment support for producing the report. USAID/Nigeria will provide: • Background documents to be provided to the evaluation team as early as possible prior to work on the evaluation, but at least two weeks before the start of the evaluation; • A list of key informants, institutions, organizations, and other stakeholders; • Ensure constant availability of Mission Point of Contact to provide technical leadership and direction for the evaluation team’s work; • Assistance with arrangements/letters of introduction for formal and official meetings, and where necessary for high‐level meetings, will accompany teams on introductory interviews; and, but not limited to; and • If necessary and deemed appropriate, assist in identifying and helping set up meetings with local development partners relevant to the assignment. Performance Period 16 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation The period of performance will run approximately 12 weeks, starting on or about May 2018 depending on the availability of consultants and available funding. USAID/Nigeria will designate a technical representative to work in coordination with the Evaluation Team; however, the Team Leader will have the primary responsibility for ensuring the final deliverables are completed in a timely manner and are responsive to the scope of work and Mission comments. Timeline and LOE for the USAID/Nigeria Agricultural Policy Activity Mid-Term Evaluation Item Period of Performance Number of Days International Team Leader Senior Evaluation Specialist - Agricultural Policy Agriculture Economist Local Evaluation Specialist Recruitment international evaluation and local team members and obtaining travel concurrence from USAID (OAA) March – May 2018 Desk Review of background documents and initial preparation work towards draft data collection tools, sampling, and workplan June 6 – 12 5 5 5 5 Preparation and submission of draft workplan & draft data collection tools (protocols) June 13 – 15 3 3 3 3 Submission of Workplan and evaluation design /protocols June 18 0 0 0 0 USAID provides feedback on work plan and evaluation design/ protocol June 20 – July 3 0 0 0 0 In-country arrival (Abuja Nigeria) for international consultants July 7 – July 8 2 2 0 0 In brief/Team Preparation Meeting with USAID July 9 -July 10 2 2 2 2 Reviewing evaluation tools to incorporate USAID comments July 11 – July 12 2 2 2 2 Travel day for field work July 13 1 1 1 1 17 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Field visits and Data Collection July 14 – August 3 18 18 18 18 Analysis of results, Report writing and preparation of ppt for USAID debrief Aug 4 – Aug 10 6 6 5 5 Debrief with USAID (preliminary findings) Aug 13 1 1 1 1 Out-country travel (for International Consultants) Aug 13 – 14 2 2 0 0 Finalizing evaluation report Aug 15 – Aug 22 7 6 6 6 Submission of draft report (to MEL) August 22 0 0 0 0 Submission of draft report (to USAID) August 27 0 0 0 0 USAID reviews and comments on final draft evaluation report August 27 – Sept 7 0 0 0 0 Submission of report back to MEL with USAID feedback Sept 10 0 0 0 0 Team addresses USAID comments and finalizes the report Sept 11 – Sept 17 6 4 3 3 Team submits reviewed report to MEL for additional review and send-off for copy editing Sept 18-–25 0 0 0 0 Submit 2-3 page summary Sept 26 1 Submission of final report to USAID Sept 28 TOTAL LOE 55 52 46 46 Title LOE/Days LOE/Hours International Team Lead 55 440 Senior Evaluation Specialist 52 416 Agriculture Economist 46 368 Evaluation Specialist 46 368 18 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation ANNEX 2: Discussion of Nigeria Agricultural Policy Program (NAPP) Performance Using AMEL Plan Indicators Evaluation of NAPP Performance Using MEP Indicators This aspect of the evaluation employs the NAPP performance indicators gleaned from the available project reports to examine the extent to which the project meets the specified targets in the execution of the various projects. A review of the monitoring and evaluation plan (MEP) shows that the specified performance indicators are relevant to a reasonable extent although they may not all be as SMART as expected. As gleaned from the quarterly and annual reports some of the indicators are not defined in the same manner from year to year as shown in Table 1. In this analysis we present highlights of the findings, conclusions and recommendations based on the quarterly and annual reports reviewed as part of this mid-term evaluation. (a) Main Findings 1. Varying definition of research output (no. 1 of FY1, FY2, FY3). From FY0 to FY2 the persistently used indicator is “number of policy research and best practice papers generated”. However, in FY3 this has changed to “number of high-quality research reports published”. In actual fact, this appears to be a better indicator and going forward it should be maintained to capture progress made in terms of measuring research output performance. 2. Weakness of indicator (Weak indicator (strengthened with time) no. 6 of FY 1 and FY2 compared with no. 4 of FY3) 3. Lack of specificity of indicator - Weak or indicator that lacks specificity - (no. 4 of FY 2) Number of formal, informal or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed. (Custom). A strong or better indicator that captures the essence of discussions of output and findings is already there in PY3 no. 8 - Number of agriculture policy communications developed and/or written for stakeholder consumption. 4. Instability of indicator - An indicator may appear once or twice and disappears thereafter. (e.g. 2) Number of stakeholder learning forums (national or global) held where findings/best practices are presented which has disappeared as at FY3. 5. Issue of Relevance – Sudden appearance of indicator with doubtful relevance (no. 10 PY3). Number of for-profit private enterprises, producers’ organizations, water users’ associations, women's groups, trade and business associations, and community-based organizations (CBOs) receiving USG food security related organizational development assistance 19 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation (b) Conclusions (i) To a large extent, the project was meeting its targets especially during the first two project years based on the activities implemented (Table 4.1) and the level of achievements (Table 4.2) (ii) Nonetheless the performance indicators are far from being SMART (iii) The quarterly assessment of activities is not based on strict definition of indicators thus making it difficult to keep track of progress made using indicators specified in the initial MEP. This is exemplified by the proliferation of indicators (essentially in the form of varying definitions) such that instead of the often mentioned 13 indicators, it is easy to find the number of indicators rising from time to time reaching an all-time high of 17 indicators at PY3 as shown in Table 2. (c) Recommendations (i) Retain only SMART indicators (ii) Stop undue proliferation of indicators and avoid shifting of goal posts (iii) Be consistent in the use and definition of indicators (iv) Discontinue indicators that are totally irrelevant (e.g. Number of for-profit private enterprises, producers’ organizations, water users’ associations, women's groups, trade and business associations, and community-based organizations (CBOs) receiving USG food security related organizational development assistance) (v) Remove redundant indicator such as the “Number of formal, informal or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed. 20 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Nature of NAPP Indicators S/N YEAR 0 (2015) YEAR (2016) YEAR 2 (2017) YEAR 3 (2018) 1 1) Number of policy research and best practice papers generated. 1) Number of policy research and best practice papers generated. 1) Number of policy research and best practice papers generated. (Custom) 1. Number of high-quality research reports published 2 2) Number of stakeholder learning forums held where findings/best practices are presented 2) Number of stakeholder learning forums (national or global) held where findings/best practices are presented. 2) Number of stakeholder learning forums (national or global) held where findings/best practices are presented. (Custom) 2. Number of participants attending project organized research and policy events 3 3) Number of collaborative research teams involving local partners formed and undertaking policy research and analysis 3) Number of participants attending project organized events 3) Number of participants attending project organized events. (Custom) 300 Number of agricultural and nutritional enabling environment policies analyzed, consulted on, drafted or revised, approved and implemented with USG assistance 4 4) Number of technical training courses offered to build technical skills and capacity for policy analysis 4) Number of formal, informal or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed. 4) Number of formal, informal or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed. (Custom) 60 4. Number of collaborative policy research work and analysis completed jointly with local partner. 5 5) Number of individuals who have received NAPP supported short￾term technical training in agricultural sector productivity or food security policy analysis 5) Number of agricultural and nutritional enabling environment policies completing the following processes/steps of development as a result of USG assistance in each case: FTF 4.5.1(24): 5) Number of agricultural and nutritional enabling environment policies completing the following processes/steps of development as a result of United State Government assistance (USG) in each case. 5. Number of individuals who have received USG supported short￾term technical training in agricultural sector productivity or food security policy analysis training. 6 6) Number of Nigerian graduate students who have received support for skill enhancement training in the US through NAPP program 6) Number of collaborative research teams involving local partners formed and undertaking policy research and analysis. 6) Number of collaborative research teams involving local partners formed and undertaking policy research and analysis. (Custom) 6. Number of individuals who have received USG supported degree￾granting agricultural sector productivity or food security training. 7 7) Number of technical training courses offered to build technical skills and capacity for policy analysis 7) Number of technical training courses offered to build technical skills and capacity for policy analysis. (Custom) 7. Number of government units or divisions that have received short￾term training (USG assistance) 8 8) Number of individuals who have received USG supported short-term 8) Number of individuals who have received USG supported short-term 8. Number of agriculture policy communications developed and/or 21 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation S/N YEAR 0 (2015) YEAR (2016) YEAR 2 (2017) YEAR 3 (2018) 9technical training in agricultural s0ector productivity or food security policy analysis training. FTF 4.5.2(7) technical training in agricultural sector productivity or food security policy analysis training. Standard FtF Indicator EG.3.2-1 written for stakeholder consumption 9 9) Number of individuals who have received USG supported long-term agricultural sector productivity or food security training FTF 4.5.2(6) 9) Number of individuals who have received USG supported degree￾granting agricultural sector productivity or food security training Standard FtF Indicator EG.3.2-2 9. Number of public private advocacy dialogues focused on policy that supports private sector investment 10 10) Number of Nigerian graduate students who have received support for skill enhancement training in the US through the project. 10) Number of Nigerian graduate students who have received support for skill enhancement training in the US through the project. (Custom) 10. Number of for-profit private enterprises, producers’ organizations, water users’ associations, women's groups, trade and business associations, and community-based organizations (CBOs) receiving USG food security related organizational development assistance 11 11) Number of organizations/entities assisted to participate in various roundtable and stakeholder meetings, seminars and conferences focused on agricultural policy issues. 11) Number of organizations/entities assisted to participate in various roundtable and stakeholder meetings, seminars and conferences focused on agricultural policy issues. (Custom) 11. Index (or scorecard) of quality of agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation to capture level of satisfaction and confidence 12 12) Index (or scorecard) of quality of agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation to capture level of satisfaction and confidence. Baseline date: 2016 12. Index (or scorecard) of quality of the institutional architecture for agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation survey to capture level of satisfaction and confidence 13 13) Index (or scorecard) of quality of the institutional architecture for agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation survey to capture level of satisfaction and confidence. Baseline date: 2016. 22 Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Source: Consultants’ compilation from quarterly and annual Activity reports. Feed the Future Nigeria Agricultural Policy Project Mid-Term Performance Evaluation Analysis of NAPP Implementation Performance Indicators PY1 OCT 2015 – SEPT 2016 PY2 OCT 2016 – SEPT 2017 PY3 OCT 2017 – SEPT 2018 Planned Actual Deviation Planned Actual Deviation Planned Actual Deviation Number of high-quality research reports published 7 11 57 8 29 263 11 5+1+3+7(1 6) 45.45 Number of participants attending project organized research and policy events 200 696 248 300 629 110 765 438+37+ 186+613* (1274) 66.54 Number of agricultural and nutritional enabling environment policies analyzed, consulted on, drafted or revised, approved and implemented with USG assistance 7 2 -71 7 2 -71 3 0+0+0 +3(3) 0 Number of collaborative policy research work and analysis completed jointly with local partner 10 10 0 9 10 11 7 3+0+4+5(1 2) 71.43 Number of individuals who have received USG supported short-term technical training in agricultural sector productivity or food security policy analysis training 100 334 234 200 921 361 850 187+374+ 310+214(1 085) 27.65 Number of individuals who have received USG supported degree-granting agricultural sector productivity or food security training. 1 1 0 1 5 400 8 2+3+0+2(7 ) -12.50 *****Number of government units or divisions that have received short-term training (USG assistance) 11 1+9+2+1(1 3) 18.18 *****Number of agriculture policy communications developed and/or written for stakeholder consumption 18 9+8+15+1 7(49) 172.22 *****Number of public private advocacy dialogues focused on policy that supports private sector investment 4 0+0+0+3(3 ) -0.25 *****Number of for-profit private enterprises, producers’ organizations, water users’ associations, women's groups, trade and business associations, and community-based organizations (CBOs) receiving USG food security related organizational development assistance 30 2+12+0+2 7(41) 36.67 Indicators PY1 OCT 2015 – SEPT 2016 PY2 OCT 2016 – SEPT 2017 PY3 OCT 2017 – SEPT 2018 Planned Actual Deviation Planned Actual Deviation Planned Actual Deviation ****Index (or scorecard) of quality of agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation to capture level of satisfaction and confidence na na 1.33 1.44 8.27 ****Index (or scorecard) of quality of the institutional architecture for agriculture and food security policy processes in Nigeria, as measured by stakeholder evaluation survey to capture level of satisfaction and confidence na na 1.48 1.74 17.56 ******Number of formal, informal or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed Number of Nigerian graduate students who have received support for skill enhancement training in the US through the project 2 2 0 5 5 0 *****Number of organizations/entities assisted to participate in various roundtable and stakeholder meetings, seminars and conferences focused on agricultural policy issues. 10 14 40 10 85 750 *****Number of stakeholder learning forums (national or global) where findings/best practices are presented 10 13 30 10 37 270 *****Number of formal, informal or ad hoc meetings held with key government or private sector partners where project outputs and findings are discussed. 60 119 98 Source: Consultants’ Computation using data from quarterly and annual Activity reports *In the version of the Fourth Quarter Report of PY3 (FY2017/2018) made available for the purpose of this evaluation, a figure of 50,613 appeared under this column. This is unlikely to be correct based on our findings from a careful review of figures for previous years. This error must be corrected in a revised version of the report. **This takes into account extra capacity building initiatives requested by ARCN ***These have been planned in Q4 in line with dissemination of Activity research results ****Expressed as annual indicator nonetheless the score for PY1 and PY2 are not available *****Irregularly tracked indicator ******Redundant indicator ANNEX 3: Quantitative Breakdown of the Capacity Building Respondents STAKEHOLDERS (N=282) Think Tank/NGO 2% Private 4% Media 10% Government 29% Academia 55% BREAKDOWN OF TRAININGS Stakeholder Software Policy Communication M&E Media (28) 0% 29% 79% 0% Think tank/NGOs (6) 17% 83% 0% 0% Government (82) 23% 38% 29% 43% Private sector (11) 73% 36% 9% 18% Academia (155) 80% 14% 14% 12% All (282) 54% 25% 24% 20% Figure A2–1. What was your experience with the Project’s capacity building activities? Figure A2–2. Did you learn any new skill from the trainings? (Answered “yes”) Figure A2–3. Are the skills applicable to your work or education? (Answered “yes”) Figure A2–4. What did you gain from the capacity building workshop? Figure A2–5. How useful were the trainings? Comments about the Capacity Building Trainings (if answered “Not Useful”) Stakeholder1: Academia "Presently not useful but during my project it was useful" "I don't use it but knowing it is an added advantage to me. Where I work we use NOMIS because of the nature" "not working at the moment" "It is not useful in what I do now " "She's not working at the moment " Stakeholder 2: Government "It is not useful in my line of work only in communicating with my colleagues " "Maybe useful in future but not now " Stakeholder 3: Private Sector "Not relevant to her field " "I do not apply it to my work " Figure A2–6. How relevant were the trainings to the Agriculture policy process In Nigeria? Figure A2–7. Impact of the Agriculture Policy Project on your career Figure A2–8. Impact of the Agriculture Policy Project on policy process from the use of collaborative research ANNEX 4: List of Key Stakeholders Contacted and Interviewed # NAME DESIGNATION FMARD 1. Dr. Andrew Kwasari Senior Technical Adviser to the Honourable Minister of Agriculture 2. Mr. Baye Sylvester Acting Director, Policy Planning and Coordination Unit 3. Mrs. Karima Babangida Director (Extension Services) 4. Dr. Rasaq Oyeleke Assistant Director (Federal Department of Agriculture) SENATE 5. Dr. Aliyu Sabi Abdullahi Chairman of the Senate Committee on Media and Public Affairs and the Senator representing Niger North Senatorial District AGRICULTURAL RESEARCH COUNCIL OF NIGERIA (ARCN) 6. Dr. Nasir Mohammed Planning and Institutional Development 7. Dr. Yarama D. Ndirpaya Agricultural Director CENTRE FOR THE STUDY OF THE ECONOMIES OF AFRICA (CSEA) 8. Dr. Chukwuka Onyekwena Executive Director AGRICULTURAL POLICY RESEARCH NETWORK (APRNET) 9. Dr. Tony Onoja APRNet President NIGERIA ASSOCIATION OF AGRICULTURAL ECONOMISTS (NAAE) 10. Professor Job Nmadu President NAAE ASSOCIATION OF DEANS OF AGRICULTURAL UNIVERSITIES IN NIGERIA (ADAN) 11. Professor Mrs. David￾Abraham ADAN member 12. Professor Mudashiru Bamikole ADAN President IITA 13. Dr. Manson Nwafor Policy Analyst MEDIA 14. Ene Rita Okwanihe Voice of Nigeria 15. John Oba Blueprint 16. Caleb Onwe New Telegraph 17. Philomena Attah News Agency of Nigeria 18. Ogunyemi Kudirat Ronke News Agency of Nigeria 19. Ginika Okoye News Agency of Nigeria UNIVERSITIES 20. Professor Godwin Abu Federal University of Makurdi 21. Professor Idiong University of Calabar 22. Dr. Ovharhe Delta State University # NAME DESIGNATION 23. Professor Umeh Ebonyi State Univeristy 24. Dr. Robert Onyeneke FUNAI 25. Professor Chris Daudu Ahmadu Bello University 26. Professor MK Othman Ahmadu Bello University 27. Dr. Gona Ayuba Kebbi State University of Science and Technology 28. Dr. Abubakar Maikasuwa Kebbi State University of Science and Technology 29. Dr. Micahel Ojo FUT-Minna 30. Dr. Likita Tanko FUT-Minna STATE MINISTRIES OF AGRICULTURE 31. Simon A. Ityo Benue 32. Izobo Lucky Benue 33. Abayol Luper Benue 34. Ikongha Odey Cross River 35. Dr. Chris Ngaji Cross River 36. Ubi Obetan Cross River 37. Stella Ovie Egedi Delta 38. Johnson Nikoro Delta 39. Chukwuma Okereke Ebonyi 40. Okpani Ndukwe Ebonyi 41. Emmanuel Oroke Ebonyi 42. Eze Ogbuagu Ebonyi 43. Yakubu Gorah Kaduna 44. Danjuma B. Tyuka Kaduna 45. Bello Shehu Kebbi 46. Abubakar Lolo Kebbi 47. Mohammed Mussa Niger 48. Alhassan Umar Niger ANNEX 5: List of Key Informants Contacted during the Evaluation Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacted Nov. 30, 2018 and Dec. 3, 2018 Dr. George Mavrotas Chief of Party IFPRI/NAPP Chief of Party 5 Email and in-person Yes Yes Nov. 30, 2018 and Dec. 3, 2018 John Mazunda Program Manager IFPRI/NAPP Program Manager 5 Email and in-person Yes Yes Dec. 3, 2018 Amina Bakubu Yashir Research Assistant IFPRI/NAPP Research Assistant 5 In-person Yes No Dec. 3, 2018 Tutsi Fazorandi M&E Specialist IFPRI/NAPP M&E Specialist 5 In-person Yes Yes Dec. 3, 2018 Dr. Saweda Liverpool￾Tasie Associate Professor MSU/NAPP Principal Investigator 2 Email and phone interview Yes Yes Dec. 3, 2018 Oyinkan Tasie Assistant Professor MSU/NAPP Led studies 2 Email and phone interview Yes Yes Dec 3, 2018 Steve Longabaugh Associate Professor MSU/NAPP Led trainings 2 Email and phone interview Yes Yes Dec 3, 2018 Chukwumaru me Rufina ARCN Communicatio ns Training 1 Email No No Dec 3, 2018 Kabir Abdulaziz ARCN Communicatio ns Training 1 Email No No Dec. 3, 2018 Aminu R. Abdullahi ARCN Communicatio ns Training 1 Email No No Dec. 3, 2018 Muhd .K. Aliyu ARCN Communicatio ns Training 1 Email No No Dec. 3, 2018 Mrs. S. S. Sada ARCN Communicatio ns Training 1 Email No No Dec.3, 2018 Charles Ebijei ARCN Training on Design and Impl. of Policy Reforms 1 Email No No Dec.3, 2018 Abdullahi M. Nasir ARCN Training on Design and Impl. of Policy Reforms 1 Email No No Dec.3, 2018 Fabiyi Alice Olukemi ARCN Training on Design and Impl. of Policy Reforms 1 Email No No Dec.3, 2018 Nwokocha Stella Information Officer ARCN Training on Monitoring 1 Email No No Dec. 4, 2018 and Dec. 14, 2018 Professor M.A. Bamikole President, Association s of Deans of Agricultura l Universitie s in Nigeria (ADAN) University of Benin Key stakeholder 2 Email Yes Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacte d Dec. 4, 2018 and Dec. 14, 2018 Professor Mrs. David Abraham Former President, ADAN Ebonyi State University Key Stakeholder 2 Email No No Dr. Gloria Ujor Agricultura l Policy Research Network (APRNet) representati ve Nigeria Environment al Study Action Team Key Stakeholder 2 Email Yes Yes Dr. Tony Onoja APRNet President Univ. of Benin Key Stakeholder 2 Innocent Ogirinye founder and strategic team leader Youth Initi ative for Sustainable Agriculture 4 Ernest Ogezi Strategic Program Manager Participated in conference Aug. 2018 4 Esther Okpete Communic ations and Capacity Building Manager Youth Initi ative for Sustainable Agriculture 4 Raymond Odey Coordinato r YISA Cooperativ e activities Youth Initi ative for Sustainable Agriculture 4 Dec. 6, 2018 Dr. Adeyinka Onabolu Senior Advisor on Food Security and Nutrition to Minister FMARD Key stakeholder 1 Email and In -person Yes Yes Dec. 11, 2018 Dr. Funso Olajide Professor University of Ibadan Graduate adviser to Wole; Visiting Nigerian Professor 2 Email and phone interview Yes Yes Bolamira Omonamo Professor University of Ibadan Graduate adviser to Charity; Visiting Nigerian Professor Dec. 4, 2018 Ene Rita Okwanihe Journalist Voice of Nigeria Participated in media training 3 Email and in -person Yes Yes Dec 3, 2018 and Dec 10, 2018 Wale Ogunleye MSU Scholar University of Ibadan 2 Email and phone interview Yes Yes Dec. 3, 2018 Charity Ekerebi MSU Scholar 2 Email and phone interview Yes Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacte d Dec 3, 2018 and Dec 10, 2018 Aisha Ibrahim MSU Scholar 2 Email and phone interview N o No Dec.3, 2018 and Dec. 6, 2018 Ufedo Monday Shaibu MSU Scholar Kogi State University 2 E -mail; telephone interview Yes Yes Oluwatoyin Motunrayo Adebowale MSU Scholar 2 Email Yes Yes Dec. 3, 2018 and Dec. 14, 2018 Stella Nwawulu Chiemela MSU Scholar 2 Email Yes Yes Dec. 6, 2018 Ronke Journalist Nigeria News Agency Participated in media training 3 Phone, and in - person interview Yes Yes Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Ekpang Kate David MsC candidate, Faculty Assistant Researcher University of Calabar Beneficiary of STATA training 2 Email, phone and in -person Yes Yes Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Esien -Umo Ekaette PhD candidate University of Calabar Beneficiary of STATA training 2 Email, phone Number switched off No Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Uwah Ime David Student University of Calabar Beneficiary of STATA training 2 Email, phone No answer Yes Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Igiri Juliana .A Assistant Lecturer University of Calabar Beneficiary of STATA training 2 Email, phone Email bounced back, no answer on phone No Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Eno Uwah Lecturer University of Calabar Beneficiary of STATA training 2 Email. Phone Yes Yes Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Ojo Otuma Oju Student University of Calabar Beneficiary of STATA training 2 Email. Phone Yes Yes Dec. 7, 2018, Dec. 12, 2018 and Dec. 13, 2018 Austin Angba Professor University of Calabar Worked on NAPP's review of State Ministry's Agric. Development Plan Email, phone, in - person Yes Yes Dec. 7, 2018 and Sylvester Baye Acting Director for Policy and FMARD Key stakeholder 1 Yes Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacted Dec. 10, 2018 Project Coordinatio n Email, phone, in￾person Dec. 7, 2018 Andrew Kwasari Adviser to Minister of FMARD; Special Adviser to Vice President FMARD Key stakeholder; member of National Advisory Committee 1 Email, phone, in￾person Yes Yes Dec. 14, 2018 Victoria Ogri Director Cross River State Ministry of Agriculture and Natural Resources 1 In-person Yes Yes Dec. 14, 2018 Arif Ofem Bassey Director Cross River State Ministry of Agriculture and Natural Resources Accounting 1 In-person Yes Yes Dec. 14, 2018 Ikonfa Odey J. Director Cross River State Ministry of Agriculture and Natural Resources PRS/SPC 1 In-person Yes Yes Dec. 14, 2018 Ogbaji Simon Oko Director Cross River State Ministry of Agriculture and Natural Resources Livestock 1 In-person Yes Yes Dec. 14, 2018 Williams O. Yere Director Cross River State Ministry of Agriculture and Natural Resources Tree crops 1 In-person Yes Yes Dec. 14, 2018 Ekiwu Sunday Deputy Director Cross River State Ministry of Agriculture and Natural Resources 1 In-person Yes Yes Dec. 14, 2018 Inyang Etim Wayag Director Cross River State Ministry of Agriculture and Natural Resources Planning 1 In-person Yes Yes Dec. 14, 2018 Joshua Odey Wonah Head Cross River State Ministry of Agriculture and Natural Resources Rural CRADP Infrastructure￾CRADP 1 In-person Yes Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacte d Dec. 14, 2018 Prof. Egrinya Eneji Cross River State Ministry of Agriculture and Natural Resources 1 Email, phone, in - person Yes Yes Dec. 14, 2018 Obelen Ubi Head Cross River State Ministry of Agriculture and Natural Resources Crops 1 Email, phone, in - person Yes Yes Dec. 14, 2018 Rev. Joseph Ugbe Permanent Secretary Cross River State Ministry of Agriculture and Natural Resources Staff have participated in trainings and studies 1 Email. Phone, in - person Yes Yes Dec. 14, 2018 Nathaniel Nkor Head Cross River State Ministry of Agriculture and Natural Resource s Arable crops 1 In -person Yes Yes Dec. 14, 2018 U.B. Ubara Officer Cross River State Ministry of Agriculture and Natural Resources Biofuels 1 In -person Yes Yes Dec. 14, 2018 I.M. Ogbin DAQ Cross River State Ministry of Agriculture and Natural Resources In -person Yes Yes Dec. 14, 2018 Christian F. Ekpor Assistant Director Cross River State Ministry of Agriculture and Natural Resources Livestock In -person Yes Yes Dec. 14, 2018 L.A. Ufono DAS Cross River State Ministry of Agriculture and Natural Resources In -person Yes Yes Dec. 5, 2018 Charles Ebojei ARCN Policy/Training 1 Phone Yes Dec. 5, 2018 Sylvester Baye FMARD Policy/Key Partner 1 Phone Ringing/As k someone to follow up Dec. 5, 2018 Prof. Olu Ajakaiye NAC Advisory 4 Phone Ringing No Repl y Dec. 5, 2018 Lizzy Igbine NAC Advisory 4 Phone Yes. Appointmen t fixed for Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacted 3pm and later rescheduled for 4pm Dec.6, 2018 Mr Saidu Madagwa Ag. Executive Secretary ARCN Phone Yes. Secured email of the Executive Secretary and phone contact Dec. 6, 2018 FMARD Policy/Key Partner 1 Phone Ringing. No reply Dec. 7, 2018 Dr Manson Nwafor Ag. Economist IITA-Abuja Policy Partner/Think Tank 2 Phone Yes. Mission introduced. He actually was responding outside the country (Ghana). He promised calling back for further discussion Dec. 7, 2018 Sylvester Baye FMARD 1 Phone Ringing no response Dec.7,2018 Dr Oyeleke Oyinloye PCU￾FMARD 1 Phone Yes. Introduction of mission. Appointmen t scheduled for 3pm on Monday Dec 10th . Dec. 7, 2018 Dr Aisha Ndayako Head, PCU, FMARD 1 Phone Yes. Introduction of mission. Promised to call me back by 4pm to decide where to meet for the interview. Dec. 7, 2018 Dr. Kehinde Makinde Country Head AGRA Collaborator 4 Phone Yes. Introduction of mission. Promised to call me back in a minute to continue the discussion Jan 4, 2019 Alhaji Rauf Adekunle Lawal President NAIDA Stakeholder 5 Phone Yes. Discussed possibility Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacted of interview next week. Demanded advanced interview schedule. Jan 4, 2019 Dr Abubakar Sadiq Ambursa Lecturer Kebbi State Univ of Science and Tech Academia 2 Phone Yes. He has not participated in NAPP. He is not aware of any activity. Jan 4, 2019 Prof A. S. Abdullahi Professor Federal University – Birnin Kebbi Academia 2 Phone Yes. He did not participate in NAPP. He met a group of ABU scholars in Enugu last year who asked him to forward biodata; bust since then no feedback. Jan 4, 2019 Prof. Godwin Abu Professor/F acilitator Federal Univ of Agriculture, Makurdi Academia 2 Phone Yes. He will be available during the week of January 7th to attend to the evaluation team. Jan 8, 2019 Mr Abayol Luper Planning Officer MANR Makurdi Policy/Capacit y Building 1 Phone Yes. Preferred Wed 9th or any day of our choice Jan 8, 2019 Mr Ityo Simon Director Planning, M&E BNARDA Makurdi Policy/Capacit y Building 1 Phone Yes. He is available on January 10th . Jan 8, 2019 Abubakar Lolo Kebbi ADP (WRONGL Y LISTED IN MANR) Policy/Capacit y Building 1 Phone Yes. He will be available for interview on Jan 14th by 10 am Jan 8, 2019 Bello Shehu Kebbi MANR (WRONGL Y LISTED IN ADP) Phone Yes. He will be available for interview on Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacted Jan 14th by 10 am Jan 8, 2019 Alhaji Rauf Adekunle Lawal President NAIDA Stakeholder 5 Phone No. Initially by 6pm. He called back later 7.20 pm. Request interview guide. Interview guide sent to him electronical ly. Personal interview tentatively fixed for Friday Jan. 11th. To be confirmed on Thursday Jan. 10th . Jan 9, 2019 Prof. Godwin Abu Professor/F acilitator Federal Univ of Agriculture, Makurdi Academia 2 Phone Yes. Appointmen t fixed for 10am on Thursday Jan. 10th at College of Ag Econs & Extension, IFS Jan 9, 2019 Abubakar Lolo Kebbi ADP (WRONGL Y LISTED IN MANR) Policy/Capacit y Building 1 Phone Yes. Hinted that Bello Shehu will not be available on Jan 14 & 15. Appointme nt with Shehu reschedule d for Sunday Jan 13th at Birnin Kebbi. Appointme nt with Lolo confirmed for Jan. 14th . Jan 9, 2019 Dr Simeon O. Ajegi Lecturer Benue State Univ. Makurdi Academia 2 Phone Yes. Thursday not suitable. Appointmen t fixed for Friday Jan. 11th at 10am Jan 9, 2019 Mr Ityo Simon Director Planning, M&E BNARDA Makurdi Policy/Capacit y Building 1 Phone Yes. Appointmen t for interview fixed for Thursday Jan 10th at 12 noon Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacte d Jan 9, 2019 Mr Solomon Haanongon Lecturer University of Mkar, Mkar Gboko Academia 2 Phone Yes. Appointmen t fixed for Jan 10th at 4pm Jan 9, 2019 Mr Ityo Simon Director Planning, M&E BNARDA Makurdi Policy/Capacit y Building 1 Phone No. He is not aware of NAPP Dec. 3, 2018 Funso Olajide UI Visiting Prof 2 E -mail Yes Dec. 3, 2018 Damisa Maiyaki ABU Visiting Prof 2 Email Yes Dec. 3, 2018 Obadina Adewale Olusegun FUNAAB Visiting Prof 2 Email Yes Dec. 3, 2018 and Dec. 17, 2018 KABIR KAYODE SALMAN UI Visiting Prof 2 Email and phone Yes Dec. 3, 2018 and Dec. 11, 2018 Agber Philip Federal University of Agriculture Makurdi Visiting Prof 2 Email and Phone Yes Dec. 3, 2018 and Dec. 4, 2018 Charles Ebijei ARCN Training on Design and Impl. of Policy Reforms 1 Phone Yes Dec. 3, 2018 Dr MICHAEL CHUKWURA H. Academia 2 Email Yes Dec. 3, 2018 Prof. Jonathan Okechukwu Academia Not participating yet 2 Email Yes Dec. 4, 2018 and Dec. 12, 2018 Abdullahi M. Nasir ARCN Training on Design and Impl. of Policy Reforms 1 Email and phone Yes Dec. 5, 2018 and Dec. 11, 2018 Fabiyi Alice Olukemi ARCN Training on Design and Impl. of Policy Reforms 1 Email and phone Yes Dec. 4, 2018 and Dec. 18, 2018 AYODEJI COKER FUT Minna Visiting Prof 1 Email Yes Dec. 5, 2018 and Dec. 12, 2018 Prof Ajakaiye Center for shared dev capacity Ibadan NAC 1 Phone and Email Yes Dec. 5, 2018 Mrs lizzy Igbene Nigerian women agro allied farmers asso NAC 1 Phone and Email Yes Dec. 5, 2018 Dr Aminu Ibrahim Nigeria ICT Forun of partnership insitutions NAC 1 Phone and Email Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacte d Dec. 5, 2018 and Dec. 8, 2018 Alhaji Abba Aucham Quality control unit of the fed fert dept, FMARD NAC 1 Phone and Email Yes Dec. 5, 2018 and Dec. 12, 2018 Prof Bamikole M.A ADAN Academia 2 Phone and Email Yes Dec. 5, 2018 and Dec. 8, 2018 Prof Mrs David Abraham ADAN Academia 2 Phone and Email Yes Dec. 4, 2018 and Dec. 10, 2018 Prof Placid Njoku NIGERIA INST. OF ANIMAL SC NAC 1 Phone and Email Yes Dec. 6, 2018 Dr Andrew Kwasari Special adviser, FMARD NAC 1 Phone and Email Yes Dec. 6, 2018 Charity Ekerebi Activity scholar 2 Phone and Email Yes Dec. 4, 2018 and Dec. 10, 2018 Wale Ogunleye UI Activity scholar 2 Phone and Email Yes Dec. 4, 2018, Dec. 18, 2018, and Dec. 22, 2018 Osayanmo Wellington Osawe UI Activity scholar 2 Phone and Email Yes Dec. 5, 2018, Dec. 12, 2018 and Dec. 14, 2018 Uwah Ime David University of Calabar Capacity building 2 Phone Yes Dec. 5, 2018, Dec. 12, 2018 and Dec. 14, 2018 Ekpang Kate David University of Calabar Capacity building 2 Phone Yes Dec. 5, 2018, Dec. 12, 2018 and Dec. 14, 2018 Eno Uwah Univeristy of Calabar Capacity building 2 Phone Yes Dec. 4, 2018 and Dec. 10, 2018 Ayinde Taiwo Oluwaseun UI Capacity building 2 Phone Yes Dec. 4, 2018 and Dec. 10, 2018 Arogbodo Abigail Oluwakemi UI Capacity building 2 Phone Yes Dec. 4, 2018 and Dec. 10, 2018 Obisesan O. O UI Capacity building 2 Phone Yes Date Name Title Employer /School Role in NAPP Category Mode of Contact Interview Conducted Contact successfully contacte d Dec. 5, 2018, Dec. 12, 2018 and Dec. 14, 2018 Prof Angba August ine Univ of Calabar Academia 2 Phone and Email Yes Dec. 5, 2018, Dec. 12, 2018 and Dec. 14, 2018 Dr Kuye Oludayo Cross River State univer Academia 2 Phone and Email No Jan. 4, 2019 and Jan. 8, 2019 Prof. P.C. Ike Delta State University, Abraka Academia 2 Phone and Email Yes Jan. 4, 2019 and Jan. 8, 2019 Mr. Chukwura Michael Federal College of Education (Tech) Asaba Academia Phone and Email Yes Jan. 4, 2019 and Jan. 8, 2019 Mrs. Abba Roseline Ebonyi State ADP Government Phone Yes Jan. 4, 2019 and Jan. 8, 2019 Mr. Ndukwe Okpani Ebonyi State Ministry of Agriculture Government Phone and Email No Jan. 4, 2019 and Jan. 7, 2019 Mrs. Stella Egedi Delta State Ministry of Agriculture Government Phone and Email Yes Jan. 4, 2019 and Jan. 8, 2019 Mr. Johnson Nikoro Delta State ADP Government Email Yes ANNEX 6 Literature References IFPRI (2016a). “Who influences Government Spending in Agriculture? The Roles of Public Actors in Sub-national Funding Allocation in Nigeria.” Policy Note 36. Synopsis of NSSP Working Paper Number 36. December. IFPRI (2015). “Optimal Rice Tariffs with Smuggling: A Spatial Analysis of Nigerian Rice Policy Options.” NASSP II Working Paper Number 29. October. Takeshima H. (2017). Distributional effects of agricultural infrastructure in developing countries: Large irrigation dams and drought mitigation in Nigeria. Journal of Developing Areas, forthcoming. (based on NSSP Working Paper 35 completed in 2016 under the Nigeria Agricultural Policy Project). USAID Monitoring, Evaluation and Learning Activity (2018a). “Final Performance Evaluation: USAID/Nigeria Livelihoods Activity—In Brief.” PowerPoint file. USAID MLE (2018b). “Evaluation of the NEXXT Project.” PowerPoint file. USAID NAPP (2018a). “The Feed the Future Nigeria Agricultural Policy Project.” IFPRI website. USAID NAPP (2018b). “Meeting with National Advisory Committee.” August 15th . USAID NAPP (2018c). “Quarterly Report: First Quarter: October 1- December 31, 2017.” Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, January. USAID NAPP (2017a). “Year Three Workplan.” November 17th version. USAID NAPP (2017b). “Annual Report FY 2017.” October. USAID NAPP (2017c). “Quarterly Report: April-June 2017.” August 31st version. USAID NAPP (2017d). “Quarterly Report: January-March 2017.” April 28th version. USAID NAPP (2017e). “Quarterly Report: October-December 2016.” February 27th, 2017 version. USAID NAPP (2017f). “Year 3 Work Plan”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, November. USAID NAPP (2017g). “Quarterly 4/Annual Report: Fourth Quarter: July 1‐ September 30, 2017/Annual: October 1, 2016 – September 30, 2017”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, November. USAID NAPP (2017h). “The Quality of Agriculture and Food Security Policy Processes at the Federal and State Level in Nigeria: Results of a Stakeholder Survey”. Authored by: Maredia, Mywish; Mazunda, John and Tasie, Oyinkan. Research Paper 74, November. USAID NAPP (2017i). “Quarterly Report: Second Quarter: April 1‐ June 30, 2017”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, August. USAID NAPP (2017j). “Quarterly Report: Second Quarter: January 1- March 31, 2017”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, June. USAID NAPP (2017k). “Monitoring and Evaluation Plan (MEP)”. Prepared by: Michigan State University (MSU) in partnership with the International Food Policy Research Institute (IFPRI), December. USAID NAPP (2016a). “Contrary to Conventional Wisdom, Smuggled Chicken Imports are not Holding Back Rapid Development of the Poultry Value Chain in Nigeria.” Feed the Future Innovation Lab for Food Security Policy. Policy Research Brief 19. November. USAID NAPP (2016b). “Quarterly Report April 1-June 30.” August version. USAID NAPP (2016c). “Assessment of Agriculture and Food Security Policy Processes in Nigeria: Stakeholder Survey 2016.” March 8th . USAID NAPP (2016d). “Quarter 4/Annual Report: July 1-September 30, 2016/October 1, 2015- September 30, 2016”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, October. USAID NAPP (2016e). “Year 2 Work Plan.” Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, October. USAID NAPP (2016f). “Quarterly Report: Third Quarter – April 1- June 30, 2016”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, August. USAID NAPP (2016g). “Quarterly Report: First Quarter – October 1- December 31, 2015”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, January. USAID NAPP (2015a). “Monitoring and Evaluation Plan.” October 27th . USAID NAPP (2015b). “Year 0 and Year 1 Work Plan”. Prepared by: Michigan State University in partnership with the International Food Policy Research Institute, October. USAID Nigeria (2015). “Grant and Cooperative Agreement AID-620-LA-15-00001.” October 27th . USAID NSSP (2015). “Optimal Tariffs with Smuggling: A Spatial Analysis of Nigerian Rice Policy Options”. Authored by Johnson, Michael and Dorosh, Paul. Nigeria Strategy Support Program Working Paper 29, August. Xie H, L You & H Takeshima. (2018). Invest in small-scale irrigated agriculture: A national assessment on potential to expand small-scale irrigation in Nigeria. Agricultural Water Management, revise and resubmit stage. ANNEX 7 Evaluation Team TEAM LEADER AND AGRICULTURE POLICY EXPERT Aderibigbe Olomola, PhD Dr. Olomola is an agricultural policy scholar with over thirty years of experience in research and teaching at the Nigerian Institute of Social and Economic Research (NISER), over fifteen years of which have involved policy research coordination and management. Dr. Olomola has also served in various leadership positions within NISER, such as the head of the Macroeconomic and Strategic Modeling Unit (MASMU), Director of the Research and Consultancy Unit, Director of the Agriculture and Rural Development Department (ARDD), and Director of the Surveillance and Forecasting Department. Dr. Olomola has served as a consultant for institutions such as Federal Government of Nigeria as well as national and international organizations including the Agricultural Resource Council of Nigeria, West Africa Agricultural Productivity Program, the World Bank, the United Nations Development Programme, the United Nations Institute for Economic Development and Planning, the International Labor Organization, the Food and Agricultural Organization, International Food Policy Research Institute (IFPRI), Overseas Development Institute (ODI), Gesellschaft für Internationale Zusammenarbeit (GIZ), and Brookings Institution in the areas of agriculture, food security, and development economics. Dr. Olomola has served as coordinator of courses on monitoring and evaluation and delivered capacity building programs on M&E at NISER, as well as supervised student at the Ph.D. level. AGRICULTURE ECONOMIST Olajide Ajao, PhD Dr. Ajao, an agriculture economist with over 20 years of teaching and research experience, has published thirty-eight publications on agriculture productivity, farming, gender and agriculture, and efficiency or effectiveness of different agricultural production techniques in Nigeria - including land use and smallholder fish farms. Dr. Ajao currently serves as Head of the Department of Agricultural Economics for Ladoke Akintola University of Technology in Ogbomoso, Oyo State. Dr. Ajao holds a PhD in Agricultural Economics from Obafemi Awolowo University in Ile-Ife, Nigeria. AGRICULTURE ECONOMIST AND EVALUATION/SURVEY SPECIALIST Ayodele Fashogbon, PhD Dr. Fashogbon has eight years of experience supporting evaluations in Nigeria and West Africa for the World Bank and USAID. He has worked on a range of evaluations with a variety of themes – from an impact assessment for a Jatropha Plantation study to an impact evaluation on commercial agriculture development. Dr. Fashogbon is experienced in developing survey instruments and evaluation plans, conducting KIIs and FDGs, and providing data collection training and supervising enumerators. His most recent consulting work was with the World Bank as an Impact Evaluation consultant in Nigeria and Tanzania where he worked on several projects, including an Agro-Processing Productivity Enhancement and Livelihoods project, a Commercial Agriculture Development project, a Promoting Safe Sex among Adolescents/Empowerment and Livelihood for Adolescents project, and a Quality Enhancement of Primary Healthcare centers in Nigeria project. On each project, Dr. Fashogbon led the technical teams or provided evaluation technical assistance. He has his PhD in Agriculture Economics from the University of Fort Hare in Alice, South Africa. SENIOR MONITORING, EVALUATION AND LEARNING (MEL) SPECIALIST Olufemi Gisanrin Mr. Olufemi Gisanrin has over 25 years of experience that cuts across private and agri-business development. His first 13 years of experience were multidisciplinary, working with multinational teams in the design and management of an integrated dairy farm involving production of various dairy products from fresh milk from exotic and cross-bred dairy cows, and processing and marketing of dairy products in Nigeria. Since 2006, Mr. Gisanrin has managed M&E contracts for key USAID agricultural development projects including: MARKETS, Bridge to MARKETS II, and MARKETS II. He has both field level and supervisory experience managing project and activity monitoring and evaluation systems. Most recently he has led a technical team of six professional on a USD 64 million agriculture project funded by the United States Agency for International development. Mr. Gisanrin guided regional teams and service providers to ensure effective value chain focused M&E processes and mechanisms were in place for accurate monitoring and reporting on over 1,500,000 beneficiaries spread across 22 states. Mr. Gisanrin is currently a Senior Monitoring, Evaluation and Learning Specialist on the USAID funded Monitoring, Evaluation and Learning (MEL) Activity, supporting the Economic Growth and Environment (EGE) program office of USAID Nigeria Mission. He has a Master’s degree in Development Studies from the Nigerian Defense Academy in Kaduna Nigeria. ANNEX 8 Conflict of Interest Statements (COI) Disclosure of Conflict of Interest for USAID Evaluation Team Members Name Aderibigbe Olomola Title Prof. Organization Nigerian Institute of Social and Economic Research (NISER). Ibadan EvaluationPosition? D Team Leader [IJ Team member Evaluation Award Number (contract or other instrument) AID-OAA -1- 15-00018/AID-620-T0-16-00001 USAID Project(s) Evaluated (Include project name(s}, implementer name(s) and award number(s}, if applicable) Project: Feed the Future Nigeria Agricultural Policy Activity Award Number: AID-620-LA- 15-00001 Implementers: MichiganState University/International Food Policy Research Institute Ihave real or potential conflicts of interest to disclose. D Yes (i] No Ifyes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Closefamily member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organizotion(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or issignificant though indirect, in the implementing orgonizotion(s} whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previou s work experience with on organization that may be seen as on industry competitor with the implementing organization(s) whose project(s) are being evaluated . 6. Preconceived ideas toward indMduols, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2} that I wi ll update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and ANNEX 9: Statement of Differences by Implementing Partners Statement of Differences (SOD) on the Feed the Future Nigeria Agricultural Policy Activity Mid-term Performance Evaluation Report The statement is in two parts. The first part is by IFPRI and addresses in-country activities delivered by both partners (IFPRI and MSU) while the second part deals with out of country matters essentially delivered by MSU. Part I. Statement of Differences on the Mid-Term Performance Evaluation Report of the FtF Nigeria Agricultural Policy Project: In-Country Activities delivered by IFPRI and MSU. Although the overall Project assessment by the evaluation team is positive, the final report still includes many false statements and strong conclusions throughout which in many cases lack any strong foundations (and as both Implementing Partners made clear in their numerous detailed comments/responses to the draft evaluation report statements and conclusions). Besides, the report uses throughout a number of Boxes to highlight various (and always negative) comments by some stakeholders on Project activities, but at the same time, and for reasons we do not understand, ignores very positive statements by other stakeholders about the project (including for example the one made – and shared with the evaluation team (see NAC August 2018 meeting minutes) – by a member of the National Advisory Committee of the Project back in August 2018, namely that “the Project has already exceeded its expectations”). This would rather lead to the inevitable conclusion that the evaluation report suffers from some degree of negativity bias by offering selected glimpses of parts of the ‘evidence’ reported here. The report concludes that the research work conducted by the Project did not take into account the federal government research priorities but instead it was conducted with the priorities of MSU and IFPRI researchers in mind. This is an entirely misleading conclusion since research topics in years 1-3 clearly took into account FMARD research priorities of relevance to the Agriculture Promotion Policy (APP) of the Government of Nigeria (GoN) and following also various discussions and consultations with FMARD colleagues. Almost 90% of the research conducted by the Project in its first 3 years is either directly or indirectly related to the policy priorities of the GoN summarized in APP. Of relevance to the above is also another hugely incorrect statement in the evaluation report, namely that the Project “has over-emphasized academic-style research that could be seen as advancing the careers of the IFPRI and MSU researchers, rather than on putting the Nigerian collaborators in the forefront”. This is a very false statement that comes as a strong surprise to the two IPs of the Project. The research conducted by the Project in its first 3 years, was inclusive in terms of feedback and collaboration with local researchers, academically rigorous (and we guess this is a good thing) but at the same time, policy relevant, to the benefit of all those using the evidence and findings emanating from the above research. More importantly, local researchers were heavily involved in the research teams conducting the research, mentoring was provided by the experienced MSU and IFPRI researchers during the research process and they were also involved in the numerous publications (see 35 Working Papers/Research Papers, 39 Policy Briefs/Policy Notes and 17 journal articles) produced by the Project so far. Another incorrect statement in the report is about the way the Project in its first 3 years disseminated within Nigeria the policy relevant research output. This comes as a huge surprise to the Project Management Team since the Project has systematically and very actively managed to enhance dissemination activities in about 80 Project-organized seminars, workshops, policy dialogues and events, including two major conferences in 2016 & 2018, attended by almost 2800 participants, with further emphasis to be placed by the Project on this front in the remaining part of the Project’s life. The report is also very critical regarding the indicators used to measure the Project’s performance in its first 3 years and various problems associated with the appropriateness of these indicators (SMART or not), their relevance, specificity, and discontinuity. It is important to stress, however, that many indicators have changed since the inception of the Project following clear instructions the USAID’s various AORs (3 in total) supervising the Project during its first 3 years. And this led to a continuous “shifting of goal posts” in terms of project management due to lack of continuity regarding indicators from one year to the other. Another confusing statement is about the “lack of linkages between the Project and FMARD’s policy coordination unit as a weakness in the Activity’s approach”. This is a rather confusing statement, however. The Project uses various focal points at the Ministry, and on a systematic and continuous basis, to alert them about Project related activities, including the policy coordination unit, with which recently we have also designed together the courses for FMARD to be delivered by the Project in Year 4 (and in line with what we do every year). Perhaps this ‘gap’ may be explained by the frequent change of focal points in the Ministry regarding coordination, and about which the Project cannot do much. The report surprisingly also fails to recognize that unpredictable academic calendars, academic association strikes, various security issues within the Nigerian setting the Project operates have at times affected the planning and the implementation of capacity building and research dissemination activities beyond the Project’s control. Another major constraint (surprisingly ignored in the report) in Year 1 of Project’s implementation was the publicity embargo imposed on the Project by USAID/Nigeria for reasons related with the official launch of the Project. The embargo was eventually lifted in September 2016 with the official launch of the Project in the First Project Conference organized in Abuja in September 2016, but it clearly affected the planned by the Project activities during the first year in terms of publicity and dissemination. Finally, we are rather surprised by the limited number of Project references and Project material used and cited in the report by the evaluation team to conduct such an important project evaluation. This partly explains many of the misleading statements in the report about the performance of the Project so far. Part 2. Statement of Differences on the Mid-Term Performance Evaluation Report of the FtF Nigeria Agricultural Policy Project: Out of Country Activities Delivered by MSU. The project notes the evaluation claim that “The most convincing evidence for sustainability is the knowledge gained by the MSU scholars, research collaborators, and capacity building beneficiaries” (p.44). The basis of this claim about an out of country activity is not substantiated. In addition, the administration and outputs of this program contradicts many of the report findings and already conforms to several of the report recommendations. The scholars’ program works with competitively selected students and their supervisors through the whole research process. This includes the conceptualization of the research problem, data collection, data analysis, writing up and presentation of results. Each scholar’s program activities are developed individually based on the scholar’s level and needs and in conjunction with their Nigerian academic advisor and MSU mentor. Each scholar has a scope of work developed with the active participation of all these stakeholders and takes courses at MSU to strengthen identified knowledge and research gaps. The data analysis skills of scholars are enhanced through training on “R” a free and open source software that addresses the challenges of unaffordability of STATA in Nigerian institutions. Mentoring which starts during their tenure at MSU continues after their return home. With the support of project researchers, scholars produce research reports, policy briefs and journal publications. Scholars are training others to benefit from the knowledge and skills gained through the MSU experience. During the evaluation period, the Project scholars were involved in the production of 27 research outputs, 21 (~80%) of which are with them as the lead or sole author. They have made over 30 research presentations. Scholars have facilitated project organized “R” training in Nigeria and developed training manuals (one for academics and another for staff at ministries). Training activities to popularize the use “R” in Nigeria continues. In just about 3 years, over 800 Nigerians have been trained by scholars on data collection and analysis methods. Research based non-technical communication pieces for farmers and other stakeholders produced by the project have been translated into Nigerian languages and are currently in circulation. Short oral pieces to be disseminated via radio, TV and social media are under development. Investing in a few scholars is benefiting hundreds of Nigerians. The project scholars program collaborates with the Nigerian Association of Agricultural Economics (NAAE) to organize capacity building events for young Nigerian researchers. Project support has been provided for them to attend NAAE conferences, where special sessions (led by project and Nigeria￾based faculty) are held for the young researchers followed by joint mentoring (by leaders in Nigeria and the USA). The exposure of Nigerian research through on-line access to the Nigerian journal of agricultural economics (NJAE) has been enhanced by a scholar-led initiative and AgEconSearch recently announced 79 papers from the NJAE were downloaded from its site 10,676 times, in 2018. Elements of this Policy Project capacity building effort have been adopted at the continent level by the Africa Section of the Agricultural and Applied Economics Association (AAEA). Abuja and East Lansing, April 5, 2019 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523