Conference Edition 12th CAADP Partnership Platform Independent Evaluation of CAADP Multi-Donor Trust Fund (MDTF) Volume 1: Main Report April 9, 2016 Chris Gerrard (Team Leader) Gem Argwings-Kodhek Ali Marouani Godfrey Mudimu Additional Assistance from Diane Bombart Andrea Deisenrieder Markus Palenberg i Table of Contents Acronyms and Abbreviations ................................................................................................... v Acknowledgments................................................................................................................... vii Program at a Glance: CAADP Multi-Donor Trust Fund ......................................................... ix Recipient-Executed Projects Supported by the CAADP MDTF ............................................. xi Executive Summary ............................................................................................................... xiii 1. Introduction to the Evaluation .............................................................................................. 1 Purpose and Scope of the Evaluation.................................................................................... 3 Evaluation Design ................................................................................................................. 3 Evaluation Issues and Questions ........................................................................................... 4 Evaluation Approach and Methodology ............................................................................... 4 Structure of the Report .......................................................................................................... 5 Main Limitations of the Evaluation ...................................................................................... 5 2. Objectives and Design of the MDTF and Associated CTF Projects .................................... 7 Original Objectives and Components ................................................................................... 7 Additional Financing and Restructuring the CTF Projects in 2014 ...................................... 9 Proposals for a Follow-on Malabo Facility ........................................................................ 10 3. The Implementation Experience of the MDTF and CTF Projects ...................................... 12 Phase 1: From Inception to the Establishment of the First Five CTF Projects ................... 12 Phase 2: From the Establishment of the First Five CTF Projects to the Mid Term Review ................................................................................................................................ 16 Phase 3: From the Mid Term Review to Restructuring and Additional Finance................ 20 Phase 4: The Two-Year Extension Phase, 2014–2015 ....................................................... 24 Overall Achievements of the MDTF .................................................................................. 29 4. Development Effectiveness ................................................................................................ 33 Relevance of Objectives and Design .................................................................................. 34 Outputs and Outcomes Achieved........................................................................................ 40 Likely Sustainability of Outputs and Outcomes Achieved ................................................. 53 5. Cross-Cutting Issues ........................................................................................................... 56 Inclusion, Gender and Climate Smart Agriculture ............................................................. 56 The Science Agenda for Agriculture in Africa (S3A) ........................................................ 59 The Continental Agribusiness Strategy .............................................................................. 62 Lessons ................................................................................................................................ 64 ii 6. Organizational Effectiveness .............................................................................................. 66 Governance and Management Arrangements ..................................................................... 66 Governance and Management Performance ....................................................................... 69 Efficiency of the Financial and Human Resources Used.................................................... 73 Monitoring and Reporting of Progress and Results ............................................................ 78 Donor Coordination ............................................................................................................ 80 7. World Bank Performance in its MDTF Roles .................................................................... 83 Preparation and Implementation Support to CTF Recipient Organizations ....................... 83 Facilitating Linkages, as a Development Partner, with the Bank’s Country Operations ... 89 Establishing and Administering the MDTF ........................................................................ 95 8. Conclusions and Lessons .................................................................................................... 97 References ............................................................................................................................. 104 Boxes Box 1. The CAADP MDTF: A Confusing Name for the Program .......................................... 2 Box 2. Strategic and Operational Plans (SOPs) ...................................................................... 13 Box 3. The Continuing Relevance of CAADP ....................................................................... 35 Box 4. The Core Values and Principles of CAADP ............................................................... 50 Box 5. Activities of the CAADP Advisor for Agricultural Science, Technology Dissemination and Adoption in Support of the Development of the Science Agenda ........................................................................................................................ 64 Box 6. The Theory and Practice of the Service Agency (SA) ................................................ 70 Box 7. The Confusing Existence of Two Results Frameworks .............................................. 80 Box 8. The World Bank’s Fiduciary Role .............................................................................. 95 Figures Figure 1. Revised CAADP MDTF Results Framework, FY2014 .......................................... 11 Figure 2. The CAADP Results Framework: 2015–2015 ........................................................ 27 Figure 3. Cumulative Country-Level Progress ....................................................................... 30 Figure 4. Cumulative Number of Agriculture Public Expenditures Reviews and Joint Sector Reviews............................................................................................................ 30 Figure 5. To what extent do you agree with the following statements? ................................. 39 Figure 6. To what extent do you feel that the enabling environment for agricultural programs and policies in your country — or the country, subregion, or region that you have worked on — has improved in the following ways? ............................ 41 Figure 7. To what extent do you attribute the improvements that have occurred to the following? ................................................................................................................... 42 Figure 8. To what extent do you feel that the CAADP activities you have worked on have contributed to the following Level 3 results in the CAADP Results Framework? ......... 43 iii Figure 9. To what extent do you feel that the CAADP activities you have worked on have contributed to the following Level 2 results in the CAADP Results Framework? ......... 43 Figure 10. To what extent has your country been successful in mobilizing investment resources for your National Agricultural Investment Plan? ...................................... 45 Figure 11. To what extent are the following stakeholder groups involved in national agricultural policy making structures in your country — or the country, region, or subregion that you work on? ....................................................................................... 46 Figure 12. How closely connected or involved are you, as the CAADP Focal Point, with the senior policy making and management functions of your Ministry of Agriculture? ......... 47 Figure 13. How closely connected are your country-level CAADP processes to the national economic policy making and budgeting processes in your country? ........... 47 Figure 14. To what extent do you feel that the internal/external communications and overall knowledge management among regional and national-level CAADP stakeholders have changed in the last five years in the following ways? ................... 49 Figure 15. To what extent do you feel that the development and management of mutually beneficial partnerships at the regional and national levels have changed in the last five years (since 2010) in the following ways? .................................................... 51 Figure 16. Organogram of the MTDF, April 2010 ................................................................. 66 Figure 17. Elapsed Time from Concept Review to Effectiveness .......................................... 87 Figure 18. Occurrences of “CAADP” in Project Appraisal Documents over Time ................... 91 Figure 19. Occurrences of “CAADP” in Country Assistance Strategies over Time .................. 92 Tables Table 1. MDTF Disbursements and Expenditures .................................................................. 32 Table 2. Membership and Mandate of the Partnership Committee ........................................ 67 Table 3. Annual Donor Contributions, Fiscal Years 2009 to 2016 ........................................ 74 Table 4. Annual Trust Fund Disbursements, FY2009–2016 .................................................. 74 Table 5. Bank-Executed Activities: Actual Expenditures vs. Original Budget ..................... 76 Table 6. Bank-Executed Activities: Components of Total Costs ........................................... 77 Table 7. Additional Development Partners Support for CTF Organizations during the Project Period ........................................................................................................ 82 Table 8. The Multiple Roles of the World Bank in the CAADP MDTF ................................ 84 Table 9. World Bank-Supported Agricultural Projects in Africa: Average Number of Months for Project Preparation............................................................................... 86 Table 10. World Bank-Supervised Public Sector GAFSP Projects to Date ) ......................... 94 v Acronyms and Abbreviations AA Administration Agreement (World Bank) AFAAS African Forum for Agricultural Advisory Services AgPER Agriculture Public Expenditure Review AU African Union AUC African Union Commission AUC-DREA AUC’s Department for Rural Economy and Agriculture ASARECA Association for Agricultural Research in Eastern and Central Africa CAADP Comprehensive Africa Agriculture Development Program CAADP PP CAADP Partnership Platform CAS Country Assistance Strategy (World Bank) CCARDESA Centre for Coordination of Agricultural Research and Development for Southern Africa CEO Chief Executive Officer CILSS Permanent Interstates Committee for Drought Control in the Sahel CMA/WCA Conference of Ministers of Agriculture of West and Central Africa COMESA Common Market for Eastern and Southern Africa CORAF Council for Agricultural Research and Development CPS Country Partnership Strategy CSA Climate smart agriculture CSO Civil Society Organization CTF Child Trust Fund DFID Department for International Development (UK) DPO Development Policy Operation DPTT Development Partners Task Team EAC East African Community EAFF East African Farmers Federation EC European Commission ECCAS Economic Community of Central African States ECDPM European Centre for Development Policy Management ECOWAS Economic Community of West African States ESAFF Eastern and Southern Africa Farmers’ Forum EU European Union FAO Food and Agricultural Organization of the United Nations FARA Forum for Agricultural Research in Africa FIF Financial Intermediary Trust Fund (World Bank) FY Fiscal year GAFSP Global Agriculture and Food Security Program GEF Global Environment Facility GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit GDPRD Global Donor Platform for Rural Development GFRP Global Food Crisis Response Program GRPP Global or regional partnership program IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report (World Bank) IDA International Development Association IEG Independent Evaluation Group (World Bank) IFAD International Fund for Agricultural Development IFDC International Fertilizer Development Center IFPRI International Food Policy Research Institute vi IGAD Intergovernmental Authority on Development ISC Interim Steering Committee IS&R Implementation Strategy and Roadmap ITR Independent Technical Review KIS Knowledge, Information and Skills (a CAADP initiative) M&E Monitoring and Evaluation M&R Monitoring and Reporting MDTF Multi Donor Trust Fund MTR Mid Term Review NAIP National Agricultural Investment Plan NEPAD New Partnership for Africa’s Development NPCA NEPAD Planning and Coordination Agency NSA Non-State Actor OECD/DAC Organisation for Economic Co-operation and Development / Development Assistance Committee PAD Project Appraisal Document (World Bank) PAFO Pan-African Farmers Organization PC Partnership Committee (CAADP MDTF) PCR Project Completion Report PDO Program or project development objective PLI Pillar Lead Institution POW Program of Work PROPAC Plate forme Sous Régionale des Organisations Paysannes d’Afrique Centrale REC Regional Economic Community ReSAKSS Regional Strategic Analysis and Knowledge Support System ROPPA Le Réseau des organisations paysannes et de producteurs de l’Afrique de l’Ouest SA Service Agency SACAU Southern African Confederation of Agriculture Unions SADC Southern African Development Community SDTF Single Donor Trust Fund SOP Strategic and Operational Plan TA Technical Assistance TAC Technical Advisory Committee TAE Tertiary Agricultural Education TEAM Tertiary Education for Agriculture Mechanism TTL Task Team Leader (World Bank) UK United Kingdom UKZN University of Kwazulu-Natal UMAGRI L’Union maghrébine des agriculteurs UNCCD United Nations Convention to Combat Desertification UNECA United Nations Economic Commission for Africa UNZA University of Zambia USA United States of America USAID U.S. Agency for International Development Fiscal Year of the World Bank July 1 to June 30 Unless otherwise indicated, $ always refers to United States dollars. vii Acknowledgments The present evaluation has been commissioned by the World Bank on behalf of the MDTF Partnership Committee in accordance with World Bank policy requiring periodic evaluations for programmatic trust funds like the CAADP MDTF, and has been financed from the MDTF. The evaluation has been prepared by a team led by Chris Gerrard following the final version of the Inception Report that was issued in October 2015. Other members of the team were Gem Argwings-Kodhek, Ali Marouani, and Godfrey Mudimu. Each of these team members has prepared Background Papers, as follows, on two of the six organizations that have been supported by the CAADP MDTF to advance, coordinate, and facilitate CAADP processes and implementation at the continental, regional, and national levels:  Gem Argwings-Kodhek — On the African Union Commission, Department of Rural Economy and Agriculture (AUC-DREA); and the Common Market for Eastern and Southern Africa (COMESA)  Ali Marouani— On the Economic Community of Central African States (ECCAS); and the Economic Community of West African States (ECOWAS)  Godfrey Mudimu— On the NEPAD Planning and Coordinating Agency (NPCA); and the Southern African Development Community (SADC). Diane Bombart, Andrea Deisenrieder, and Markus Palenberg of the Institute for Development Strategy in Munich have also contributed to this evaluation by (a) administering the CAADP Focal Point and Partner Surveys, (b) analyzing the preparation time of the seven recipient-executed Child Trust Fund (CTF) projects in comparison with other World Bank-supported agricultural projects in Africa, and (c) undertaking keyword searches of World Bank Country Partnership Strategies of African countries and project documents of African agricultural projects. The evaluation team gratefully acknowledges the time, documentation, and assistance provided by the members of the World Bank’s CAADP team: Willem Janssen (current Team Leader), David Nielson (Team Leader from inception to May 6, 2014), Tim Robertson (Evaluation Manager), Christian Berger, Melissa Brown, Catherine Defontaine, Bremala Nathan, Adetunji Oredipe, Florence Sakyiama, and Dwede Tarpeh. The Evaluation Manager has worked closely with the Team Leader to ensure technical rigor at all stages of the evaluation, and to facilitate the work of the evaluation without hindering its independence. The evaluation team especially wishes to acknowledge the cooperation and assistance of the management and staff of the six continental and regional organizations listed above. Each organization prepared a self-assessment of their activities, outputs, and outcomes which were supported by the CTF grants from the MDTF. Each facilitated week-long site visits by evaluation team members to their organizations, without which it would not have been possible to complete this evaluation, and responded to additional enquiries after the site visits were completed. (See Annex L for the complete list of persons met and consulted at each of the six organizations, and elsewhere.) viii The team also acknowledges everyone else who made their time available for interviews and meetings, who completed one of the two surveys, and who otherwise provided useful information and insights which made this evaluation possible. This includes, in particular, the representatives of the contributing donors to the MDTF: Monique Calon (Netherlands), Jeff Hill (USA), Liz Kirk (UK), Earnan O’Cleirigh (Ireland), and Aude Sauvaget (European Commission). A Reference Group of five independent experts was established at the outset to enhance the independence of the evaluation: Arne Cartridge, Professor Monty Jones, Antoine Mvonde Ze, Janah Ncube, and Ruerd Ruben. They have reviewed and provided helpful and constructive comments on both the Inception Report and the draft Final Report. The draft Final Report was circulated for review and comment to the six CTF organizations, the contributing donors, and the World Bank’s CAADP Team. Comments received from all parties have been taken into account in finalizing the report. ix Program at a Glance: CAADP Multi-Donor Trust Fund Start date The Africa Region authorized the establishment of the CAADP MDTF on September 30, 2008, and the program document was published on November 10, 2008. The two parent trust funds supporting this program became effective on September 30, 2008, and December 15, 2008, respectively. Program Development Objective (PDO) The initial PDO was “African agricultural programs and institutions at the national, regional, and continental levels are scaled up and more effective through improved access to (a) technical guidance and support, (b) political support, and (c) financial support.” The revised PDO, which approved on January 24, 2014, has been an “improved enabling environment for African agricultural programs and policies at national, regional and continental level.” Major Activities Generally speaking, the CAADP MDTF has been strengthening the capacity of continental and regional African organizations to advance, coordinate, and facilitate CAADP processes and implementation at the national and regional levels. About half the activities have been executed by seven African organizations, as follows, that have received multi-year funding in the form of recipient-executed child trust fund (CTF) grants from the parent trust funds:  African Union Commission (AUC)  NEPAD Planning and Coordinating Agency (NPCA)  Common Market for Eastern and Southern Africa (COMESA)  Conference of Ministers of Agriculture of West and Central Africa (CMA/WCA)  Economic Community of Central African States (ECCAS)  Economic Community of West African States (ECOWAS)  Southern African Development Community (SADC) Other MDTF activities have been executed by the World Bank in support of the following areas:  Supporting the implementation of the above CTF grants  Helping to develop the overall CAADP framework, including CAADP guidelines and Pillar framework documents  Capacity building within CAADP priority areas and themes  Supporting national and regional CAADP processes  Supporting country-level agricultural public expenditure reviews (AgPERs)  Contributing to MDTF governance and administration Donor Contributions Six donors have contributed $65.2 million to the parent trust funds. The three largest donors (EU, USAID, and UK-DFID) have contributed 82 percent of the funds and the other three donors (Netherlands, Ireland, and France) have contributed 18 percent. x Locations The recipients of the CTF grants are located as follows:  AUC – Addis Ababa, Ethiopia  NPCA – Midrand, South Africa  COMESA – Lusaka, Zambia  CMA/WCA – Dakar, Senegal  ECCAS – Libreville, Gabon  ECOWAS – Abuja, Nigeria  SADC – Gaborone, Botswana All the members of the World Bank’s task team have been located in Washington, DC, with the exception of Adetunji Oredipe who is located in Abuja, Nigeria. Website www.caadp.net . The CAADP MDTF has not maintained its own website separate from the CAADP website that is maintained by NPCA. Governance and management The MDTF has been governed by a Partnership Committee whose membership has evolved over time. Since 2013, this has comprised representatives of the following:  AUC (Chair)  NPCA (Secretariat)  RECs (rotating annually among the four RECs)  World Bank (Fund administrator)  Contributing donors (appointed by the Development Partner group)  Knowledge institution  Private sector (rotating annually among private sector firms/organizations)  Civil society (rotating annually among civil society organizations) Previous evaluations/ reviews William Kingsmill, Amdissa Teshome, and Stephen Tembo, “Mid Term Review of the Multi Donor Trust Fund Supporting the Comprehensive Africa Agriculture Development Programme,” November 2011. European Centre for Development Policy Management (ECDPM), Laboratoire d'Analyse Régionale et d'Expertise Sociale (LARES) and the Economic and Social Research Foundation (ESRF), “Independent Assessment of the CAADP Multi-Donor Trust Fund,” February 2014. Michael Wales and Patrick Tawonezvi, “CAADP: Service Agency Review,” May 15, 2015. xi Recipient-Executed Projects Supported by the CAADP MDTF (presented in chronological order by date of effectiveness, from the earliest to the latest projects, and with the current Task Team Leaders) Project Name Task Team Leadera Identifi￾cation Mission Joint Technical Review Bank Approval Effective￾ness Closing Common Market for Eastern and Southern Africa (COMESA) Child Trust Fund Melissa Brown 4-May￾2009 1-Feb￾2010 8-Jun￾2010 27-Aug￾2010 31-Dec￾2015 Conference of Ministers of Agriculture of West and Central Africa (CMA/WCA) Child Trust Fund David Nielson 14-Sep￾2009 25-Jan￾2010 8-Jun￾2010 10-Sep￾2010 31-Mar￾2012 African Union Commission Child Trust Fund Tim Robertson 10-Jun￾2009 22-Feb￾2010 8-Jun￾2010 29-Oct￾2010 31-Dec￾2015 Economic Community of Central African States (ECCAS) Child Trust Fund Christian Berger 14-Sep￾2009 22-Feb￾2010 8-Jun￾2010 22-Nov￾2010 31-Dec￾2015 NEPAD Planning and Coordinating Agency (NPCA) Child Trust Fund Tim Robertson 14-Sep￾2009 25-Jan￾2010 8-Jun￾2010 11-Apr￾2011 31-Dec￾2015 Economic Community of West African States Adetunji Oredipe b 17-Feb￾2010 2-Jan￾2012 23-Aug￾2013 4-Oct￾2013 31-Dec￾2015 SADC Secretariat Child Trust Fund Melissa Brown 13-May￾2009 7-Mar￾2011 26-Aug￾2013 16-Oct￾2013 31-Dec￾2015 a. David Nielson was initially the formally designated Task Team Leader of all the CTF Projects. b. Adetuni Oredipe is the current Task Team Leader for the ECOWAS project. Christian Berger was the Task Team Leader until August 8, 2014. xiii Executive Summary Background and Context 1. The Comprehensive Africa Agriculture Development Programme (CAADP) — which was first launched at the African Union Summit in Maputo, Mozambique, in 2003 — is the AU’s vision and strategy for the development of African agriculture. As of 2007, however, only one country (Rwanda) had formally held a Roundtable meeting and signed a CAADP Compact (in March 2007). Consequently, at the 2nd meeting of the CAADP Partnership Platform (CAADP PP) in Addis Ababa in September 2007, the AU called for the establishment of a multi-donor trust fund (MDTF) to be administered by the World Bank as a mechanism through which financial support from Development Partners could be channeled to accelerate regional and country-level CAADP processes. The African Union and key donors felt that the World Bank, which was already providing financial assistance in many areas of CAADP, had the administrative capacity to manage the fund and the professional capacity to provide technical leadership. 2. Six donors — the European Union, USA, UK, Netherlands, Ireland, and France — have since contributed $65.2 million to the MDTF as of December 31, 2015. Administratively, the World Bank established recipient-executed Child Trust Fund (CTF) grants for the AU Commission, the NEPAD Planning and Coordinating Agency (NPCA), and four regional economic communities (RECs) to accelerate regional and country-level CAADP processes; and for the Council of Ministers of West and Central African States (CMA/WCA) — the designated Pillar Lead Institution (PLI) for CAADP Pillar II on improving rural infrastructure and trade-related capacities for market access. The World Bank has also administered a Technical Assistance Fund (TA Fund) to provide technical assistance in support of CAADP processes, particularly before the CTF projects were up and running. PURPOSE AND SCOPE OF THE EVALUATION 3. The present evaluation has had two major purposes:  To assess the achievements of the MDTF and related CTF projects, including (a) their contributions to improving the enabling environment for African agricultural programs and policies; (b) their organizational effectiveness; and (c) the performance of the World Bank in the multiple roles that it has played in the MDTF.  To draw lessons from the experiences of the MDTF and related CTF projects that might be incorporated into the objectives and design of a follow-on “Malabo” facility in which Development Partners continued to pool financial resources in support of CAADP implementation. 4. The evaluation has covered the time period from the initial approval of the MDTF in September 2008 to the present day. xiv 5. The evaluation has covered a series of evaluation issues and questions, structured around the two dimensions of development effectiveness and organizational effectiveness:  Development Effectiveness o Relevance of objectives and design o Efficacy (outputs and outcomes achieved) o Likely sustainability of outputs and outcomes achieved o Cross-cutting issues  Organizational Effectiveness o Governance, management, and administration of the MDTF o Efficiency of financial and human resources used o Monitoring and reporting of progress and results o World Bank performance in its multiple MDTF roles APPROACH AND METHODOLOGY 6. The evaluation has based its findings, conclusions, and lessons on the following data collection and analysis, and the triangulation of evidence collected from these different sources:  Background information on the MDTF and CTF projects  Review of previous evaluations and assessments  CAADP Annual Trends and Outlook Reports prepared by the Regional Strategic Analysis and Knowledge Support System (ReSAKSS)  MDTF semi-annual Stakeholder Status Reports prepared by the World Bank  Self-assessment reports prepared by each of the six CTF organizations and the World Bank’s task team – focusing on their activities, outputs, and outcomes  Site visits to each of the six CTF organizations  Stakeholder interviews with contributing donors, other Development Partners, government officials, informed academics, non-state actors, etc.  In-depth reviews of selected activities – focusing on their relevance of objectives and design, theory of change, relevance and effectiveness of partnerships, outputs, outcomes, and likely sustainability of benefits  CAADP Focal Point and Partner surveys  Governance and management analysis  Elapsed time analysis for the preparation of the CTF projects  Keyword searches of World Bank Partnership Strategies in African countries and World Bank appraisal documents of African agricultural projects. 7. The first survey was administered to CAADP Focal Points in 47 countries, to which 35 current and former Focal Points from 32 countries responded, for a response rate of 68 percent in terms of countries. The second survey was administered to 97 CAADP Partners who have worked with one or more of the six CTF organizations at the regional and continental levels. A total of 41 Partners responded for a response rate of 42 percent. Both surveys were administered in English and French. xv 8. Ten of the questions in the two surveys asked essentially the same question, but from the different perspectives of Focal Points working at the country level and Partners working at the regional and continental levels. Triangulating these survey results, particularly with the self￾assessment reports prepared by the six CTF organizations, has provided an overall assessment of the MDTF’s achievements from these three perspectives. Overall, there were no significant differences in the distribution of survey responses between CAADP Focal Points and Partners in 41 of the 48 subquestions that were the same in the two surveys (such as the five subquestions in Figure S-2 below). In general, the Partners felt that more had been achieved in the various areas than the Focal Points, but not significantly so. This and the relatively high response rates for surveys of this kind enhance the credibility of the evaluation’s findings. Major Achievements 9. Both the relevance and the effectiveness of the MDTF have been highly dependent on the relevance and effectiveness of CAADP. CAADP Partners continue to find CAADP to be highly relevant on a number of dimensions, starting with the high degree of African ownership. That CAADP went through numerous reforms during the period of MDTF operations, and is still in the process of developing its own architecture and results frameworks, obviously affected MDTF operations and performance. Nonetheless, the MDTF and associated CTF projects can point to a number of achievements in terms of outputs and outcomes. COUNTRY AND REGIONAL CAADP PROCESSES 10. When the CTF projects closed on December 31, 2015, 41 countries had signed CAADP Compacts (compared to one country in 2008), 37 countries had completed Independent Technical Reviews of their NAIPs, and 32 countries had held Business Meetings (Figure S-1). Another 15 countries had received grants for public sector investment projects from the Global Agriculture and Food Security Program (GAFSP), totaling $586 million as of 2014. Another 17 countries had completed Agriculture Public Expenditure Reviews (supported by the Gates Foundation and the MDTF), and 11 countries had conducted Joint Sector Reviews. Ten African countries have signed cooperation agreements under the New Alliance for Food Security and Nutrition, and 12 countries are participating in the Grow Africa partnership to attract and support private sector investment in Africa’s agriculture sector. 11. ECOWAS, ECCAS, and COMESA have signed their regional Compacts. ECOWAS is currently implementing its Regional Investment Plan. ECCAS has a regional investment plan in place. SADC has a Regional Compact and Regional Results Framework in place. SADC and COMESA have prepared drafts of their Regional Investment Plans. 12. The MDTF has also supported countless meetings and events and the development of virtually all the CAADP framework processes, including the preparation of the pillar framework documents, the various CAADP implementation guidelines, and most recently the Sustaining CAADP Momentum exercise, the Malabo Declaration, the CAADP Results Framework, and the Implementation Strategy and Roadmap. The preparation of the basic CAADP documents was a substantial accomplishment and has been invaluable in informing xvi country-level discussions from the perspective of the professional consensus underlying these documents. Then, without the Sustaining CAADP Momentum process that led to the 2014 Year of Agriculture activities, there likely would have been no Malabo Declaration, and CAADP could have faded into a historical event with limited current relevance. Figure S-1. Country-Level Progress Source: NPCA and Annex F. 13. Evidence from the evaluation shows that the MDTF and related CTF projects have contributed much to these achievements. ECOWAS also contributed its own resources to support the preparation of CAADP Compacts and NAIPs, while the TA Fund supported at least 3 Roundtables, 6 Independent Technical Reviews, and 8 Business Meetings in ECOWAS countries. Countries in all regions have also contributed their own resources to their CAADP processes. 14. Other factors, such as propitious timing, have also contributed to these achievements. The MDTF was established when global food prices were rising in 2007–2008 and the 2009 G8 Summit in L’Aquila, Italy, pledged to mobilize $20 billion over the next three years for sustainable agricultural development in Africa in particular. The RECs have played a major role in raising awareness, advocating, and motivating their member states to implement the CAADP process — COMESA and ECCAS largely from resources provided by the MDTF and ECOWAS from its own resources. 0 5 10 15 20 25 30 35 40 45 2007 2008 2009 2010 2011 2012 2013 2014 2015 Number of Countries Signed CAADP Compacts Business Meetings Held Independent Technical Reviews of NAIPs CAADP MDTF Effective First Four CTFs Effective CTFs Closing xvii OUTCOME-LEVEL RESULTS 15. The original Program Development Objective (PDO) in September 2008 was that “African agricultural programs and institutions at the national, regional, and continental levels are scaled up and more effective through (a) technical guidance and support, (b) political support, and (c) financial support.” The revised PDO, formally approved in January 2014, has been an “improved enabling environment for African agricultural programs and policies at national, regional and continental level,” where an improved environment is defined as “systems, structures and institutions that are: inclusive; evidence￾based; scaled up and more effective in leveraging technical, financial and political support; and more robust in identifying and delivering on priorities.” 16. The evaluation has assessed the outcomes of the MDTF and related CTF projects against the revised PDO and Results Framework because the CTF projects were formally restructured and because this restructuring took place as the result of a collaborative process among the key MDTF partners. These revisions to the PDO and the Results Framework were informed both by the Mid Term Review in 2012 and by the collective view that the initial Results Framework was not capturing the full scope of the MDTF’s contributions to CAADP. 17. More than 80 percent of the country Focal Points and more than 88 percent of the CAADP Partners responding to the surveys felt that the enabling environment for agricultural policies and programs in their country — or in the country, subregion, or region that they have worked on — had improved much or some in relation to the indicators in the MDTF Results Framework (Figure S-2). CAADP Partners felt that the improvements had been greater according to every indicator, but not significantly so. An average of 43 percent of CAADP Partners and 33 percent of Focal Points felt there had been “much improvement” across the five indicators. An average of 53 percent of CAADP Partners and 52 percent of Focal Points felt there had been “some improvement.” 18. Country Focal Points and CAADP Partners attributed the improvements to more inclusive policy making processes, training for the CAADP country teams, technical assistance, capacity building and information sharing, the preparation of the NAIPs, the independent technical reviews, and in some countries the Joint Sector Reviews. There has been a growing concurrence of the important role that the agricultural sector can and should play in the economic transformation of African countries. Both the country Focal Points and the CAADP Partners attributed the improvements most of all to the countries’ own governments and the support of Development Partners, followed by the contributions of their respective RECs, non-state actors, and AUC and NPCA. 19. Countries have so far had less success in mobilizing investment resources for their NAIPs. West African countries have made the most progress in this regard — 6 out of 11 countries with NAIPs reported that more than 50 percent of their NAIPs had been funded. Only 3 out of 24 countries (two in COMESA and one in ECCAS) reported that most of their NAIPs had been funded. There is some evidence from GAFSP that the quality of the NAIPs has increased over time. The program’s independent Technical Advisory Committee, which reviews countries’ requests for funding, has observed an increase in the technical quality scores of the NAIPs over time and a decrease in the dispersion of these scores over time. xviii Figure S-2. To what extent do you feel that the enabling environment for agricultural programs and policies in your country — or the country, subregion, or region that you have worked on — has improved in the following ways? None of the response rates between CAADP Focal Points and Partners are significantly different. 20. Most countries have now institutionalized more inclusive multi-stakeholder structures in national-level decision making. An average of 26 percent of Focal Points felt that non￾state actors (farmers’ organizations, agribusiness, and civil society organizations) were involved with “much influence” while 41 percent felt that NSAs had “some influence.” However, only 8 percent of CAADP Partners felt that NSAs had “much influence” on decision making, while another 48 percent felt that NSAs had “some influence.” Both Focal Points and Partners recommended a number of actions — on both the supply and the demand side — to bring about greater involvement and influence of NSAs. For example, when governments failed to implement consensus resolutions, or reversed policies without prior consultation, this reduced the incentives for NSAs to participate. 21. CAADP Partners felt that internal/external communications and knowledge management among CAADP stakeholders had improved significantly more than did the country Focal Points. The CAADP Partners, working at the continental and regional levels, appear to have better access to CAADP communications products than do the Focal Points working at the country level. 22. About 20 percent of CAADP Partners felt that there had been “much improvement” in the development and management of mutually beneficial CAADP partnerships in the last 0% 20% 40% 60% 80% 100% Through African policy positions advocated at regional, continental, and global levels Through the quality and utility of CAADP-related sector planning, programming and accountability mechanisms and structures Through investment plans and programs demonstrating alignment with CAADP-evidenced priority sectors Through effective national structures in agricultural policy making, as evidenced by the alignment of country-level changes with CAADP principles Through inclusive national structures in agricultural policy making, as evidenced by the meaningful participation of different stakeholders CAADP Focal Points (n=33) CAADP Partners (n=31) Percent "Much improvement" and "Some improvement" xix five years, and another 65 percent felt that there had been “some improvement.” They felt that the least improvements had been with respect to (a) the level of private sector investments into African agriculture, and (b) the level of future funding sources identified and brokered for CAADP. 23. However, these improvements are only the first steps in fully achieving agriculture’s contributions to wealth creation, food and nutrition security, and poverty alleviation, at the top (Level 1) of the CAADP Results Framework, 2015–2025. The MDTF and associated CTF projects have mostly been contributing to “strengthening systemic capacity to deliver results” — Level 3 of the Results Framework. Less than 50 percent of CAADP Partners felt that the activities they had worked on had contributed, beyond a modest degree, to the Level 2 outcomes of “agricultural transformation and sustained inclusive agricultural growth” in the form of increased agricultural productivity, increased intra-African trade, expanded value chain development, etc. It simply takes time for the many documented outputs of programs like the MDTF to contribute to these desired longer-term outcomes. This is particularly true in an economic sector like agriculture, comprised of hundreds of thousands of small farmers cultivating small plots of land, no matter how efficiently they are doing so. CAPACITY BUILDING 24. The MDTF has aimed to strengthen the capacity of continental and regional organizations in three dimensions:  Individual human capacity: Strengthening the skills of individual staff members to analyze development needs; to design and implement strategies, policies, and programs; and to monitor results.  Organizational capacity: Strengthening the organizations’ internal structures, processes, systems, staffing, and other resources to achieve each organization’s goals.  Institutional or system-level capacity: Strengthening the institutional context within which the organizations operate as intergovernmental organizations in the context of sovereign states. 25. At the individual level, the CTF projects enabled the organizations to recruit new staff to implement their activities, although the recruitment processes were often protracted. Staff were often assigned multiple roles when key positions were not filled, and staff recruited for one purpose, such as monitoring and evaluation, were often pulled into other, deemed higher priority, tasks. Individual training focused primarily on improving organizational processes such as financial management and procurement with positive results. The training of operational staff consisted primarily of mentoring and on-the-job experience. Now that the CTF projects have closed, some of the staff who were recruited have been terminated. 26. At the organizational level, the Bank’s implementation support missions to the six CTF organizations have focused broadly on enhancing the alignment between their organizational objectives and their visions, and between their organizational priorities and their objectives. The missions have focused more narrowly on ensuring acceptable standards in the CTF organizations for annual work planning, financial management, procurement, xx staffing and reporting, while highlighting actions and recommendations intended to improve the overall implementation of the CTF projects. The closing missions (at the end of 2015) rated the financial management of three CTF projects as satisfactory and one as moderately satisfactory, and the procurement experience of one CTF project as satisfactory, two as moderately satisfactory, and one as moderately unsatisfactory. (Two CTF projects have not yet been rated.) 27. At the system level, the CTF projects have clearly enhanced the capacity of AUC and NPCA to lead continental processes, and the capacity of COMESA and ECCAS to support country and regional CAADP processes, in the case of ECCAS in partnership with FAO, IFPRI, and HubRural. The MDTF, the CTF projects, and the pre- and post-Malabo activities have improved the institutional setting for these four organizations, and thereby contributed to the institutional sustainability of CAADP. The organizations now have the opportunity to build on the past to become even stronger and more relevant. The long-term vision of sustainability for the CAADP organizations and the RECs is to be funded by their member states. Improving the planning, financial management, procurement, staffing, and reporting mechanisms has been a necessary step towards building the financial confidence of the member states to systematically support the organizations. 28. The end of the MDTF has demonstrated how vulnerable some of the organizations are to vagaries of external support. In the absence of a follow-on “Malabo” facility, some Development Partners such as the EC, GIZ and USAID will no doubt continue to fund some staff positions and activities in each of the organizations, which will enable them to sustain some or many of their activities. However, this is unlikely to represent the kind of five to ten￾year capacity development strategy that the independent Mid Term Review recommended in 2011. THE WORLD BANK’S PERFORMANCE 29. The World Bank has played multiple roles in the MDTF, including (a) establishing and administering the MDTF, (b) supporting the preparation and implementation of the CTF projects; (c) mobilizing technical assistance activities financed by the TA Fund; (d) contributing to MDTF governance and management; (e) contributing to donor coordination and harmonization; and (f) facilitating linkages to the Bank’s own agricultural operations in African countries. 30. The original program document aimed to establish the CTF projects by January 2010 to provide at least 4 years of implementation before their initial closing dates of December 2013. However, this process took longer than expected (a) because the CTF organizations operated in complex institutional environments as intergovernmental organizations funded by member states and donors, (b) because the projects had to comply with a number of World Bank policies for recipient-executed grants including the preparation of strategic and operational plans, and (c) because the World Bank faced a changing institutional architecture of its own for trust funds at the time. However, an elapsed time analysis conducted for this evaluation showed that the first five CTF projects became effective more rapidly than standard Bank investment projects in the agriculture sector in African countries — taking an average of 15.5 months from concept review to effectiveness compared to 24.2 months for xxi standard Bank investment projects. The last two CTF projects for ECOWAS and SADC took longer to prepare largely due to an initial lack of interest on their part, but also because of delays associated with the two-year extension of the MDTF in 2012–13. 31. The World Bank’s own agricultural strategy for Africa has been strongly aligned with CAADP since at least 2008. Its 2008 regional strategy initially had a dual focus on (a) an immediate short-term response to the high food prices that peaked in May 2008, and (b) longer-term support for agricultural growth in alignment with CAADP. A keyword search and analysis of the World Bank’s project appraisal documents of African agricultural projects found that references to CAADP have increased over time from less than 5 percent of documents in 2006 and 2007 to an average of more than 60 percent of documents during 2012–2015, but that only about 26 projects (16 percent) were strongly aligned with CAADP in terms of building on identified CAADP processes at the country or regional level. Linkages have been strongest for regional projects and development policy operations, and where members of the Bank’s MDTF team have been task managing projects. The World Bank is also the supervising entity for six projects that have received funding from GAFSP (in Burkina Faso, Ethiopia, Rwanda, Tanzania, Togo, and Uganda), four of which are also receiving blended World Bank investment financing. DONOR COORDINATION 32. The evaluation confirms the finding of a 2014 independent assessment, commissioned by DFID on behalf of the contributing donors, that the MDTF has considerably increased coordination at different levels, particularly among African lead institutions, between sectors, and among MDTF contributing donors. The CAADP Results Framework includes an integrated reporting structure which, if implemented, will likely improve coordination even more. 33. However, common to many partnership programs, the present evaluation has found that the contributing donors have had somewhat different implicit objectives for the MDTF, which has created some tensions among them. Some seemed to be more interested in leveraging their own contributions, others in institutional capacity building, and others in achieving country-level results. Some complained about their lack of influence in the Partnership Committee with respect to the allocation of MDTF resources. As a result, some contributing donors seem more willing than others to contribute to a follow-on “Malabo” facility. 34. One of the key assumptions underlying the MDTF was that establishing an alliance of donors supporting the MDTF would foster a more unified/aligned approach towards the development of CAADP institutions and processes. Prior to the MDTF, donor support was fragmented, each donor with its own set of rules and reporting — the essential counterfactual to the MDTF. Harmonizing donor support in a single instrument aimed to reduce the transactions costs to both donors and recipients associated with multiple funding sources, procedures, and reporting requirements. 35. The evaluation concludes that this objective was only partially achieved at the level of the CTF recipient organizations. Most of the organizations continued to seek and receive xxii funding from multiple sources for their CAADP work. Some of this was from MDTF donors, some of it was core support like that from the MDTF, some of it was project-based, and some of it was in-kind. Interviews with the financial management officers on the site visits to the CTF organizations indicated that they were still managing multiple funds on multiple donor schedules. The Final Report of the System for “Catching the picture of CAADP support flows” found that AUC-DREA and NPCA were being supported by over 10 different donors. Lessons for a Follow-On “Malabo” Facility 36. CAADP is a continental framework, primarily for continental and regional initiatives, to help individual African countries reach and sustain a higher path of economic growth through agriculture-led development. As such, it has been a unique process within Africa and within the agriculture sector, with no equivalent in terms of scale or ambition in Africa or other developing regions. 37. The MDTF, in turn, has represented collective action by six contributing donors, their African partners, and the World Bank, also primarily at the continental and regional levels, to achieve objectives that each organization could not achieve, or not achieve as efficiently, on its own. As one of the contributing donors has said, the MDTF represented an opportunity to provide support to CAADP implementation at the continental and regional levels, beyond what it was already doing at the country level, and to align this support with a number of other donors. 38. The design of MDTF was, at some level, a step into the dark for the donors, African partners, and the World Bank, there being no prior blueprint for the partners to follow to support continental and regional CAADP processes. Therefore, except for the legal constraints associated with (a) the Administration Agreements between the World Bank and the contributing donors for the receipt of funds, and (b) the grant agreements between the World Bank and the recipient organizations for the disbursement of funds, the MDTF adopted a flexible “thousand flowers bloom” approach to its design and implementation. 39. Overall, the design of the MDTF worked in terms of fostering country and regional CAADP processes, and, along with the flexible approach to its implementation, produced a second chance — the Malabo Declaration. This does not mean that a follow-on facility should follow the same design as the MDTF. The evaluation draws the following lessons for the consideration of those responsible for designing a follow-on facility in which contributing donors might once again pool their financial resources to support activities, primarily at the continental and regional levels, to achieve agreed objectives over time. DIVERSITY 40. A follow-on facility needs to accommodate the diversity among both RECs and countries. ECOWAS and SADC are more coherent economic communities with more established commissions and secretariats, respectively, and with existing agriculture departments. ECOWAS is also a powerful organization with a concrete economic vision and substantial financial resources levied from member states. It opted to provide each member state with $450,000 to organize a Roundtable, sign a Compact, and prepare a NAIP. xxiii COMESA is very spread out geographically from Swaziland in the south to Egypt in the North, and to the Comoros, Madagascar, Mauritius, and the Seychelles in the Indian Ocean. COMESA did not have an existing agriculture unit before its CTF project was established, and ECCAS had an agricultural unit with only one staff member. ECCAS member states did not consider ECCAS to be a significant player on agricultural issues before the establishment of its CTF project. 41. The CTF grant instrument turned out to be more of a “one size fits all” approach than originally intended which worked better for some recipient organizations like COMESA and ECCAS than for others. Both ECOWAS and SADC found it difficult to work with this instrument, among other things, for their own bureaucratic and administrative reasons. They found it very difficult to align their own systems with those the World Bank for preparing and implementing recipient-executed activities. ECOWAS would have preferred to receive its support entirely from the TA Fund, which worked better for itself in terms of preparing its Strategic and Operational Plan and for its member countries in terms of supporting their country-level CAADP processes. SUPPLY AND DEMAND-SIDE RELEVANCE 42. A follow-on facility needs to support African voices to continually raise awareness and motivate countries to fulfill their Malabo commitments. Both the MDTF program and the CTF projects have been strongly aligned with CAADP principles and broadly defined strategies. Still, the MDTF has been largely a supply-driven program of the continental and regional organizations and the contributing donors. The Sustaining CAADP Momentum exercise, the 2014 Year of Agriculture, the Malabo Declaration, and the Implementation Strategy and Roadmap have also been supply-driven efforts, created and delivered by the organizations made more relevant by the existence of these documents, and the declarations and processes that flow from them. That these documents exist is a step forward, particularly the Malabo Declaration, since it gives national policy constituencies a document endorsed by their head of state or government from which advocate domestic policy reform. These documents have been among the crowning achievements of the AUC’s and NPCA’s involvement in the MDTF. But the existence of these documents alone cannot cause or result in better agricultural policies and investment plans in member states. 43. Although African Heads of State and Government have now renewed their commitment at the highest political level to the principles and values of the CAADP process, this does not always reflect uniform demand at the national level or translate into effective action at the national level. The Malabo Declaration views the agriculture sector and agricultural transformation as principal catalysts for economic growth and development. Yet in African countries where extractive natural resources (such as oil, diamonds, or wood) are important in the economy, their governments less frequently consider agriculture to be a strategic sector. Only 30 percent of country-level CAADP Focal Points, responding to their survey, felt that their country’s CAADP processes had “much connection and influence” on national economic policy making and budgeting. Another 30 percent felt their CAADP processes had “some connection and influence,” and the remaining 40 percent “little influence or connection.” The need for articulation around CAADP is still high, preferably by Africa’s own voices. xxiv SUBSIDIARITY 44. A follow-on facility should continue to focus its support at the continental and regional levels while providing some well-tailored support directly to countries — with the active engagement of the RECs. The evaluation found strong continuing support for the subsidiarity principle. By and large, the AUC, NPCA, and the RECs have played their roles envisaged in the CAADP guidelines, taking into account their capacity at various stages in MDTF implementation. The AUC has been the convener of continent-wide meetings as the political umbrella for CAADP implementation. NPCA has provided strategic guidance and technical backstopping as the technical agency of the AU responsible for agriculture. The RECs have encouraged and facilitated both country and regional CAADP processes. 45. However, member states would have preferred to receive direct financial support from the MDTF to undertake CAADP-related activities rather than through the RECs or the TA Fund. ECOWAS was the only REC that took the subsidiarity principle to the next level down by providing each member state with $450,000 from its own resources to organize a Roundtable, sign a Compact and prepare a NAIP. ECOWAS also negotiated funding from donor agencies to support regional technical partners in their work with member states. While recognizing this, there already are many sources of funds for agricultural planning, policy making, and investments at the country level, and an increasing number of agriculture sector donor working groups. What collective action at the continental level can do is more unique. This can support (a) AUC, NPCA, and the RECs in undertaking activities that only they are positioned to do such as developing continental and regional policies and programs, while also (b) providing support for country-level planning, policy making, and accessing investment finance. 46. The evaluation finds an even more prominent role for the RECs in the future, given their closer relationship with their member states and the emphasis on increasing intra￾African trade in the Malabo Declaration. To date, they have played an important, albeit largely non-technical role, in getting countries through their CAADP processes from Compacts to Business Meetings. They should continue to be responsible for monitoring and tracking progress at the country level in their member states. But they need more technical capacity to support the growing agricultural trade agenda and for addressing other transnational issues such as managing shared natural resources (e.g. river basins), linking producers to markets, and dealing with droughts and other natural and man-made disasters. A follow-on facility could help strengthen the REC’s technical capacities in these areas while also strengthening the engagement between AUC, NPCA, and the RECs as an essential part of the CAADP architecture. NEED FOR FOCUS 47. A follow-on facility should focus its direct country-level support on helping countries improve their agricultural policies in line with the Malabo commitment to “evidence-based planning, policy efficiency, dialogue, review, and accountability.” The CAADP agenda has expanded considerably since the MDTF was established in 2008 to embrace, among other things, a science agenda, an agribusiness agenda, tertiary agricultural education, women and youth, expanding intra-African trade, enhancing resilience, and xxv climate smart agriculture. This having been said, there is a need to focus future support on specific policy issues of particular relevance to individual countries. The Sustaining CAADP Momentum paper, which was formally endorsed by African Heads of State and Government in early 2013, put more attention on improving agricultural policies, private sector development, and knowledge management. It encouraged organizations supporting CAADP to engage more directly in strategic and technical analysis of key agricultural policies and political economy issues. 48. Supporting countries to improve their agricultural policies involves more than evidence-based research on agricultural policies in Africa countries, although this is important. It involves assisting governments in managing the process and the substance of specific policy reforms in individual countries. It is one thing to organize an inclusive stakeholder process such as the NAIP to mobilize more external resources to benefit all stakeholders. It is more difficult to organize such processes to improve policies that may have domestic winners and losers. Implementing such processes requires both financial resources and more engagement of Africa’s professional communities in universities, policy think tanks, and subregional research organizations, etc., to an extent that did not occur under the MDTF. THE PILLAR FRAMEWORKS 49. A follow-on facility could continue to support the development of continental strategies on selected topics, but these need to be organized and managed well to obtain quality products that are implementable in member states. The second component of the MDTF program — to support the development of continental and regional frameworks for each of the four CAADP Pillars — was never implemented as originally designed. Only one PLI met the requirements to receive a CTF grant — the Council of Ministers of Agriculture for West and Central African States — although a second PLI — the Forum for Agricultural Research in Africa (FARA) — did receive support from a separate Bank-administered trust fund. The AUC and NPCA decided in 2010, with the concurrence of the MDTF Partnership Committee, to take a different approach to harnessing Africa’s professional communities in support of CAADP, which led to the preparation of the Knowledge, Information, and Skills (KIS) initiative. But this initiative was never implemented due to the focus on the 2014 Year of Agriculture and follow-up activities to the Malabo Declaration. And the Malabo Declaration envisages a new organizational structure for the provision of technical support, which requires yet another rethinking of KIS. Thus, the MDTF missed an opportunity to focus CAADP processes more. 50. However, work has taken place on developing continental strategies in some Pillar￾related areas. The development of Science Agenda for Agriculture in Africa in Pillar IV and the Continental Agribusiness Agenda in Pillar II represent contrasting efforts in this regard. A follow-on facility could continue to support the development of continental strategies on topics such as small farmer development, incentives to private sector (agribusiness) investments, intra-African trade, data collection and reporting, since the latter feeds into the proposed and important peer review process. But these should not be developed by a small, relatively closed group of AUC or NPCA staff, like the way in which the Continental Agribusiness Strategy was developed. Their staff should be the conveners, not the authors, xxvi working with leading organizations in each field to convene the best team to complete each task, like the way in which the Science Agenda was developed. The follow-on facility could spend some resources for such continental level activities that are prepared for endorsement by the AU Summit and reported back to the Summit. Each effort would pull the evidence together, present it in the AU hierarchy of meetings, get it adopted by Heads of State and Government, and then feed it into the biennial peer review mechanism. This could also assist in focusing the Malabo agenda on fewer topics. MONITORING AND REPORTING 51. A follow-on facility needs to take a more strategic and committed approach to addressing the overall results agenda. The six CTF organizations have been responsible for monitoring progress in relation to their project’s Results Frameworks, and the RECs have also been responsible for monitoring the progress of CAADP implementation in their region in partnership with ReSAKSS. Monitoring and reporting have been the weakest aspects of the implementation of both the CTF projects and CAADP. Some CTF organizations failed to recruit monitoring and evaluation (M&E) specialists. Others recruited specialists who were insufficiently trained for the task, or who got drawn into other, deemed higher priority, tasks. Interviews with CTF staff revealed the absence of a results culture in their organizations, and limited demand for M&E for use in planning, since planning was based more on seeking financial resources, primarily from donors, than on monitoring past results. Member states have also experienced difficulties in recruiting M&E specialists, so that no one — not even ReSAKSS — has been able to determine systematically to what extent CAADP has contributed to the achievement of Level 2 outcomes (in the CAADP Results Framework) at the country level. 52. The appearance of two separate results frameworks towards the end of the CTF projects has also caused some confusion — one project-based framework for the MDTF in 2013 and the second program-based framework for CAADP in 2014. This is symptomatic of a common issue. While Development Partners have often expressed good intentions to coordinate and streamline their monitoring and evaluation processes with those of recipient countries, this has often proven more difficult to achieve in practice. Development partners have their own requirements for project-level M&E to satisfy their own domestic constituencies which are not necessarily consistent with the efforts of recipient countries to establish their own more sector-based M&E systems. Development partners also typically have more funds available to implement their project-based systems. There is a need to address these tensions and for Development Partners to put a lot of effort in the post-Malabo phase to align their M&E systems with the CAADP Results Framework. Experience in the health sector, for example, shows that this is more than a short-term exercise. GOVERNANCE AND MANAGEMENT 53. A follow-on facility needs more transparent and accountable governance and management arrangements. The MDTF has been governed by a small stakeholder body — the Partnership Committee — chaired by the AU and supported by the NPCA as the Secretariat and the World Bank as the Fund Administrator. The four RECs, civil society organizations, the commercial private sector, and knowledge institutions have also had one xxvii voting representative each, and the contributing donors two voting representatives. The Partnership Committee was initially accountable to the CAADP Partnership Platform, and subsequently to the AUC–DREA Commissioner and the NPCA Chief Executive Officer. 54. However, the Partnership Committee has not been a typical governing body of a partnership program (a) that exercises strategic direction and oversight of the program, (b) that is accountable for all the activities supported by the program, and (c) in which its secretariat has signing authority over the expenditures of the program. It has functioned more like an internal management committee. Its primary function has been to make recommendations to the World Bank concerning the allocation of MDTF resources. 55. The Partnership Committee has been a legitimate body composed of the primary CAADP stakeholders, in which the World Bank has endeavored to respect and sustain the African ownership of CAADP and the MDTF without compromising its legal trusteeship obligations to the contributing donors. But the work of the Partnership Committee has suffered from a lack of transparency and mutual accountability. It chose not to establish its own website independent of the CAADP website maintained by NPCA. Based on the difficult experience that this evaluation has had in assembling documentary evidence, the program should have at least established a private, “members only” website to make more accessible the most important MDTF documents, including the minutes and supporting documents of the PC meetings. INSTITUTIONAL ARRANGEMENTS 56. A follow-on Malabo facility needs to be designed for the long term, with institutional arrangements that could last a long time. While there seems to be some willingness among potential donors to support the establishment of a follow-on Malabo facility, there appears to be little support for a repeat of something close to the MDTF. While the independent Mid Term Review in 2011 found that the CTF projects had been highly appropriate instruments that supported the priority organizations, the present evaluation found less enthusiasm for proceeding with second round of CTF grants. 57. Even for the four CTF projects with satisfactory outcomes, the recipient organizations have found the administrative burden of complying with all World Bank procedures for recipient-executed grants to be high. They have appreciated the organizational strengthening that has occurred through the Bank’s implementation support, but they have questioned the necessity for the Bank to apply the same degree of oversight and compliance for these relatively small grants as for regular Bank-supported investment projects. Clearly capacity has been built in some important areas, but there are also clear limitations as to what a CTF grant can accomplish, particularly in the areas of accountability and sustainability. 58. Finally, if a follow-on Malabo facility were to provide direct support to countries to improve their agricultural policies, to access investment finance, and to establish more effective monitoring and reporting systems, then establishing CTF projects in multiple countries is obviously not practical. Some other support mechanism needs to be explored, such as block grants or competitive grants. xxviii 59. The above lessons imply that a follow-on facility should have several sub-facilities — for overall CAADP coordination, for capacity building at the continental and regional levels, and for assisting countries in improving their agricultural policies and their access to investment finance — while harnessing Africa’s own professional communities for these purposes. Such a facility should not attempt to provide investment finance because this would dilute its focus and compete with already existing facilities such as GAFSP. 60. The biggest challenge will be putting in place institutional arrangements at the continental level that provide for legitimate and effective governance and management of the facility, efficient fiduciary trusteeship of the donor funds, transparent allocation of resources among activities, effective supervision of the activities supported, and effective monitoring and reporting of the results. Letting a thousand flowers bloom is a useful approach to begin with, if only to find out where the flowers grow best. But eventually things need to be institutionalized to bring more order and discipline to any process that wants to achieve sustainable long-term results. 1 1. Introduction to the Evaluation 1.1 The Comprehensive Africa Agriculture Development Programme (CAADP) — which was first launched at the African Union Summit in Maputo, Mozambique, in 2003 — is the AU’s vision and strategy for the development of African agriculture. It is a framework for advocacy and action crafted through extensive consultation with a broad range of stakeholders. Its goal is to help African countries reach and sustain a higher path of economic growth through agricultural-led development that reduces hunger and poverty and enables food and nutrition security and growth in exports through better strategic planning and increased investment in the sector. 1.2 As of 2007, however, only one country (Rwanda) had formally held a Roundtable meeting and signed a CAADP Compact (in March 2007) — among the government, AU/NEPAD, COMESA, Development Partners, the private sector, and civil society — to collectively pursue the Government of Rwanda’s vision of a “private sector led underpinned by public sector investment, market oriented agriculture as a key element of its strategy to achieve faster, broad based growth.”1 1.3 Consequently, at the 2nd meeting of the CAADP Partnership Platform (CAADP PP) in Addis Ababa in September 2007, the AU called for the establishment of a multi-donor trust fund (MDTF) to be administered by the World Bank as a mechanism through which financial support from Development Partners could be channeled to accelerate regional and country￾level CAADP processes. The African Union and key donors felt that the World Bank, which was already providing financial assistance in many areas of CAADP, had the administrative capacity to manage the fund and the professional capacity to provide technical leadership. This call was reiterated at the November 2007 meeting of the Africa Partnership Forum in Algiers and the CAADP MDTF was formally established in September 2008 (Box 1). 1.4 Six donor partners — the European Union, USA, UK, Netherlands, Ireland, and France — have since contributed $65.2 million to the MDTF as of December 31, 2015. About $28.7 million (44 percent) of the funds have been “recipient-executed” by seven African continental and regional organizations, and $27.5 million (42 percent) have been executed directly by the World Bank, leaving $8.9 million (14 percent) unspent as of December 31, 2015. About $7.3 million of the $8.9 million represented unspent commitments to the recipient organizations. 1.5 Administratively, the World Bank has established a recipient-executed Child Trust Fund (CTF) grant for each of the seven organizations listed on page ix. Each of the CTF grants is a World Bank “project” with its own project paper that has been identified, prepared, appraised, and approved according to standard World Bank procedures for recipient-executed activities (whether grants, loans, or credits). Bank staff have also provided implementation support to the grantees, after each grant has become effective, according to standard Bank procedures. 1. Rwanda CAADP Compact, Kigali, March 31, 2007, p. 3. 2 1.6 Each of these recipient-executed grants has now closed, and there are currently ongoing discussions among the MDTF partners and other CAADP stakeholders about establishing a follow-on “Malabo Facility” to build upon the achievements of the CAADP MDTF — the term “Malabo Facility” coming from the 2014 Malabo Declaration in which African heads of state and government, meeting in Malabo, Equatorial Guinea, on June 26– 27, 2014, recommitted themselves to the principles and values of the CAADP process. Box 1. The CAADP MDTF: A Confusing Name for the Program The Program Document that was published on November 10, 2008, refers to the program as a “Multi-Donor Trust Fund to Support the Comprehensive African Development Programme (CAADP)” and has come to be called the CAADP MDTF for short. This has been a confusing name for several reasons. First, it has identified the program by the financial instrument that supports it. Trust funds are not programs in themselves, but financial vehicles for channeling aid resources from governmental and non-governmental donors to programs and activities agreed between the donor(s) and a trustee organization such as the World Bank.a Second, the World Bank administers more than 50 multi-donor trust funds that are supporting global or regional partnership programs, including five other MDTFs that are supporting CAADP processes — namely, for the African Forum for Agricultural Advisory Services (AFAAS), for the Association for Agricultural Research in Eastern and Central Africa (ASARECA), for the Centre for Coordination of Agricultural Research and Development for Southern Africa (CCARDESA), for the Forum for Agricultural Research in Africa (FARA), and for the West and Central Africa Council for Agricultural Research and Development (CORAF). Third, the program has been supported by one MDTF (for contributions from the EU, France, Ireland, Netherlands, and the UK), and one single-donor trust fund (SDTF, for contributions from USAID). It is not always clear whether references to MDTF are to the program as a whole or to one of the two parent trust funds that are supporting the program. Fourth, it represents bad communication. While insiders may know that the CAADP MDTF is not just a trust fund but also a regional partnership program with a governance, management, and administration structure, well-meaning outsiders can be justifiably confused and give up supporting the program rather than making the additional effort required to comprehend what is going on. However, this train left the station eight years ago. Therefore, this evaluation refers to the program as the CAADP MDTF, or MDTF for short. Unless otherwise indicated, all references to “MDTF” refer to the CAADP MDTF program as a whole. It is a good thing that the ongoing discussions of a possible follow-on programs are calling this the “Malabo Facility” rather than MDTF2 — after the Malabo Declaration of June 2014. a. Independent Evaluation Group (IEG) of the World Bank, 2011, Trust Fund Support for Development: An Evaluation of the World Bank’s Trust Fund Portfolio, Overview Report, p. 8. 3 Purpose and Scope of the Evaluation 1.7 The present evaluation has two major purposes:  Accountability: To conduct an independent assessment of the achievements of the CAADP MDTF and related CTF projects to provide accountability for the resources used.  Learning lessons: To draw lessons from the experience of the CAADP MDTF and related CTF projects. 1.8 The first purpose includes assessing (a) the contribution of the MDTF to improving the enabling environment for African agricultural policies and programs at national, regional and continental levels; (b) the outputs and outcomes achieved by the six organizations implementing the CTF projects; (c) the organizational effectiveness of the governance and management of the MDTF; and (d) the performance of the World Bank in the multiple roles that it has played in the MDTF, including as administrator of the MDTF. 1.9 The second purpose includes drawing lessons that might be incorporated into the objectives and design of a follow-on Malabo facility in which Development Partners continued to pool financial resources in support of CAADP implementation. Learning lessons is particularly important because the CAADP process is unique in African agriculture. Nothing like this has been tried before. There were no blueprints for how CAADP or the MDTF should operate, so that both have been an exercise in learning by doing. 1.10 The evaluation has covered the time period from the initial approval of the MDTF in September 2008 to the present day. Evaluation Design 1.11 In accordance with their grant agreements with the World Bank, each CTF organization is required to prepare a Project Completion Report (PCR) not later than six months after the closing date of each CTF project — that is, by June 30, 2016. Each World Bank Task Team Leader (TTL) is also required to complete an Implementation Completion and Results Report (ICR) for each CTF project within six months of CTF closing. 1.12 The present evaluation covers not only the CTF projects, but also the Bank-executed activities associated with the MDTF, starting with the identification and preparation of the MDTF as a whole in 2007. Nonetheless, the evaluation has been designed to complement and contribute to the above end-of-project evaluation requirements for the six CTF projects that closed on December 31, 2015. Essentially, each of three evaluation team members has been responsible for covering two of the six CTF projects, respectively, and preparing a background report on each CTF project, similar (but not identical) in scope and format to a standard ICR that will then be translated (by the World Bank’s Task Team) into a standard ICR. The main evaluation report has drawn upon these six background papers and other evaluation instruments as described below. 4 Evaluation Issues and Questions 1.13 The evaluation issues and questions have been structured around two dimensions: development effectiveness and organizational effectiveness. “Development effectiveness” is understood to comprise the OECD/DAC standards of relevance, efficacy, and sustainability. “Organizational effectiveness” is understood to comprise the proficiency of the MDTF’s governance, implementation, and administrative structures, functions, and processes in facilitating the achievement of the MDTF’s objectives in an efficient and transparent manner. This includes the monitoring and reporting on the MDTF’s activities and results. 1.14 The evaluation covers the following evaluation issues, structured around these two dimensions of development effectiveness and organizational effectiveness:  Development Effectiveness o Relevance of MDTF objectives and design o Efficacy (outputs and outcomes achieved) o Likely sustainability of outputs and outcomes achieved o Cross-cutting issues  Organizational Effectiveness o Governance, management, and administration of the MDTF o Efficiency of financial and human resources used o Monitoring and reporting of progress and results o World Bank performance in its multiple MDTF roles 1.15 These evaluation issues have been derived from generally accepted principles for development evaluation, from some specific features of partnership programs like the CAADP MDTF, and from the World Bank’s harmonized criteria for Implementation Completion Reporting (ICRs) and Independent Evaluation Group (IEG) Evaluations. The complete list of evaluation questions under these various issues can be found in Annex C (Evaluation Matrix). Evaluation Approach and Methodology 1.16 There have been no significant changes to the evaluation approach and methodology as laid out in the final version of the Inception Report, issued in October 2015. As indicated in that report, the evaluation has based its findings and conclusions on the following data collection and analysis, and the triangulation of evidence from the different sources:  Background information on the MDTF and CTF projects  Review of previous evaluations/assessments  CAADP Annual Trends and Outlook Reports prepared by the Regional Strategic Analysis and Knowledge Support System (ReSAKSS)  MDTF semi-annual Stakeholder Status Reports prepared by the World Bank  Attendance at the CAADP MDTF Retreat in Cape Town, June 2015  Self-assessment reports prepared by each of the six CTF organizations and the World Bank’s task team – focusing on their activities, outputs, and outcomes  Site visits to each of the six CTF organizations 5  Stakeholder interviews with contributing donors, other Development Partners, government officials, informed academics, non-state actors, etc.  In-depth reviews of selected activities (up to four per organization) – focusing on their relevance of objectives and design, theory of change, relevance and effectiveness of partnerships, outputs, outcomes, and likely sustainability of benefits  CAADP Focal Point and Partner surveys  Governance and management analysis  Elapsed time analysis for the preparation of the CTF projects  Keyword searches of World Bank Partnership Strategies in African countries and World Bank appraisal documents of African agricultural projects Structure of the Report 1.17 This report has six substantive chapters. Chapter 2 is a brief description of the objectives and design of the CAADP MDTF and its associated CTF projects. Chapter 3 is chronological narrative of the implementation of the MDTF, focusing on the major activities and outputs and on the key decisions that were made during the implementation process that affected the outputs. Chapters 4 addresses the development effectiveness of the MDTF and Chapter 5 the cross-cutting issues of inclusion, gender, climate smart agriculture, the agriculture science agenda, and the continental agribusiness strategy. Chapter 6 addresses the organizational effectiveness of the MDTF and Chapter 7 the performance of the World Bank in its multiple MDTF roles. Chapter 8 is a concluding chapter that draws lessons and recommendations from the experience of the MDTF. Main Limitations of the Evaluation 1.18 This is an evaluation of the CAADP MDTF against the objectives of the MDTF, not an evaluation of CAADP itself. Therefore, the evaluation has not undertaken impact assessments of individual activities or clusters of activities because such impacts go beyond the MDTF Results Framework and the PDO of an “improved enabling environment for African agricultural programs and policies at national, regional and continental levels.” 1.19 The evaluation has not assessed the extent to which MDTF/CTF activities have contributed the five Level 2 outcomes in the CAADP Results Framework 2015–2015:  Increased agricultural production and productivity  Increased intra-African regional trade and better functioning of national and regional markets  Expanded local domestic agro-industry and value chain development inclusive of women and youth  Increased resilience of livelihoods and improved management of risks in the agriculture sector  Improved management of natural resources for sustainable agriculture. (AUC and NPCA, 2015a. See also Figure 2 in Chapter 3.) 6 1.20 Neither the MDTF nor CTF projects have viewed themselves as directly or immediately accountable for achieving these Level 2 outcomes because these go beyond the PDO of an “improved enabling environment for African agricultural programs and policies at national, regional, and continental levels.” Impact assessments also use specialized quantitative methods to assess the attribution or contribution of well-defined activities to selected development impacts. Such studies require dedicated budgets and timeframes that are beyond the scope of this evaluation. 1.21 Nor has the evaluation reviewed and assessed the performance of continental CAADP institutions such as CAADP Partnership Platform (PP) and the Development Partners Task Team (DPTT). This would in any event have duplicated the recently completed review of the CAADP Partnership Architecture, commissioned by NPCA and AUC with support from GIZ, and prepared by Genevesi Ogiogio of the African Centre for Institutional Development (Ogiogio, 2016). 1.22 This having been said, the evaluation has examined the extent to which the MDTF has contributed to the institutional establishment and coordination of CAADP, since such outcomes represent value added of the MDTF to CAADP processes. Such outcomes are contained in the current MDTF Results Framework — particularly the second outcome area, “high quality and useful CAADP-related sector planning, programming and accountability mechanisms and structures” — and contribute to the achievement of the PDO of an “improved enabling environment for African agricultural programs and policies at national, regional and continental levels.” While CAADP had largely formed its philosophical approach before the MDTF was established, the MDTF has assisted CAADP in preparing many framework documents, implementation guidelines and processes. 1.23 The World Bank also administers five other multi-donor trust funds related to CAADP processes – namely, for the African Forum for Agricultural Advisory Services (AFAAS), for the Association for Agricultural Research in Eastern and Central Africa (ASARECA), for the Centre for Coordination of Agricultural Research and Development for Southern Africa (CCARDESA), for the Forum for Agricultural Research in Africa (FARA), and for the West and Central Africa Council for Agricultural Research and Development (CORAF). The evaluation has not undertaken an in-depth assessment of these related Trust Funds. Rather, the evaluation has viewed these as parallel initiatives — similar to the Global Agriculture and Food Security Program (GAFSP) and the Regional Strategic Analysis and Knowledge Support System (ReSAKSS). They are partners particularly with respect to component (2) of the CAADP MDTF on supporting the development of CAADP Pillar programs. 1.24 No member of the evaluation team has suffered from actual or perceived conflicts of interest in undertaking this evaluation. 7 2. Objectives and Design of the MDTF and Associated CTF Projects Original Objectives and Components 2.1 The CAADP MDTF has been a programmatic trust fund mechanism (a) to support African organizations in leading the adoption and utilization of CAADP processes and implementation across the continent, (b) to strengthen the capacities of these organizations to deliver this support, and (c) to enhance the alignment and coordination of Development Partners’ support to CAADP processes and African agriculture more broadly. 2.2 The initial Program Development Objective (PDO) in the original MDTF Program Document was as follows: To scale up and make more effective African agricultural programs and institutions at the national, regional, and continental levels through improved access to: (a) technical support and guidance; (b) political support; and (c) financial support (World Bank, 2008, p. 31). 2.3 The essential theory of change of the MDTF has been to strengthen the capacity of the continental and regional organizations to provide support to country-level agricultural policy making and planning. Then more countries would prioritize agriculture and put in place improved policies and investment plans. These would in turn attract both public funds and private sector investment. Then better policy and planning and increased levels of investment would generate sustained growth in the agriculture sector which would in turn deliver developmental benefits in terms of jobs, income, and food security.2 2.4 The key assumptions underlying this theory of change have been the following: (1) That broadly based agricultural growth is very effective way to create wealth and reduce poverty in agriculture-based economies in Africa. (2) That strengthening the capacity of continental and regional organizations would (a) institutionalize the CAADP processes, and (b) enable these organizations to lead/enable/encourage national governments to establish their own CAADP processes. (3) That public expenditure planning and budgeting are a very good strategic entry point to improve country-level agricultural policies and institutions. (4) That establishing an alliance of donors supporting the MDTF would foster a more unified/aligned approach towards the development of CAADP institutions and processes. (5) That establishing a multi-year funding envelope for continental and regional organizations would provide them with the predictable financing required to strengthen their institutional capacity, build their continental/regional credibility, and deliver results on the ground more effectively. 2. Adapted from the European Centre for Development Policy Management, 2014, “Independent Assessment of the Multi-Donor Trust Fund,” Discussion Paper No. 158, p. 50. 8 2.5 The counterfactual to establishing the MDTF was not the absence of external assistance from Development Partners, but assistance that would continue to be fragmented in source and focus, each donor with its own set of rules and reporting requirements. The MDTF aimed to provide less fragmented and more focused assistance, in the form of core funding to key CAADP organizations on a predictable, multi-year basis to strengthen their capacity to provide support to country and regional CAADP processes in a sustainable way. Harmonizing donor support in a single instrument also aimed to reduce the transactions costs — to both donors and recipients — associated with multiple funding sources, procedures, and reporting requirements. 2.6 The MDTF would aim to address the unmet needs of the key CAADP organizations in their efforts to carry out their respective roles and responsibilities under CAADP, as outlined in the CAADP Guide (NPCA, 2010a). It would not, however, supplant any of the existing arrangements for supporting CAADP processes at the various levels. Nor would the MDTF finance agricultural investment programs at any level, except on a limited and pilot basis. Investment programs would continue to be financed under national agricultural development budgets, with support from regular multilateral and bilateral assistance programs and global initiatives such as the Global Agriculture and Food Security Program (GAFSP).3 2.7 As designed and approved in September 2008, the MDTF had three components: (1) CAADP Support Platforms ($17.5 million over five years): To support the development and implementation of CAADP processes (especially the CAADP Partnership Platform, Country Roundtable processes, and Regional Roundtable processes) and the organizations leading these processes (especially NEPAD and the RECs). (2) CAADP Pillar Frameworks ($20.0 million): To support the development, implementation, monitoring and evaluation of CAADP pillar programs and the organizations responsible for them at continental, sub-regional, national and local levels. (3) Technical Assistance and Trust Fund Management ($12.5 million): To support the strategic management of the Trust Fund and the harmonization of the activities of Africa’s Development Partners in their support of CAADP. 2.8 Component (1) would provide support, in the form of recipient-executed CTF grants, to two continental organizations (AUC and NPCA) and to four regional economic commissions (COMESA, ECCAS, ECOWAS, and SADC) to carry out CAADP processes at the continental, regional, and national levels. 3. GAFSP is a global facility that support medium- and long-term investments in agriculture in the poorest countries in the world, premised on the experience that improving agricultural performance in low-income countries is the most effective way of reducing poverty and hunger. It is explicitly linked to CAADP processes in the case of African countries in order to benefit from the investment of African countries and regional organizations in the technical networks and procedures of the CAADP process. Funding proposals submitted by African countries and regional organizations must provide documented evidence that they have been through a CAADP or CAADP-like due diligence process, and are aligned to the four technical CAADP pillars. See www.gafspfund.org for more information. 9 2.9 Component (2) would provide support, also in the form of recipient-executed CTF grants, to designated organizations to develop continental and regional frameworks in four thematic areas:  Pillar I: Extending the area under sustainable land management and reliable water control systems  Pillar II: Improving rural infrastructure and trade-related capacities for market access  Pillar III: Increasing food supply, reducing hunger, and improving responses to food emergency crises  Pillar IV: Improving agriculture research, technology dissemination and adoption. 2.10 Component (3) would support, in the form of Bank-executed activities, the World Bank’s administration of the MDTF, technical assistance in support of CAADP processes, and the preparation, appraisal, and implementation support for the CTF grants in components (1) and (2). Component (3) would also finance Bank-executed activities in components (1) and (2) prior to the establishment the respective CTF projects. 2.11 The World Bank initiated the preparation of CTF projects for all ten continental and regional organizations in components (1) and (2) in 2009. The first four CTF projects in component (1) to AUC, NPCA, COMESA, and ECCAS, were approved in June 2010, and had similar but not identical PDOs and components to the parent MDTF. (See Annex Table B-3.) As laid out in their project papers, their major activities included:  Coordinating agriculture investment planning processes nationally and regionally  Strengthening African ownership of the policy analysis for and technical assistance to CAADP processes  Identifying and maintaining links between CAADP and strategically relevant global and regional initiatives  Designing and managing systems that monitor and evaluate the progress of CAADP  Building the organizational capacity of the teams and units that support the implementation of CAADP. 2.12 Only one CTF project was approved for component (2), namely, in June 2010 for the Conference of Ministers of Agriculture of West and Central Africa as the designated organization for Pillar II on improving rural infrastructure and trade-related capacities for market access. This CTF project closed after 1½ years, and the MDTF partners subsequently adopted a different approach to supporting the activities in component (2), which came to be called the Knowledge, Information, and Skills (KIS) initiative (NPCA and AUC, 2013b). Additional Financing and Restructuring the CTF Projects in 2013–14 2.13 The World Bank and the participating donors approved an extension of the CAADP MDTF in 2013 “to solidify gains achieved to date, scale up activities, and extend 10 implementation to address new CAADP priorities and challenges for an additional two years.”4 The Mid Term Review of the CAADP MDTF in April 2012 was instrumental in leading to this additional financing, to a formal restructuring of the first four CTF projects in MDTF component (1) in January 2014, and to the adoption of a common PDO and Results Framework across all four CTF projects to better capture the contributions that the CTF projects were making to CAADP and to better reflect the closely linked nature of the CTF interventions. The two additional CTF projects for ECOWAS and SADC, which were approved in August 2013, also adopted the same PDO and Results Frameworks. 2.14 The original MDTF PDO was formally revised in January 2014 to an “improved enabling environment for African agricultural programs and policies at national, regional and continental levels,” where an improved environment is defined as “systems, structures and institutions that are: inclusive; evidence-based; scaled up and more effective in leveraging technical, financial and political support; and more robust in identifying and delivering on priorities.” The common Results Framework also contained four intermediate results areas and four outcome indicators (Figure 1). 2.15 The present evaluation has assessed the outcomes of the MDTF and related CTF projects against the revised PDO and Results Framework in Figure 1 because the DTF projects were formally restructured and because this restructuring took place as the result of a collaborative effort among the CTF organizations, the Development Partners, the World Bank, and other CAADP stakeholders. These revisions to the Results Framework were informed not only by the recommendations of the Mid Term Review in 2012, but also by the collective view that the initial Results Framework was not capturing the full scope of the MDTF’s contributions to CAADP. So focusing on the new Results Framework also gives the evaluation a more complete understanding of the MDTF. Proposals for a Follow-on “Malabo” Facility 2.16 There have discussions among MDTF partners and other CAADP stakeholders about the possibility of renewing the MDTF for a second phase after the first MDTF closes in June 2016. Such discussions have taken place since the Mid Term Review in 2012. At the current time, while it has been decided not to establish an MDTF2 similar to the original MDTF, the contributing donors and CTF organizations have been discussing the objectives and design of a possible follow-on “Malabo Facility.” The term “Malabo Facility” comes from the 2014 Malabo Declaration of the African Union Heads of State and Government, meeting in Malabo, Equatorial Guinea, on June 26–27, 2014, which renewed their collective commitments to the principles and values of the CAADP process. 4. Memorandum from David Nielson (World Bank TTL) to Ian Bannon, Acting Director, Strategy, Operations and Regional Integration, AFCRI, November 8, 2013, “CAADP MDTF Concept Memorandum for Proposed Additional Financing and Restructuring of Recipient-Executed Child Trust Funds.” 11 Figure 1. Revised CAADP MDTF Results Framework, FY2014 Source: Constructed from the CAADP MDTF Results Framework, January 2014. CTF Financing (Recipient￾executed) Other MDTF Financing (Bank-executed) Other Donor Co-financing of CTF/MDTF Activities AUC, NPCA, RECs, and World Bank’s Own Resources Inputs Intermediate Results Areas 1. Sustainable capacity for the planning and implementation of CAADP processes and systems at continental, regional and national levels 2. Accountability and evidence-based planning and decision making at continental, regional and national levels 3. Internal/external communication and overall knowledge management among CAADP stakeholders at international, continental, regional and national levels 4. Development and management of mutually beneficial CAADP partnerships at continental, regional and national levels Activities Outputs Activities Outputs Activities Outputs Activities Outputs Outcomes Effective and inclusive national structures in agricultural policy making High quality and useful CAADP-related sector planning, programming and accountability mechanisms and structures Investment plans and programs demonstrating alignment with CAADP￾evidenced priority sectors African agricultural policy positions advocated at continental and global levels CAADP MDTF Development Objective Improved enabling environment for African agricultural programs and policies at national, regional and continental levels. 12 3. The Implementation Experience of the MDTF and CTF Projects 3.1 The chapter presents a chronological narrative of the implementation of the MDTF and related CTF projects. It focuses on the major activities and outputs of the seven organizations (AUC, NPCA, the four RECs, and the World Bank) and on the key decisions that were made which affected the implementation process.5 3.2 The implementation of the MDTF and related CTF projects has had four major phases, as follows:  From inception in September 2008 to the approval and effectiveness of the first five CTF projects with COMESA, AUC, ECCAS, CMA/WCA, and NPCA in 2010–11.  From the establishment of the first five CTF grants to the Mid-Term Review in 2011–12.  From the Mid-Term Review to the establishment of the last two CTF projects with ECOWAS and SADC, and the restructuring and additional financing of the first five CTF projects in 2013–14.  The two-year extension of the program during 2014–15 to the closing of the CTF projects on December 31, 2015. Phase 1: From Inception to the Establishment of the First Five CTF Projects 3.3 The principal focus of the first two years was the establishment of the CTF projects for both the support and pillar organizations in MDTF components (1) and (2). At the same time, an MDTF governance structure was established, and the technical assistance (TA) Fund was used to support activities that would ultimately be taken over by the CTF recipients once they were established, such as CAADP Framework processes (largely at the continental level), and national and regional CAADP processes. ESTABLISHMENT OF THE CTF PROJECTS 3.4 The World Bank administered the CTF projects like other Bank-supported projects, except that these were grants (as opposed to loans or credits), and the grants would be provided not to governments but to intergovernmental organizations, to regional organizations like CILSS (for Pillars I and III) and FARA and ASARECA (for Pillar IV), and to universities like the University of Zambia (UNZA for Pillar I) and the University of Kwazulu-Natal (UKZN for Pillar III). 5. Much of this chapter is based on the semi-annual Stakeholder Status Reports that have been produced by the World Bank’s Task Team with inputs from the CTF organizations. For ease of reading, much material has been paraphrased without direct citations. 13 3.5 The CTF grants were intended to provide core support, as opposed to project-based support, to these organizations over a multi-year period to strengthen their capacity to fulfill their roles as defined and summarized in the various CAADP Guides. These roles included, among other things, the preparation of CAADP framework documents and the provision of political and technical support to countries and regional entities going through the prescribed CAADP sequence: launching a CAADP process, stocktaking and analysis, drafting the CAADP Compact, organizing Roundtables for signing the Compacts, and preparing National Agricultural Investments Plans (NAIPs) followed by Independent Technical Reviews (ITRs) of the NAIPs and Business Meetings to raise financing for the NAIPs. Hence, one of the main pre-requisites for the establishment of a CTF grant was for each potential recipient organization to prepare a multi-year Strategic and Operational Plan (Box 2). Box 2. Strategic and Operational Plans (SOPs) CTF recipients had to prepare plans that included (a) a description of activities for financing and how these would contribute to the CAADP process, objectives and targets; (b) a results framework, including measurable progress indicators and targets; (c) a budget; (d) an implementation schedule and procurement plan; (e) a safeguards assessment; (f) a simple operational manual describing institutional and implementation arrangements including procurement and financial management procedures; (g) the organizational structure and functions proposed for implementation and coordination of the activities included in the proposal; (h) detailed procedures for procurement of goods and selection of consultants; and (i) any capacity building needed to enable the recipient institutions to carry out their roles. In addition, the Bank had to complete financial management (FM) and procurement assessments to determine that each organizations had (a) acceptable financial management arrangements to ensure that the funds were used only for the intended purposes in an efficient and economical way, ensure the preparation of accurate, reliable and timely periodic financial reports, and safeguard the entity’s assets; and (b) adequate procurement and related systems to administer procurement in general and CTF-financed procurement in particular. Source: CAADP MDTF, World Bank Status Report, September 2008 to March 2010, p. 12. 3.6 The Bank’s Task Team began organizing a number of identification, pre-appraisal, and appraisal missions to the potential recipient organizations starting in May 2009 to COMESA and SADC. The Bank found that none of the organizations had fully articulated their plans at the outset. So the MDTF had to make significant investments in assisting the organizations in preparing their SOPs. The World Bank team and consultants worked closely with the recipient organizations to develop systems for strategic and annual planning, financial management, procurement, staffing, and performance monitoring intended to strengthen each organization’s capacity to plan and prioritize resources towards clear strategic objectives. Then, after the CTFs were established, the World Bank Team would continue to provide technical support throughout the implementation support process to strengthen these various management processes. 3.7 At the 5th CAADP Partnership Platform meeting in Abuja in November 2009, the key MDTF partners also agreed to widen the process of reviewing the SOPs to enable other African institutions and Development Partners to comment and contribute to the SOPs. These 14 Joint Technical Reviews assessed each organization’s CTF proposal for technical content and consistency with CAADP mandates, identified technical or capacity gaps that needed to be addressed, and made formal recommendations to the CAADP Partnership Committee on whether the proposals should be financed. 3.8 These preparation processes revealed much diversity among the potential recipients. ECOWAS and SADC were more coherent economic communities with more established commissions and secretariats, respectively, and with already existing agriculture units. ECOWAS was also a powerful organization with a concrete economic vision which plays a significant role in peacekeeping and conflict resolution. The ECOWAS Commission had substantial financial resources levied from member states and opted to provide each member state with $450,000 to organize a Roundtable, sign a Compact, and prepare a NAIP. COMESA was much more spread out geographically from Swaziland in the south to Egypt in the North, and to the Comoros, Madagascar, Mauritius, and the Seychelles in the Indian Ocean. COMESA did not have an existing agriculture unit before its CTF project was established, and ECCAS had an agricultural unit with only one staff member. ECCAS member states did not consider ECCAS to be a significant player on agricultural issues before the establishment of its CTF project. NEPAD was in the process of formally becoming a technical agency of the AUC, renamed the NEPAD Planning and Coordinating Agency, which process contributed to delaying the effectiveness of the NPCA CTF project until April 2011. 3.9 Among the Pillar Lead Institutions (PLIs) only CMA/WCA succeeded in submitting a full CTF proposal that went through a Joint Technical Review leading to a recommendation from Partnership Committee for funding. The weaker capacity of other PLIs, as well as the complications of preparing a consolidated plan for two organizations within Pillars I and III, caused delays in developing their proposals. Given some uncertainty at the time around the role of the PLIs and a lack of consensus around the selection of institutions to fulfill the role of Pillar institutions (for Pillars I, II, and III), the AUC and NPCA requested the Bank, with the concurrence of the Partnership Committee, to halt further preparations of CTF projects for Pillars I and III. The Partnership Committee did approve support for CMA/WCA for a one￾year period — an interim measure intended to allow activity to begin while a consensus on the longer-term structure or approach to the Pillars was being worked out. For pillar IV, FARA already had access to substantial funding through a separate dedicated MDTF (the FARA MDTF) also administered by the World Bank. 3.10 The first five CTF projects (for COMESA, AUC, ECCAS, CMA/WCA, and NPCA) were formally approved by the World Bank in June 2010 and became effective between August 2010 and April 2011. MDTF GOVERNANCE 3.11 In accordance with the MDTF Program Document, an informal Interim Steering Committee (ISC) was established at the outset, while longer-term governance arrangements were being developed. The ISC consisted of the head of the agricultural unit at NEPAD, one representative each of the RECs and the Pillar organizations, one donor partner (as an observer), and the World Bank (as an observer). Beginning in November 2008, the ISC met 15 regularly, about every two weeks, usually by audio or video conference. It also held five face-to-face meetings between February and September 2009. 3.12 A formal governance structure and operational procedures were developed by a consulting firm financed by the MDTF. This proposal for the Partnership Committee and draft operations manual were formally endorsed at the 5th CAADP PP meeting in Abuja in November 2009, and the first meeting of the PC took place in April 2010, immediately prior to the 6th PP meeting in Johannesburg. As originally designed, the PC was composed of seven members and three observers representing the key CAADP constituencies — the AUC, the RECs, the Pillar Institutions, civil society, the private sector and two Development Partners as members, and NPCA, the World Bank, and the Global Donor Platform for Rural Development as observers. NPCA was designated as the Secretariat to the PC and the World Bank as the Fund Administrator. The PC has since met twice a year from 2011 to 2014, and three times in 2015 (Annex D). Its membership was revised somewhat after the Mid Term Review, as discussed below. In addition to providing broad leadership to the MDTF, the Committee’s main function has been to review and assess funding requests to the MDTF from eligible institutions engaged in CAADP processes. SUPPORT FOR CAADP FRAMEWORK PROCESSES 3.13 While the CTFs were being established, the Bank-executed component was the only MDTF component that could provide funds for CAADP processes at any level. At the continental level, the Bank-executed TA Fund provided support to AUC and NPCA to play their lead political and technical roles in developing CAADP conceptual frameworks and supporting their implementation at country and regional level. This included financial support for the final validation of the framework documents for Pillars I, II, and III in 2009, which had been developed through extensive consultative processes by expert working groups of representatives of international, continental and national organizations. (The validation process for Pillar IV had already been completed.) This also included financial support for developing the Post-Compact Guidelines, technical and peer review modalities, M&E and Mutual Accountability Frameworks, and re-conceptualizing the Pillar approaches under CAADP. The TA Fund also provided financial support for more than 20 meetings and events in 2009–10 such as CAADP PPs and meetings of Ministers of Agriculture, in addition to Roundtable and Business Meetings. (See Annex H for the list of meetings and events supported.) SUPPORT FOR REGIONAL AND NATIONAL PROCESSES 3.14 The TA Fund also provided direct financial support to 10 Roundtables, 11 Independent Technical Reviews, and 10 Business Meetings in 2009–2010 in COMESA and ECOWAS countries. It helped CAADP institutions, under the leadership of NPCA, to develop a standard methodology for reviewing NAIPs — focusing on alignment with CAADP targets, principles, and processes; coherence and consistency with long-term growth and poverty reduction objectives; embodiment of best practices as articulated in Pillar framework documents; operational quality and implementation readiness; and alignment with Compact commitments by Government, Development Partners, and other key country stakeholders. The ITRs involved multi-disciplinary teams who provided expert feedback on investment plans based on in-depth discussions at the country level. 16 SUPPORT FOR AGRICULTURE PUBLIC EXPENDITURE REVIEWS 3.15 Finally, the Interim Steering Committee approved an MDTF contribution in 2009 to a program of agriculture public expenditure reviews (AgPERs). In line with the 10 percent target in the Maputo Declaration, this program aimed to increase the level and efficiency of public expenditures on African agriculture by providing analytical support to reviewing and strengthening agriculture public expenditure programs in African countries. Financed by a grant of $4.9 million from the Bill and Melinda Gates Foundation and an $820,876 contribution from the MDTF, the program envisaged contributing to these objectives by means of the following outputs:  Country-specific reports  The placement of these analyses into expenditure program management processes through national workshops with stakeholders and direct work with Ministry of Agriculture staff  The promotion of a network of practitioners through wide dissemination of reports  The availability of guidelines and toolkits for public expenditure analysis through updated websites, and regional workshops for training and cross-fertilization.  Cross-country examination to clarify evidence of agricultural expenditure’s impact on agricultural growth and rural poverty reduction. The program would be executed by the World Bank’s Africa Region in close collaboration with NPCA. Phase 2: From the Establishment of the First Five CTF Projects to the Mid Term Review ONGOING SUPPORT TO CTF ORGANIZATIONS 3.16 The World Bank’s Task Team switched from project preparation to supporting AUC, NPCA, COMESA, ECCAS, and CMA/WCA in the implementation of their strategic and operational plans as their projects became effective. This included the implementation of their staffing and human resource capacity building plans, and helping them articulate and effectively perform their respective roles in the continental, regional, and country-level CAADP processes. The World Bank also supported ECOWAS and SADC in formulating their strategic and operational plans for eventual approval of their CTF projects. But the dominant issue that the MDTF partners faced during this phase was how best to support the CAADP Pillar functions. SUPPORTING THE PILLAR FUNCTIONS, THE SERVICE AGENCY, AND THE KNOWLEDGE, INFORMATION AND SKILLS INITIATIVE 3.17 One of Africa countries’ aspirations for CAADP has been to develop an effective system to enable Africa’s own professional communities of practice to bring their knowledge, information, and experience to support planning and policy making at the country and regional levels. The concept of pillars had been developed for this purpose, and the Pillar Lead Institutions (PLIs) had been identified to take on the challenge. However, 17 while the PLIs had made important contributions in the development of four pillar framework documents (completed in 2009), most Pillar organizations had not yet been successful in developing effective mechanisms for providing technical support to country and regional CAADP processes. 3.18 Initial consultations in 2009 had not generated any firm conclusions about how best to harness Africa’s professional communities in support of CAADP, but led to an interim terms of reference for the existing PLIs with a mandate to support country and regional CAADP processes for one year. In 2010, the MDTF partners agreed on a process to reconsider the longer-term structure and mandate for the Pillars. AUC and NPCA commissioned a comprehensive review in July 2011, using resources from the TA Fund, to identify approaches to meet the demand for knowledge, information and experience. First, a consulting firm was contracted to undertake consultations with CAADP stakeholders in September and October. Then a design team of consultants was established (a) to explore possible models to improve the effectiveness of knowledge and information systems within CAADP, and (b) to facilitate discussion and decision making around the adoption of a more comprehensive system. The initial outputs were presented at the CAADP PP Business Meeting in November 2011 and a more detailed proposal was finalized in May 2012 (Wales, Terry and Ashley, 2012). Then an operational plan for KIS was ultimately finalized in May 2013 (NPCA and AUC, 2013b). 3.19 In the meantime, neither the World Bank’s Task Team nor NPCA were able to effectively administer the growing demand for technical support from countries and RECs implementing CAADP processes. As a result, the AUC, NPCA, and the World Bank proposed to put in place an interim arrangement in the form of a “Service Agency” to facilitate the contracting and delivery of technical expertise in response to demands from the country and regional level. The agency would provide administrative and managerial support to AUC, NPCA, and the RECs in mobilizing networks of experts with global and African experience to assist with the technical requirements of country and regional Post-Compact processes. The link between the Knowledge, Information, and Skills (KIS) initiative and the Service Agency (SA) was clear at the outset; the MDTF partners viewed the Service Agency as an interim arrangement until the KIS initiative was operational. 3.20 An expression of interest (EOI) to provide this service was issued in October 2010. The Service Agency would be paid from the Bank-executed portion of the MDTF, since there was no provision for funding such an agency in either the AUC or NPCA CTF grants. Therefore, the EOI and international tender process was conducted according to World Bank procedures. Nine qualifying firms were long-listed and three were ultimately short-listed. While the subsequent Review of the Survey Agency (completed in May 2015) concluded that the tender process was conducted properly, a considerable amount of time elapsed before the tender was awarded in September 2011 to a United Kingdom firm, HTSPE Ltd. The AUC and NPCA management were surprised when the tender was not awarded to an African firm (Wales and Tawonezvi, 2015, p. 8). The Service Agency finally started operating in April 2012. 18 CLIMATE SMART AGRICULTURE 3.21 Most of the MDTF-supported activities in 2011 focused on providing support for country CAADP processes, either indirectly through the CTF projects, or directly through the TA Fund. Nonetheless, some new strategic initiatives were started in 2011, including on climate change, a science agenda for African agriculture, private sector investment, and tertiary agricultural education. 3.22 The AUC and NPCA launched an initiative in mid-2011 to integrate climate change mitigation and adaptation activities into CAADP country processes. In response to their request the MDTF financed a screening of CAADP investment plans and supported aspects of an FAO Council Meeting in Rome for African Ministers on the integration of climate change into the CAADP process. SUPPORTING A SCIENCE AGENDA FOR AFRICAN AGRICULTURE 3.23 MDTF resources supported a workshop in Dublin on June 30 and July 1, 2011, to bring African institutions and the CGIAR together to support an African science agenda for African agriculture. Attended by 30 participants from the CGIAR, FARA, ASARECA, CORAF, and other Development Partners, the workshop agreed that alignment around a common agenda for African agriculture should enhance the effectiveness of African and CGIAR research programs. It explored, among other things, how priorities arising from CAADP processes, such as African country and regional agricultural investment plans, could inform CGIAR, African regional, and country agricultural research programs, thereby leading to more relevant and higher quality research with the CGIAR System and national programs. AGRIBUSINESS, THE PRIVATE SECTOR AND NON-STATE ACTORS 3.24 Following the World Economic Forum on Africa in Cape Town on May 4–6, 2011, the AUC, NPCA, and the World Economic Forum jointly founded the Grow Africa Partnership under the CAADP Framework in 2011. Grow Africa seeks to mobilize more private sector investment in agriculture to help countries realize the potential of their agriculture sector for economic growth and job creation. Working with public and private actors to improve the investment environment in a limited number of countries initially, Grow Africa brokers collaboration between governments, international and domestic agriculture companies, and smallholder farmers in order to lower the risk and cost of investing in agriculture, and improve the speed of return to all stakeholders. MDTF resources contributed to a Grow Africa workshop which took place in Dar es Salaam on November 8-9, 2011, as a first step in this process. SUPPORTING TERTIARY AGRICULTURAL EDUCATION (TAE) 3.25 At the Ministerial Conference on Higher Education in Agriculture in Africa in Kampala in November 2010, African leaders drew attention to the need for deepening human capital in the agricultural sector. Noting the decline in external support for TAE, the mushrooming growth in enrollment in agricultural universities, and the difficulty of maintaining the quality of the education offered at these universities, they called for reform 19 and renewed investment in TAE. In response, and at the request of CAADP partners, MDTF resources have contributed to the development of a strategy for the future of TAE in Africa, to the development of tools and processes to facilitate mainstreaming TAE in CAADP planning and policy processes, and to the establishment of an agency — referred to as the Tertiary Education for Agriculture Mechanism for Africa, or TEAM Africa — to coordinate the efforts of stakeholders in this area. THE MID-TERM REVIEW OF THE CAADP MDTF 3.26 The World Bank carried out a Mid-Term Review of the MDTF between end-2011 and early 2012, together with MDTF recipients and other stakeholders. This consisted of a number of elements including an independent review of the MDTF commissioned by DFID on behalf of the contributing donors (Kingsmill, Teshome and Tembo, 2011), joint implementation support missions to the CTF organizations, an aid effectiveness review, an internal audit of MDTF management, an analysis of the MDTF/CTF Results Frameworks, a self-assessment by the World Bank’s MDTF team (World Bank, 2012), and a stakeholder workshop in Johannesburg on April 17–19, 2012, attended by nearly 50 stakeholders and partners. 3.27 The collective review process found that the MDTF was making good progress towards its program development objectives. The MDTF had been relevant in supporting numerous CAADP processes across the continent — directly or indirectly supporting the preparation of Roundtables and Compacts in 22 countries, Investment Plans in 18 countries, Technical Reviews in 17 countries, and Business Meetings in 12 countries. The CTF projects had been highly appropriate instruments that supported the priority institutions. The Bank had also been able to mobilize support quickly and flexibly through the TA Fund to fill urgent gaps while the CTFs were being prepared. 3.28 The AUC, NPCA, COMESA, and ECCAS had used CTF funding to recruit significant additional staff, but the sustainability of this approach to capacity building was questionable. The CTF projects had taken longer to get established than originally anticipated. They needed more time and more resources to achieve their development objectives. The MDTF partners should begin developing options for a finance mechanism (including a possible MDTF2) to support an expanded African rural transformation agenda. Both the governance of the MDTF and the initial Results Frameworks of the MDTF and CTFs had been deficient. And the concept of Pillar Lead Institutions had proven to be problematic, as discussed above. 3.29 The MDTF partners essentially adopted all the recommendations that emerged from the stakeholder workshop in April 2012. They agreed to undertake the following:  To develop a concept note articulating the rationale and proposal for an extension of the current MDTF  To initiate the development of an options paper that described priorities and options for future funding under CAADP. This would build on the recently initiated visioning process for the future of CAADP — what came to be called “Sustaining CAADP Momentum” exercise. 20  To review and revise the MDTF governance structure in line with the MTR findings — both implementing some immediate priority governance recommendations and developing options for a long-term governance system.  To review and update the Results Frameworks and monitoring systems at both the MDTF and CTF levels to improve their relevance and alignment to reflect the full scope of the MDTF/CTF activities and their mutual coherence. There were no recommendations arising from the MTR or workshop in relation to the Pillar frameworks or institutions, apparently because this issue was already being reviewed separately, as discussed above. 3.30 There were clearly tensions during the MTR process arising from the fact that the contributing donors commissioned their own independent review in addition to the World Bank’s self-assessment. There are still allegations that the World Bank has not accepted some of the recommendations of the independent review. But the two reviews proved to be complementary. The Bank’s self-assessment was more descriptive, providing a good narrative of the MDTF experience to date, while the independent review was more analytical. At the end of the day, the recommendations arising from the workshop at which both reviews were discussed proved to be very consequential for the remainder of the MDTF. Phase 3: From the Mid Term Review to Restructuring and Additional Finance 3.31 After the Mid Term Review, the remaining months of 2012 and 2013 were dominated by the implementation of the MTR recommendations, the “Sustaining CAADP Momentum” exercise, and the initial functioning of the Service Agency. THE TWO-YEAR EXTENSION OF THE MDTF 3.32 The contributing donors agreed in principle at the April 2012 workshop to a two-year extension of the MDTF from June 30, 2014 to June 30, 2016, implying that the four CTF projects would also be extended for two years from December 31, 2013, to December 31, 2015. The World Bank formally approved the MDTF extension in October 2012, and issued signed Administration Agreement amendment letters to the MDTF donors for countersignature in early November 2012. The extension became effective in June 2013 with the receipt of all the countersigned donor amendment letters. The donors subsequently contributed an additional $15 million, making $65 million overall. 3.33 In the meantime, the Bank started to work with the AUC, NPCA, COMESA, and ECCAS in 2012 on preparing formal restructuring papers with additional finance for approval of the Africa Region Vice-President. The process of formalizing the MDTF extension also caused further delay in the preparation of the CTF grants for ECOWAS and SADC. Their preparation had lagged mostly because of issues on the recipient side such as a low degree of ownership, involvement, and interest. Work resumed after the MDTF extension became official. The ECOWAS and SADC projects were formally approved on August 23 and 26, 2013, respectively, and became effective on October 4 and 16, respectively. 21 MDTF GOVERNANCE 3.34 The World Bank took the lead in reviewing and revising the governance arrangements to address the deficiencies identified by the Mid Term Review. The Bank presented a draft Governance Paper to the Partnership Committee meeting in Addis Ababa in January 2013. The PC reviewed the paper and agreed upon a process of further analysis and exploration. Then the Bank worked with the European Commission to agree and submit a set of recommendations to the July 2013 meeting of Partnership Committee in Accra, aiming to clarify the roles of the PC and improve its overall efficiency and effectiveness. A revised Operations Manual, reflecting the agreements reached in July was issued in October 2013. This now provided for the following membership on the PC:  One permanent member from the AUC, who serves as the Chair  One permanent member from the NPCA, as the Secretariat  One rotating member from the four RECs  One permanent member from the World Bank, as Fund administrator  Two members appointed by the contributing donors  One representative of a knowledge institution  One rotating member from civil society organizations  One rotating member from private sector firms/organizations MDTF RESULTS FRAMEWORK 3.35 The Mid Term Review had raised a number of questions about the MDTF/CTF Results Frameworks including (a) the extent to which the various Results Frameworks combined to provide a clear and coherent assessment of progress of the MDTF; (b) the degree to which there existed an effective linkage between the various levels in the Results Framework; and (c) potential gaps in coverage in relation the full scope of the MDTF and CTF activities. The Bank contracted a consulting firm (Social Development Direct) to work with the Bank team and the CTF organizations to revise and improve the Results Frameworks and develop tools and mechanisms to improve overall M&E systems. A lengthy process of consultation and consensus-building culminated in a workshop held during the 9th CAADP PP meeting in Addis Ababa in March 2013. 3.36 The key outcomes of this process were:  Adjusting the PDO and associated indicators for both the MDTF and the CTF projects to better capture the contributions that they were making to CAADP outcomes.  Identifying four new Intermediate Results Area along with a set of intermediate results indicators that were drawn from the individual CTF projects for AUC, NPCA, and the RECs.  Introducing management indicators for the Bank-executed activities, and performance management indicators for the MDTF and Bank-executed TA Fund to capture, track and develop feedback mechanisms. 22 3.37 Each of the six CTF projects now had a common PDO, four common PDO results indicators, and four common Intermediate Results Areas to facilitate aggregating their achievements into a coherent assessment of the overall achievements of the MDTF. Each of the six CTF projects adopted slightly different indicators in each of the four Intermediate Results Areas, reflecting their own specific circumstances. 3.38 The new Results Framework was first incorporated into the project papers for the ECOWAS and SADC CTF projects that were approved in August 2013. They were formally adopted for AUC, NPCA, COMESA, and ECCAS at the conclusion of their restructuring and additional financing processes in January 2014. However, the Bank and the CTF organizations began the transition to collecting data to measure progress against the new Results Framework before then. The Bank recruited a team of two consultants to assist the organizations in creating and strengthening the architecture for reporting as well as developing the tools to collect data. Then the Bank started integrating the results team into the regular implementation support missions, aiming to simultaneously establish the importance of results reporting without creating undue burdens for the organizations. “SUSTAINING CAADP MOMENTUM” AND EXPLORING FUTURE FINANCING OPTIONS 3.39 The AUC and NPCA initiated a visioning exercise in 2012, with the support of MDTF resources, called “Sustaining CAADP Momentum” to review the first ten years of CAADP implementation, to develop an agenda for the next phase that built upon the achievements to date, and to draw lessons to scale up performance and deliver more tangible and substantial results. This comprised a mix of analytical work, policy dialogue and advocacy around moving the CAADP agenda forward. It sought to examine the drivers which influenced implementation of CAADP and the delivery of results. It also examined the implications of global developments that impacted both positively and negatively on Africa’s agricultural growth potential. 3.40 The exercise concluded that the CAADP vision was just as valid as it had been in 2003, and that significant progress had been made in building systems and capacity for planning, prioritization, and formulation of investment plans. It recommended moving beyond the initial areas of CAADP focus, among other things, to more focus on improving agricultural policies, private sector development, and knowledge management. It encouraged organizations supporting CAADP to engage more directly in strategic and technical analysis of key agricultural policies and political economy issues. The Sustaining CAADP Momentum paper, which was formally endorsed by AU Heads of State and Government in early 2013, had immediate implications for the future forms of financial support for CAADP implementation after the MDTF closed. 3.41 Pursuant to the MTR recommendation, the Bank drafted a paper outlining potential options for future support to CAADP, which was presented to the Partnership Committee meeting in Accra in July 2013. Drawing heavily upon the strategic direction outlined in the Sustaining CAADP Momentum paper, the paper explored the pros and cons of financing CAADP processes vs. investment mechanisms; the opportunities for enhancing voice and accountability via private sector, civil society and other non-state actors; and approaches to building the capacity of CAADP institutions. While the PC did not formally endorse the 23 paper, because this did not sufficiently address a number of issues relating to the options for financing CAADP, the PC requested the Bank to draft a concept note outlining the design of a second phase MDTF. The Bank did draft such a note which formed the basis for subsequent consultations about a potential MDTF2. SERVICE AGENCY 3.42 The Service Agency became operational in April 2012, to provide administrative and contracting services in the management and delivery of expert support to CAADP processes. Initial work supported by experts contracted by the Service Agency included the preparation of investment plans, technical reviews, the “Sustaining CAADP Momentum” initiative, and the finalization of the KIS Concept Note. With each experience, the process of identifying and responding to technical demands at the national level were steadily refined and improved. 3.43 The initial contract provided for $2 million worth of services to be delivered over a period of two years. As the Service Agency demonstrated its effectiveness, the demand for its services rose sharply and it became necessary to add additional resources to its contract based on projected demand. The Bank extended the Service Agency contract to the end of June 2014 with a second round of additional financing in the amount of $900,000, to ensure that there was adequate time and resources in place to support the transition to an African-based service agency. 3.44 With the impending operationalization of KIS, the Bank drafted an exit strategy to manage the transition period. In recognition of the need to carefully manage this transition and of the potential lessons for KIS implementation, the Bank undertook a review of the Service Agency focusing on its overall progress, key lessons learned, and the utility of the mechanism, which was completed in September 2013. Based on the achievements of the Service Agency and on the demonstrated value of outsourcing specific functions for greater efficiency and effectiveness, the report concluded that the Agency should continue to be utilized, but with a number of modifications to ensure greater clarity, transparency, and accessibility. A SECOND INDEPENDENT REVIEW OF THE MDTF 3.45 In 2013, DFID commissioned an independent assessment of the MDTF on behalf of the contributing donors, which was issued in February 2014. This review, which was led by the European Centre for Development Policy Management (ECDPM), focused on three major questions:6  The extent to which the MDTF was building the capacity of the continental and regional organizations to support the implementation of CAADP.  The extent to which CAADP implementation support through MDTF-sponsored organizations was contributing to changes in agricultural policy making and planning in African countries? 6. European Centre for Development Policy Management (ECDPM), the Laboratoire d'Analyse Régionale et d'Expertise Sociale (LARES) and the Economic and Social Research Foundation (ESRF), “Independent Assessment of the CAADP Multi-Donor Trust Fund,” February 2014. While this was called an “assessment”, it was in effect the second such review commissioned by DFID after the first one by William Kingsmill et al. in 2011. 24  The extent to which the MDTF was improving alignment and coordination in CAADP support. 3.46 First, the assessment found that capacity building had focused mainly on recruiting staff and on enhancing the organizational and financial management capacity of the CAADP organizations. Building capacities in knowledge management and policy and strategic analysis had received less attention. Second, the assessment found that MDTF-supported activities had played a major role in raising awareness, putting agriculture at the center of African economic growth and food security, and providing an open forum for discussion on agricultural issues at continental, regional and national levels. However, progress at the national level had been mixed, to a large extent due to domestic political economy issues. Third, the assessment found that the MDTF had only modestly improved alignment in CAADP support. The MDTF had increased coordination among the AUC, NPCA, and the RECs, and among the MDTF contributing donors, but there remained limited alignment of broader CAADP support to actual investment priorities identified in national and regional investment plans. (See also Annex Table E-2, for a more extensive summary of the assessment’s findings, conclusions, and recommendations.) 3.47 These findings had little impact on the remaining two years of MDTF implementation, primarily due to its timing, coming at the end of two years in which the MDTF had been formally extended, the final two CTF projects approved, and the other four projects restructured with additional financing. However, the present evaluation largely agrees with findings, and the findings are still very relevant for the current discussions regarding the objectives and design of a follow-on Malabo facility. Phase 4: The Two-Year Extension Phase, 2014–2015 3.48 The last two years of MDTF implementation were dominated by the 2014 Year of Agriculture and Food Security in Africa, the Malabo Declaration in June 2014, the finalization of the CAADP Results Framework, and the preparation of the Implementation Strategy and Roadmap and Program of Work to implement the Malabo Declaration. On the other hand, the MDTF partners failed to reach a consensus on options for future financing of CAADP, and the implementation of the KIS program was delayed, so that the Service Agency continued to function much as before. The ECOWAS and SADC CTF projects also suffered implementation delays. And the first four CTF organizations became increasingly concerned about the sustainability of their capacity and their activities with the closing of the CTF projects in December 2015. 2014 YEAR OF AGRICULTURE AND FOOD SECURITY 3.49 In July 2012, the AU Heads of State and Government declared 2014 to be the Year of Agriculture and Food Security to mark the 10th anniversary of the adoption of CAADP. Officially launched at the AU Summit on January 21, 2014, this set in motion a large series of meetings and events leading to the Malabo Declaration in June 2014. The TA Fund earmarked $1.1 million to the Service Agency to support these meetings and events. This proved to be a convenient mechanism for channeling funds from the MDTF for the Year of Agriculture and other events such as the Post-Malabo meetings and Partnership Platform meetings. 25 3.50 The 10th meeting of CAADP Partnership Platform in Durban in March 2014 set up 9 thematic groups to establish goals, targets and actions for the next decade to accelerate the transformation of agriculture on the continent within the CAADP framework. Then these were summarized into 5 sub-thematic papers and adopted by the Joint Conference of Ministers of Agriculture, Rural Development, Fisheries and Aquaculture meeting in Addis Ababa in May that covered 7 areas and adopted 90 percent of the 63 actions proposed. The Ministerial decisions were reviewed by the Heads of State who adopted a new set of actions and commitments that became the June 2014 Malabo Declaration. THE MALABO DECLARATION 3.51 The Heads of State and Government of the African Union renewed their commitment to the principles and values of the CAADP process in the “Malabo Declaration on Accelerated Agricultural Growth and Transformation for Shared Prosperity and Improved Livelihoods” in Malabo, Equatorial Guinea, on June 27, 2014. The Malabo Declaration, which aimed to revitalize CAADP as the overarching framework for African agricultural development, must be regarded as one of the crowning achievements of the CTF organizations, particularly AUC and NPCA. The Declaration also emphasized some additional thematic priorities around nutrition/zero hunger, gender and youth, sustainable intensification in agricultural production, intra-regional trade, private sector development, and agricultural financing. Countries should be in the lead in moving from planning to implementing investment programs that improved the quality of agricultural spending to increase agricultural growth. The AUC, NPCA and the RECs should support country initiatives, monitor progress and take leadership on those elements of the Malabo Declaration that could only be handled at the regional or continental level. 3.52 The European Union and the World Bank jointly organized a Senior Officials Meeting of Development Partners in Washington, DC, on October 11, 2014, to mobilize support for the Malabo Declaration. Meeting participants welcomed the foresight and holistic view as articulated by the African Heads of State and Government. Participants committed themselves to enhancing their collaboration with African nations in a coordinated manner. They appreciated the Declaration’s focus on mobilizing African resources and expertise. The resulting statement was subsequently transmitted to the AUC Commissioner for Rural Economy and Agriculture. With the Malabo Declaration adopted by the African Heads of State and Government and endorsed by a large number of Development Partners, CAADP is now fully immersed in a transition process. THE CAADP RESULTS FRAMEWORK 3.53 The second major product to emerge from the Sustaining CAADP Momentum exercise was the CAADP Results Framework (AUC and NPCA, 2015a). In response to a growing demand for results and impacts in the agricultural sector, the Sustaining CAADP Momentum developed the first iteration of a continental CAADP Results Framework to define a set of goals and results that should be pursued in the transformation of the agricultural sector. Then the AUC, in partnership with NPCA and ReSAKSS, convened a series of events in 2014 aimed at increasing CAADP’s focus on results and accountability, including the ReSAKSS annual conference in October 2014 which focused on tracking 26 CAADP’s key indicators and the joint meeting of NPCA and ReSAKSS in which the CAADP Results Framework was finalized. 3.54 Level 1 of the framework represents the impact-level results to which the agriculture sector aims to contribute (Figure 2). Level 2 represents the desired results of a well performing agricultural sector in terms of production and productivity, increased intra￾African trade, inclusive value chain development, increased agricultural resilience, and improved management of agricultural natural resources. Level 3 represents the level at which the MDTF has largely been contributing to strengthening institutional and human resource capacity across the agricultural sector. THE IMPLEMENTATION STRATEGY AND ROADMAP AND THE PROGRAM OF WORK 3.55 The Malabo Declaration directed the AUC and NPCA to develop an implementation strategy and roadmap (IS&R) to operationalize the Declaration. Hence, the two organizations assembled a task team for this purpose that produced a draft roadmap informed by a series of stakeholder consultations, a detailed survey among Member States, various commissioned reports, and a validation meeting in November 2014. African Heads of State and Government subsequently endorsed the IS&R, which was formally launched at a side event at the AU Summit in January 2015. Following this, AUC and NPCA organized (with support from MDTF resources) a series of workshops with African and donor partners focused on defining future modalities of operation in which countries were in the lead. Then the 11th CAADP Partnership Platform, which was held (using MDTF resources) in Johannesburg in March 2015, focused on the implementation of the IS&R. 3.56 Then, the AUC and NPCA finalized a more detailed Program of Work (POW) as a companion document to the IS&R at the semi-annual planning and review meeting of the AUC, NPCA, and RECs in February 2015. The POW identifies prioritized sets of actions to guide implementation of the IS&R at multiple levels and across sectors to promote the Malabo vision of the transformation of African agriculture. It presents the mandates of the CAADP implementing organizations at continental, regional and national levels, and proposes specific activities in each of the Strategic Action Areas in the IS&R. KIS AND THE SERVICE AGENCY 3.57 Because the AUC and NPCA had to focus most of their efforts in 2014 on the Year of Agriculture activities, the Malabo Declaration, and CAADP Results Framework, and the IS&R, and because these took precedence over KIS activities, the implementation of KIS program was delayed. However, the MDTF financed an external review of the Service Agency which was issued in May 2015 (Wales and Tawonezvi, 2015). 3.58 The review found that the Service Agency delivered 2,515 person days of TA between April 2012 and February 2015, and disbursed almost $5 million, of which $1.1 million was for managing events associated with the 2014 Year of Agriculture. The technical support delivered from the Service Agency contributed to seven NAIPs, 9 Independent Technical Reviews, and 4 Business Meetings, and to the preparation of one GAFSP project for $31.3 million in 2013. At the continental level, the Service Agency 27 Figure 2. The CAADP Results Framework: 2015–2015 28 contributed to the design of the KIS program, to the Sustaining CAADP Momentum exercise, to formulating the CAADP Results Framework, and to the IS&R for the Malabo Declaration. However, the role of the Service Agency in recruiting consultants was much less than originally envisaged. Virtually all requests for technical services — from countries, RECs, AUC, and the NPCA itself — were channeled through the NPCA. The NPCA retained control over the key tasks of (a) the formulation of TORs, (b) the selection of consultants, and (c) quality assurance. The Service Agency was simply tasked with hiring consultants as requested by NPCA, making logistical arrangements, and making payment to the consultants when cleared by NPCA. The Service Agency proved efficient in delivering technical experts on time for requested assignments despite being faced with this single channel through NPCA, difficult communications, and complicated visa and travel arrangements. On average, the Service Agency successfully deployed individuals and teams within two weeks. DESIGNING A SECOND PHASE MDTF 3.59 At the request of the Partnership Committee, the Bank’s MDTF team continued with the preparation of a potential MDTF2 to ensure continuity in support to AUC, NPCA, and the RECs. Following extensive consultations with Development Partners and agreement on the proposed design with AUC and NPCA, the Bank’s MDTF team submitted an MDTF2 proposal to an internal quality enhancement review on October 20, 2014, and then for review at a stakeholder consultation meeting attended by the RECs and countries in Addis Ababa on November 20–21, 2014. . 3.60 By early December 2014, however, the Bank suspended the preparation of a second phase MDTF at the request of several Development Partners. There was insufficient clarity on the future direction and modalities of CAADP, most specifically regarding the IS&R. While some Development Partners viewed the MDTF to be an optimal funding mechanism, in which the content could still be modified, there was also consensus that more time was needed to see how the implementation of CAADP was going to proceed post-Malabo, and for which it was not fully clear that an MDTF was the best financial instrument to support CAADP going forward. Development partners also expected the present evaluation to draw lessons from the MDTF experience for the objectives and design of a follow-on facility, what is now being called the Malabo Facility. SUSTAINING THE CTF ORGANIZATIONAL CAPACITY AFTER PROJECT CLOSINGS 3.61 Even if a follow-on facility is agreed upon, this will not become operational before 2017 at the earliest. The Bank’s MDTF team has explored a number of options to help AUC, NPCA, COMESA, and ECCAS retain staff that they had recruited with CTF resources such as providing small grants to each of these organizations for this purpose. Resources have not been the principal issue; unspent CTF funds that reverted to the parent MDTF when the six CTF projects closed on December 2015 amounted to more than $7 million. How to make the funds available administratively has been the issue. At the end of the day, the best available suggestion administratively has been to provide unspent MDTF funds to the Service Agency to recruit and retain these staff as consultants. However, this approach also requires the contributing partners to agree to a no-cost extension of the MDTF beyond June 30, 2016. 29 Overall Achievements of the MDTF 3.62 The effectiveness of the MDTF has been highly dependent on the effectiveness of CAADP. That CAADP went through numerous reforms during the period of MDTF operations, and is still in the process of developing its own architecture and results frameworks, obviously affected MDTF operations and performance. Nonetheless, the MDTF and associated CTF projects can point to a number of achievements in terms of outputs and outcomes. 3.63 When the CTF projects closed on December 31, 2015, 41 countries had signed CAADP Compacts (compared to one country in 2008), 37 countries had completed Independent Technical Reviews of their NAIPs, and 32 countries had held Business Meetings (Figure 3 and Annex F). Another 15 countries had received grants for public sector investment projects from GAFSP totaling $586 million, as of 2014 (Annex Table F-4). Another 17 countries had completed Agriculture Public Expenditure Reviews (supported by the Gates Foundation and the MDTF), and 11 countries had conducted Joint Sector Reviews (Figure 4). Ten African countries have signed cooperation agreements under the New Alliance for Food Security and Nutrition, and 12 countries are participating in the Grow Africa partnership to attract and support private sector investment in Africa’s agriculture sector.7 3.64 ECOWAS, ECCAS, and COMESA have signed their regional Compacts. ECOWAS is currently implementing its Regional Investment Plan. ECCAS has a regional investment plan in place. SADC has a Regional Compact and Regional Results Framework in place. SADC and COMESA have prepared drafts of their Regional Investment Plans. 3.65 The MDTF has also supported countless meetings and events; the development of virtually all the CAADP framework processes, including the preparation of the pillar framework documents and the various CAADP implementation guidelines; and more recently the Sustaining CAADP Momentum exercise, the Malabo Declaration, the CAADP Results Framework, and the Implementation Strategy and Roadmap. The preparation of the basic CAADP documents was a substantial accomplishment and has been invaluable in informing country-level discussions from the perspective of the professional consensus underlying these documents. Then, without the Sustaining CAADP Momentum process that led to the 2014 Year of Agriculture activities, there likely would have been no Malabo Declaration, and CAADP could have faded into a historical event with limited current relevance. 3.66 Evidence from the evaluation shows that the MDTF has contributed much to these achievements. ECOWAS contributed its own resources to support the preparation of CAADP Compacts and NAIPs. Even here, the MDTF contributed resources from the TA Fund to 7. O. Badiane and T. Makombe (eds). 2015. Beyond a Middle Income Africa: Transforming African Economies for Sustained Growth with Rising Employment and Incomes, ReSAKSS Annual Trends and Outlook Report 2014. International Food Policy Research Institute (IFPRI), p. 182. ReSAKSS — the Regional Strategic Analysis and Knowledge Support System — was established in 2006 to support efforts to promote evidence and outcome-based policy planning and implementation as part of the CAADP agenda. See www.resakss.org/about for more information. In particular, ReSAKSS provides data and related analytical and knowledge products to facilitate benchmarking, review and mutual learning processes. Based in Addis Ababa, it is facilitated by the International Food Policy Research Institute in partnership with the AUC, the NPCA, and the RECs. 30 Figure 3. Cumulative Country-Level Progress Source: NPCA and Annex F. Figure 4. Cumulative Number of Agriculture Public Expenditures Reviews and Joint Sector Reviews Source: NPCA, ReSAKSS, and Annex F. 0 5 10 15 20 25 30 35 40 45 2007 2008 2009 2010 2011 2012 2013 2014 2015 Number of Countries Signed CAADP Compacts Business Meetings Held Independent Technical Reviews of NAIPs CAADP MDTF Effective First Four CTFs Effective CTFs Closing 0 2 4 6 8 10 12 14 16 18 2011 2012 2013 2014 2015 Number of Countries Agriculture Public Expenditure Reviews Joint Sector Reviews 31 support at least 3 Roundtables, 6 Independent Technical Reviews, and 8 Business Meetings in ECOWAS countries. Countries in all regions have also contributed their own resources to their CAADP processes. 3.67 Other factors, such as fortuitous timing, have also contributed to these achievements. Global food prices rose dramatically in 2007–2008, and the countries represented at the G8 Summit in L’Aquila, Italy, on July 8–10, 2009, aimed to mobilize “$20 billion over three years through this coordinated, comprehensive strategy focused on sustainable agriculture development, while keeping a strong commitment to ensure adequate emergency food aid assistance.”8 Both the global food crisis and these commitments motivated many African governments to initiate country-level CAADP processes. They saw involvement in the CAADP process as leading to significantly more external support for investments in African agriculture. The present evaluation has not unearthed any documentary evidence that the establishment of the MDTF was motivated by the global food crisis — the two events occurred contemporaneously — but the MDTF was in place to support CAADP implementation once countries became more motivated to initiate country-level CAADP processes. 3.68 But, as discussed in this chapter, this support has not been provided precisely as expected at the outset (Table 1a). Of the total resources spent of $56.3 million, 49 percent of the funds were spent by the six CTF projects in component 1 compared to the initial budget of 35 percent; 35 percent was spent by the TA Fund compared to the initial budget of 20 percent; and 14 percent was spent on Trust Fund Management, Administration and Supervision compared to the initial budget of 5 percent. Only 2 percent of the funds were spent by the one CTF project in component 2 compared to the initial budget of 40 percent because of the decision in 2010 to take a different approach to the Pillar frameworks in the form of the Knowledge, Information and Skills (KIS) program. 3.69 However, the picture looks better if one recognizes that the TA Fund spent $6.2 million supporting national and regional processes (the object of component 1), and another $12.5 million supporting CAADP Framework Processes (the object of component 2). Furthermore, the AUC and NPCA CTF projects also supported CAADP Framework Processes at the continental level. Then the resources spent on components 1 and 2 correspond more closely to the original budgets (Table 1b). The World Bank did spend significantly more on trust fund management, administration and supervision than originally expected because the original program document had clearly under-budgeted the resources required for CTF implementation support. 3.70 The following chapters will address the extent to which these outputs have contributed to the expected outcomes. 8. G8 Summit 2009, “L’Aquila Joint Statement on Global Food Security,” p. 6. 32 Table 1. MDTF Disbursements and Expenditures a. By Financial Instrument (i.e. Recipient-Executed or Bank-Executed Trust Funds) Original Budget Share Commitments Disbursements Share Component 1: CAADP Support Platforms 17,500 35% 34,900 27,646 49% NPCA 8,000 7,665 COMESA 6,200 6,195 AUC-DREA 6,000 5,656 ECCAS 5,900 5,726 ECOWAS 4,900 500 SADC 3,900 1,903 Component 2: CAADP Pillar Frameworks 20,000 40% 1,100 1,100 2% CMA/WCA 1,100 1,100 Component 3: Component 3a: Technical Assistance 10,000 20% 19,456 35% CAADP Framework Processes and Thematic Development 12,525 Support to National and Regional Processes 6,198 Agriculture Public Expenditure Reviews 733 Component 3b: Trust Fund Management, Administration and Supervision 2,500 5% 8,080 14% CTF Implementation Support 6,666 MDTF Governance and Administration 1,414 Total 50,000 100% 56,282 100% b. By Function (i.e. Contribution to Component Objectives) Original Budget Share Disbursements Share Component 1: CAADP Support Platforms 17,500 35% 21,256 38% COMESA, ECCAS, ECOWAS & SADC CTFs 14,325 Support to National and Regional Processes a 6,198 Agriculture Public Expenditure Reviews 733 Component 2: CAADP Pillar Frameworks 20,000 40% 26,946 48% AUC and NPCA CTFs 13,321 CAADP Framework Processes and Thematic Development a 12,525 CMA/WCA 1,100 Component 3: Component 3a: Technical Assistance 10,000 20% 0 0% Component 3b: Trust Fund Management, Administration and Supervision 2,500 5% 8,080 14% CTF Implementation Support 6,666 MDTF Governance and Administration 1,414 Total 50,000 100% 56,282 100% a. The Service Agency accounted for about 49 percent ($3,021) of the TA Fund’s support to National and Regional Processes, and for about 13 percent ($1,627) of the TA Fund’s support to CAADP Framework Processes and Thematic Development. 33 4. Development Effectiveness 4.1 This chapter addresses the first major set of evaluation issues in this evaluation. It addresses the relevance and effectiveness of the MDTF, and the sustainability of the outcomes achieved. The chapter draws heavily on two surveys and a series of self￾assessments by the six CTF organizations followed by the site visits of evaluation team members to each of the organizations. 4.2 The first survey was administered to CAADP Focal Points in 47 countries, to which 35 current and former Focal Points from 32 countries responded, for a response rate of 68 percent in terms of countries. The second survey was administered to 97 CAADP Partners who have worked with one or more of the six CTF organizations at the regional and continental levels. A total of 41 Partners responded for a response rate of 42 percent. The complete results of the two surveys, including the written responses to the open-ended questions are contained in Volume 3 of this evaluation. Both surveys were administered in English and French. 4.3 Many of the survey questions were drawn directly from the revised MDTF Results Framework (Figure 1 and Annex Table B-4) that was adopted by the parent MDTF and the six CTF projects in 2013 after the Mid Term Review. The revised Results Frameworks of the six CTF projects have a common Project Development Objective, four common PDO Level results indicators, and four common Intermediate Results Areas to facilitate aggregating the achievements into a coherent assessment of the overall achievements of the MDTF. Then each of the six CTF projects adopted slightly different indicators, reflecting their own specific circumstances, in each of the four Intermediate Results Areas. The two surveys have incorporated most of the latter indicators into the survey questions because the surveys were administered continent-wide. 4.4 Ten of the questions in the two surveys asked essentially the same question, but from the different perspectives of Focal Points working at the country level and Partners working at the regional and continental levels. Triangulating these survey results, particularly with the self-assessment reports prepared by the six CTF organizations that were working with both the Focal Points and the Partners, has provided an overall assessment of the MDTF’s achievements from these three perspectives. 4.5 Overall, there were no significant differences in the distribution of survey responses between CAADP Focal Points and Partners in 41 of the 48 subquestions that were the same in the two surveys (such as the five subquestions in Figure 6 below). In general, the Partners felt that more had been achieved in the various areas than the Focal Points, but not significantly so. This and the relatively high response rates for surveys of this kind enhance the credibility of the evaluation’s findings. 4.6 As indicated in the previous chapter, one cannot attribute the following achievements only, or even mainly to the MDTF. One can only say that the MDTF has contributed to them. ECOWAS contributed its own resources to support country-level CAADP processes in its region, while the TA Fund provided support for ECOWAS’s Strategic and Operational Plan, and for Roundtables, ITRs, and Business Meetings in a number of countries. SADC has been a relative late-comer to CAADP. COMESA has provided most of the support, with help from its 34 CTF grant, for CAADP processes in the eight countries that are members of both COMESA and SADC, and also for at least one SADC country (Botswana) that is not a member of COMESA. Countries in all regions have also contributed their own resources to their CAADP processes. 4.7 Some of the RECs have also conducted similar surveys using the same software program — Survey Monkey — as this evaluation. The evaluation chose to administer its own survey continent-wide in order to obtain a continent-wide perspective of the progress that has been made. The survey responses represent the personal perspectives of key players and participants, in part because this is the nature of the revised MDTF Results Framework, requiring the subjective judgments of CAADP beneficiaries. The results do not pretend to represent hard data on such things as improvements in agricultural productivity, increases in intra-Africa trade, or improved management of natural resources for sustainable agriculture. Relevance of Objectives and Design 4.8 The evaluation has assessed the relevance of the MDTF along five dimensions, as follows. Following standard practice in development evaluation, all dimensions of relevance are assessed against current conditions, which are not necessarily that same as those which existed at the time when the program was designed.  Supply-side relevance and design — The extent to which the objectives of MDTF program and CTF projects have been aligned with CAADP principles and broadly defined strategies being advocated by CAADP.  Demand-side relevance — The extent to which the objectives of MDTF/CTF projects have been consistent with the needs and priorities of the immediate country￾level beneficiaries such as policy makers and planners.  Vertical relevance (subsidiarity) — The extent to which activities have been undertaken at the most appropriate level (continental, regional, and national) in terms of filling gaps, efficient delivery, and responsiveness to the needs of beneficiaries.  Horizontal relevance — The absence of more efficient alternative sources of supply for the same goods and services.  Relevance of design and theory of change — The extent to which the strategies and priority activities of the program have been appropriate for achieving the objectives. SUPPLY-SIDE RELEVANCE 4.9 Both the MDTF program and CTF projects have been strongly aligned with CAADP principles and broadly defined strategies, and the four regional CTF projects have been strongly aligned with each region’s agricultural strategies. Their respective objectives have been so closely aligned that the relevance of the MDTF is highly dependent on the relevance of CAADP as a whole (Box 3). This having been said, the MDTF and CTF projects have largely been contributing to the Level 3 objectives in the 2015 CAADP Results Framework, “Strengthening Systemic Capacity to Deliver Results.” The development objective of the 35 MDTF and CTF projects has been an “improved enabling environment for African agricultural programs and policies at national, regional and continental levels,” where an improved environment is defined as “systems, structures and institutions that are: inclusive; evidence-based; scaled up and more effective in leveraging technical, financial and political support; and more robust in identifying and delivering on priorities.” The MDTF and CTF projects have not viewed themselves as directly or immediately accountable for achieving the higher-order and longer-term Level 2 outcomes in the CAADP Results Framework of "agricultural transformation and sustained inclusive agricultural growth.” Box 3. The Continuing Relevance of CAADP CAADP Partners, responding to our Survey, continue to find CAADP highly relevant. The following are some representative examples of what they consider to be the major strengths of CAADP:  African-owned. A continental mandate backed by the Heads of State.  A comprehensive vision for the continent for agricultural development, poverty reduction and food security. Raising the importance of food security, nutrition, agriculture and rural development in Africa. Its power and appeal to position agriculture as the main driver behind Africa’s transformation process.  A set of common principles at the continental level. A rallying point for all stakeholders around a common agenda. Helping countries to look at themselves from a different but unique and homegrown perspective.  Its potential to galvanize political commitment and influence governments. Its potential to track progress in an aggregated way. Its potential to link up with other sectors, particularly after the Malabo Declaration, rather than looking only at agricultural production and productivity. Its huge potential in resource mobilization, including domestic resource mobilization.  The participatory and inclusive process in the design. Inculcating multi-stakeholder interactions in the pursuit of national and regional initiatives.  The emphasis on evidence as the basis for reform. Coordination and peer review exert pressure on countries to make progress on agreed-upon resolutions.  Providing a forum for knowledge sharing and learning. Supporting countries with less developed policy and planning capacity to improve the quality of their processes. Bringing together the donor community around shared goals. Source: Survey of CAADP Partners, responses to Question 5, “What do you consider to be the major strengths of CAADP.” 4.10 The MDTF has been largely a supply-driven program of the continental and regional organizations and the contributing donors. The Sustaining CAADP Momentum exercise, the 2014 Year of Agriculture, the Malabo Declaration, and the Implementation Strategy and Roadmap have also been supply-driven efforts, created and delivered by the organizations made more relevant by the existence of these documents, and the declarations and processes that flow from them. That these documents exist is a step forward, particularly the Malabo Declaration, since it gives national policy constituencies a document endorsed by their head of state or government from which to advocate domestic policy reform. Indeed, these 36 documents have been among the crowning achievements of the AUC’s and NPCA’s involvement in the MDTF. Starting with the Sustaining CAADP Momentum exercise, one activity led to another as the authors tried to turn continental level policy documents into country level action. But the existence of these documents alone cannot cause or result in better agricultural policies and investment plans in member states. DEMAND-SIDE RELEVANCE 4.11 Although African Heads of State and Government have now renewed their commitment at the highest political level to the principles and values of the CAADP process, this does not always reflect uniform demand at the national level or translate into effective action at the national level. The Malabo Declaration views the agriculture sector and agricultural transformation as principal catalysts for economic growth and development. Yet in African countries where extractive natural resources (such as oil, diamonds, or wood) are important in the economy, agriculture is less frequently considered a strategic sector. Also, when new politicians are elected, they often choose to start anew along different policy lines. 4.12 In these respects, the MDTF appears to have benefited from propitious timing. The MDTF was established when global food prices were rising in 2007–2008 and the G8 Summit in L’Aquila, Italy, pledged to mobilize $20 million over the next three years for sustainable agricultural development in Africa in particular. Still, the RECs have played a major role in the MDTF implementation raising awareness, advocating, and motivating their member states to implement the CAADP process. By and large, COMESA, ECCAS, and ECOWAS have played this role well — COMESA and ECCAS largely from resources provided by the MDTF, and ECOWAS from its own resources supplemented by assistance from the TA Fund for organizing independent technical reviews and Business Meetings. No doubt some countries were motivated by the perceived pot at the end of the rainbow in the form of more external resources for agricultural development rather than by strong beliefs in the transformative role of agricultural development. The need for articulation around CAADP is still high. A follow-on Malabo facility needs to support African voices to continually raise awareness and motivate countries to fulfill their Malabo commitments. 4.13 Also, the Sustaining CAADP Momentum paper, which was formally endorsed by African Heads of State and Government in early 2013, put more attention on improving agricultural policies, private sector development, and knowledge management. It encouraged organizations supporting CAADP to engage more directly in strategic and technical analysis of key agricultural policies and political economy issues. VERTICAL RELEVANCE: SUDSIDIARITY 4.14 Chapter 3 of the CAADP Guide sets out specific roles and responsibilities in country￾level CAADP processes for national governments, RECs, NEPAD, AUC, and the Pillar Lead Institutions in the line with the principle of subsidiarity.9 The Guide defines subsidiarity as “decision-making authority should reside at the lowest possible level — i.e. among country 9. NPCA, 2010, Accelerating CAADP Country Implementation: A Guide for Implementors. Chapter 3 is entitled “Who does what? Institutional arrangements for the CAADP implementation process.” 37 stakeholders for the actual design of the CAADP investment programme — with higher level actors providing support and guidance and intervening on regional and continental matters.” 4.15 The evaluation found strong continuing support for the principle of subsidiarity in interviews and site visits, although not a uniform agreement on what subsidiarity means. The above definition pays insufficient, only implicit attention to the issues of efficiency and economies of scale in the delivery of services and the exercise of legitimate political authority. 4.16 By and large, the evaluation found that the AUC, NPCA, and the RECs have played their roles as envisaged in the CAADP Guide, taking into account their capacity at various stages in MDTF implementation. The AUC has been the convener of continent-wide meetings as the political umbrella for CAADP implementation. NPCA has provided strategic guidance and technical backstopping as the technical agency of the AU responsible for agriculture. The RECs have encouraged and facilitated both country and regional CAADP processes. Countries have sometimes requested technical support directly from NPCA, rather than through the RECs. Eighty-one percent of CAADP Focal Points and 82 percent of CAADP Partners said that they were highly or considerably familiar with the roles of continental and regional organizations in supporting country-level CAADP processes (Volume 3, pp. 40 and 79). 4.17 Notwithstanding this, the evaluation found that many CAADP partners, both in interviews and from the survey, found this system of complementary and coordinated roles to be lacking in clarity, very costly, poorly managed, and lacking in quality communications among the continental and regional organizations. They pointed to excessive and unexpected delays by NPCA in organizing independent technical reviews of NAIPs and RAIPs, and weak overall communication and insufficient interaction between AUC/NPCA on the one hand and the RECs on the other. AUC and NPCA have addressed some of these concerns by holding semi-annual planning meetings with the RECs, staring in 2010, to foster (a) alignment and harmonization; (b) program review and associated accountability and (c) peer learning among the three levels of responsibility to support country action (AUC and NPCA, 2015c, p. 8). 4.18 Moreover, member states would have preferred to receive direct financial support from the MDTF to undertake CAADP-related activities rather than through the RECs or the TA Fund. ECOWAS was the only REC that took the subsidiarity principle down to the next level by providing each member state with $450,000 from its own resources to organize a Roundtable, sign a Compact and prepare a NAIP. ECOWAS also negotiated funding from donor agencies to support regional technical partners in their work with member states (IFPRI, FAO, CILSS, CORAF, IFDC, etc.). 4.19 While recognizing this, there already are many sources of funds for agricultural planning, policy making, and investments at the country level, and an increasing number of agriculture sector donor working groups. What collective action at the continental level can do is more unique. This can support (a) AUC, NPCA, and the RECs in undertaking activities that only they are positioned to do such as developing continental and regional policies and programs, while also (b) providing support for country-level planning, policy making, and accessing investment finance. A follow-on facility should continue to focus its support at the continental and regional levels while providing some well-tailored support directly to countries, especially in the light of the growing consensus for more support to improving country-level agricultural policies. 38 4.20 The evaluation finds an even more prominent role for the RECs in the future, given their closer relationship with their member states and the emphasis on increasing intra-African trade in the Malabo Declaration. To date, they have played an important, albeit largely non-technical role, in getting countries through their CAADP processes from Compacts to Business Meetings. They should continue to be responsible for monitoring and tracking progress at the country level in their member states. But they need more technical capacity to support the growing agricultural trade agenda and for addressing other transnational issues such as managing shared natural resources (e.g. river basins), linking producers to markets, and dealing with droughts and other natural and man-made disasters. A follow-on facility could help strengthen the REC’s technical capacities in these areas while also strengthening the engagement between AUC, NPCA, and the RECs as an essential part of the CAADP architecture. HORIZONTAL RELEVANCE: ALTERNATIVE SOURCES OF SUPPLY? 4.21 The African Union and key donors approached the World Bank in 2007 to establish and administer the MDTF because they felt that the Bank, which was already providing financial assistance in many areas of CAADP, had the administrative capacity to manage the fund and the professional capacity to provide technical leadership. The World Bank was also by far the leading trustee of global and regional partnership programs like the MDTF,10 and had the convening power to bring the MDTF stakeholders together. 4.22 While many Development Partners had been supporting CAADP in different ways for the previous four years since the Maputo Declaration, this support had been difficult to access at times, of too short duration to be programmed well, fragmented, and not harmonized. The AU wanted a longer-term program that would bring the donors together to provide core support to continental and regional organizations to strengthen their capacity to support regional and country CAADP processes. And the donors wanted someone with the African presence to effectively supervise the technical assistance provided. 4.23 Could another organization have played met these various needs? FAO, which had played a significant role in the initial establishment of CAADP, was providing technical assistance, but mostly in-kind and mostly at the country level. The International Food Policy Research Institute, which was the facilitator of the newly established ReSAKSS program (in 2006), was also engaged in strengthening capacity of African organizations, but mostly at the country level and more narrowly on evidence-based policy research. The International Fund for Agricultural Development, the United Nations Economic Commission for Africa, and the African Development Bank were other possibilities. Without gainsaying that some other organization could have met the needs, the World Bank was nonetheless a relevant choice. 10. To be the trustee does not necessarily imply responsibility for oversight and supervision of the implementation of a program’s activities. In the case of Financial Intermediary Trust Funds (FIFs) like that for the Global Fund to Fight AIDS, Tuberculosis and Malaria, the World Bank does not play an operational role in the program. In the case of other FIFs such as the Global Environment Facility and the Global Agriculture and Food Security Program, the Bank is only one of several supervising entities for individual projects. 39 RELEVANCE OF DESIGN AND THEORY OF CHANGE 4.24 Both the MDTF and the CTF projects adopted a two-pronged approach to achieving their objectives: (a) to strengthen the capacity of continental and regional organizations so that they could (b) advance, coordinate, and facilitate CAADP processes and implementation at the national and regional levels. The multi-year funding envelopes associated with the recipient-executed CTF grants were intended to provide the continental and regional organizations with the predictable financing required to strengthen their institutional capacity, build their continental/regional credibility, and deliver results on the ground more effectively. Then more countries would prioritize agriculture and put in place improved policies and investment plans. The TA Fund would also provide technical assistance to support CAADP processes, particularly before the CTF projects were up and running. 4.25 Both the CAADP Focal Points and Partners strongly agreed with three key assumptions underlying this theory of change (Figure 5). Figure 5. To what extent do you agree with the following statements? None of the response rates between CAADP Focal Points and Partners are significantly different. 4.26 As discussed in Chapter 3, the principal things that did not turn out as designed were the following:  It took longer to establish the CTF projects than expected and longer for the CTF projects to start implementing activities. The RECs had to spend more time than anticipated motivating member states to embark on the CAADP process. The two￾year extension of the closing date of the CTF projects from December 2013 to December 2015 proved crucial. 0% 20% 40% 60% 80% 100% Strengthening the capacity of continental and regional institutions is a very effective way of supporting regional and country-level CAADP processes. Public expenditure planning and budgeting are a very good strategic entry point to improve country-level agricultural policies and institutions. Broadly based agricultural growth is a very effective way to create wealth and reduce poverty in agriculture-based economies in Africa. CAADP Focal Points (n=35) CAADP Partners (n=41) Percent "Agree" and "Strongly Agree" 40  Two of the RECs were less motivated to meet the requirements to receive CTF grants to support CAADP processes in their regions.  Only one Pillar Lead Institution met the requirements to receive a CTF grant. AUC and NPCA decided, with the concurrence of the MDTF Partnership Committee, to pursue a different approach to harnessing Africa’s professional communities in support of CAADP, which led to the preparation of the Knowledge, Information and Skills initiative. But this was never implemented due the focus on the 2014 Year of Agriculture and follow-on activities.  Neither the World Bank’s Task Team nor NPCA were able to effectively administer the growing demand for technical support from countries and RECs implementing CAADP processes, so that a Service Agency was established in April 2012 to facilitate the contracting and delivery of these technical services.  The original design underestimated the difficulty of putting in place effective monitoring and evaluation systems. 4.27 Overall, the design of the MDTF worked in terms of fostering country and regional CAADP processes, and, along with the flexible approach that was adopted to its implementation, produced a second chance — the Malabo Declaration. This does not mean that a follow-on facility should follow the same design as the MDTF. 4.28 The agenda has also expanded considerably to embrace a science agenda, an agribusiness agenda, tertiary agricultural education, women and youth, expanding intra￾African trade, enhancing resilience, and climate smart agriculture. But the original design of the MDTF cannot be faulted for not including all these topics. It was primarily trying to see if the proposed architecture could be made to work. 4.29 This having been said, there is a need to focus future support on specific policy issues of particular relevance to individual countries. This involves more than evidence-based research on agricultural policies in Africa countries, although this is important. It involves assisting governments in managing the process and the substance of specific policy reforms in individual countries. It is one thing to organize an inclusive stakeholder process such as the NAIP to mobilize more external resources to benefit all stakeholders. It is more difficult to organize such processes to improve policies that may have domestic winners and losers. Implementing such processes requires both financial resources and more engagement of Africa’s professional communities in universities, policy think tanks, and subregional research organizations, etc. to the extent that did not occur under the MDTF. A follow-on facility should focus its direct country-level support on helping countries improve their agricultural policies in line with the Malabo commitment to “evidence-based planning, policy efficiency, dialogue, review, and accountability.” Outputs and Outcomes Achieved 4.30 This section assesses the efficacy of the MDTF and CTF projects in contributing to the Program Development Objective (PDO) and the four Intermediate Results Areas, as measured by the indicators specified in the aggregated MDTF Results Framework (Annex Table B-4). Of necessity, the findings draw heavily on the results of the two surveys, since 41 the majority of the indicators require the subjective judgments of CAADP beneficiaries, supplemented by the self-assessment reports, site visits, and interviews. PDO LEVEL ACHIEVEMENTS 4.31 More than 80 percent of the country Focal Points and more than 88 percent of the CAADP Partners responding to the surveys felt that the enabling environment for agricultural policies and programs in their country — or in the country, subregion, or region that they have worked on — had improved much or some in relation to all the indicators in the MDTF Results Framework (Figure 6). CAADP Partners felt that the improvements had been greater according to every indicator, but not significantly so. An average of 43 percent of CAADP Partners and 33 percent of Focal Points felt there had been “much improvement” across the five indicators. An average of 53 percent of CAADP Partners and 52 percent of Focal Points felt there had been “some improvement.” Figure 6. To what extent do you feel that the enabling environment for agricultural programs and policies in your country — or the country, subregion, or region that you have worked on — has improved in the following ways? None of the response rates between CAADP Focal Points and Partners are significantly different. 4.32 Based on their responses to the open-ended survey questions, the country Focal Points attributed the improvements to more inclusive policy making processes, training for the CAADP country teams, technical assistance, capacity building and information sharing, the preparation of the NAIPs, the independent technical reviews, and in some countries the Joint Sector Reviews (Volume 3, pp. 3–6). There has been was a growing concurrence of the important role that the agricultural sector can and should play in the economic transformation of African countries. 0% 20% 40% 60% 80% 100% Through African policy positions advocated at regional, continental, and global levels Through the quality and utility of CAADP-related sector planning, programming and accountability mechanisms and structures Through investment plans and programs demonstrating alignment with CAADP-evidenced priority sectors Through effective national structures in agricultural policy making, as evidenced by the alignment of country-level changes with CAADP principles Through inclusive national structures in agricultural policy making, as evidenced by the meaningful participation of different stakeholders CAADP Focal Points (n=33) CAADP Partners (n=31) Percent "Much improvment" and "Some improvement" 42 4.33 Both the country Focal Points and the CAADP Partners attributed the improvements most of all to the countries’ own governments and the support of Development Partners, followed by the contributions of their respective RECs, non-state actors, and AUC and NPCA (Figure 7). The country Focal Points give slightly more credit to Development Partners and the Partners give slightly more credit to the countries’ own governments, but the differences are small and insignificant. Respondents’ open-ended comments also highlighted the contributions of FAO, HubRural, and IFPRI at the country and regional levels, and emphasized the collective nature of the effort involving all stakeholders to bring about the improvements. Figure 7. To what extent do you attribute the improvements that have occurred to the following? None of the response rates between CAADP Focal Points and Partners are significantly different. 4.34 However, these improvements are only the first steps in fully achieving agriculture’s contributions to wealth creation, food and nutrition security, and poverty alleviation, at the top (Level 1) of the CAADP Results Framework, 2015-2025. The MDTF and the CTF projects have mostly been contributing to “strengthening systemic capacity to deliver results” — Level 3 of the Results Framework (Figure 8). Less than 50 percent of CAADP Partners felt that the activities that they had worked on had contributed, beyond a modest degree, to the Level 2 outcomes of “agricultural transformation and sustained inclusive agricultural growth” in the form of increased agricultural productivity, increased intra-African trade, expanded value chain development, etc. (Figure 9). It simply takes time for the many documented outputs of programs like the MDTF to contribute to desired longer-term outcomes. This is particularly true in an economic sector like agriculture, comprised of hundreds of thousands of small farmers cultivating small plots of land, no matter how efficiently they are doing so. 0% 20% 40% 60% 80% 100% To continental organizations (AUC-DREA and NPCA) To non-state actors such as farmers’ organizations To regional organizations (COMESA, ECCAS, ECOWAS, and SADC) To countries’ own governments and country￾level CAADP processes To development partners (such as the six donors and the World Bank supporting the CAADP MDTF) CAADP Focal Points (n=33) CAADP Partners (n=36) Percent "High degree" and "Considerable degree" 43 Figure 8. To what extent do you feel that the CAADP activities you have worked on have contributed to the following Level 3 results in the CAADP Results Framework? (n=34) Figure 9. To what extent do you feel that the CAADP activities you have worked on have contributed to the following Level 2 results in the CAADP Results Framework? (n=31) 0% 20% 40% 60% 80% 100% Effective and accountable institutions, including assessing implementation of policies and commitments Increased public and private investments in agriculture Increased capacity to generate, analyze and use data, information, knowledge and innovations Strengthened capacity for evidence-based planning, implementation and review Improved multi-sectoral coordination, partnerships and mutual accountability in sectors related to agriculture Effective and inclusive policy design and implementation processes High degree Considerable degree Modest degree Not at all 0% 20% 40% 60% 80% 100% Expanded local agro-industry and value chain development inclusive of women and youth Improved management of natural resources for sustainable agriculture Increased intra-African regional trade and better functioning of national and regional markets Increased resilience of livelihoods and improved management of risks in the agriculture sector Increased agricultural production and productivity High degree Considerable degree Modest degree Not at all 44 INTERMEDIATE RESULTS AREA 1: SUSTAINABLE CAPACITY FOR THE PLANNING AND IMPLEMENTATION OF CAADP PROCESSES AND SYSTEMS 4.35 The responses to this set of questions largely confirmed the progress that countries have made in signing CAADP Compacts, preparing and reviewing NAIPs, and holding Business Meetings (Figure 3 and Annex F). The one area where country Focal Points felt that there had been significantly less progress than the CAADP Partners, and where more effort was needed, was training at the national level to support CAADP implementation (Volume 3, pp. 10–11 and 54–56). This has not been for lack of effort. Respondents referred to regional workshops and training sessions in team building, policy analysis, food security, sustainable land management, gender, M&E, and many others. Such training has involved not just the public sector but also the private sector and civil society organizations in becoming leaders and champions for change in the CAADP process. The self-assessments and site visits confirmed both the amount of training that has taken place and areas where more training is needed, particularly in national-level monitoring and evaluation. 4.36 The country Focal Points exhibited less satisfaction with the immediate results of the Business Meetings and the mobilization of investment resources for their NAIPs. They had expected Development Partners to come to the Business Meetings with firm commitments of external support. Rather, Development Partners needed time to firm up their commitments based on the information received at the Business Meetings. They also found some of the early NAIPs to have unrealistic expectations relative to the capacity of countries to efficiently absorb external support. 4.37 Only 3 out of 24 countries responding to the survey said that most of their NAIP had been funded (Figure 10). Another 8 countries said that more than 50 percent of their NAIPs had been funded. West African countries have made the most progress in this regard — 55 percent of countries reporting that more than 50 percent of their NAIPs had been funded — followed by COMESA countries. Only one of the two non-COMESA SADC countries that have held Business Meetings as of December 2015 completed the survey and reported that very little of its NAIP had been funded. Countries that have successfully mobilized resources attribute their success to strong government leadership and commitment; to effective collaboration between government, Development Partners and non-state actors; and to the systematic approach of the CAADP process as advised by NPCA and the RECs. There was disappointment, however, that some Development Partners were still pursuing project-by-project approaches to development assistance in the agriculture sector rather than aligning with CAADP’s programmatic approach, and, of course, with the overall level of external support compared to the $20 billion pledge at the G8 Summit in L’Aquila. Several interviewees alleged that many Development Partners had not significantly changed the way in they operate at the country level as a result of CAADP — the World Bank included, an issue that will be investigated more thoroughly in Chapter 7. 4.38 There is some evidence that the quality of NAIPs has increased over time. The Global Agricultural and Food Security Program (GAFSP) has so far issued three calls for proposals. It initially required countries to have had an independent technical review of their NAIP to be eligible to submit proposals, which requirement was subsequently extended to having held the Business Meeting. An independent Technical Advisory Committee (TAC) has reviewed these proposals using a scoring system in which 30 percent is based on the technical quality of the 45 NAIP, 40 percent on the technical quality of the proposal, and 30 percent on need. GAFSP has found the technical quality of the NAIPs to have increased over time, based on the scores of the TAC, and the dispersion of the TAC scores to have decreased over time. Figure 10. To what extent has your country been successful in mobilizing investment resources for your National Agricultural Investment Plan? (n=24) INTERMEDIATE RESULTS AREA 2: ACCOUNTABILITY AND EVIDENCED-BASED PLANNING AND DECISION MAKING 4.39 The responses to this set of questions confirmed that most countries have institutionalized more inclusive national structures in policy making: 84 percent of country Focal Points and 97 percent of CAADP Partners said that their country, or the country they work on mostly, now maintains multi-stakeholder platforms for participation in national-level decision making (Volume 3, pp. 24–27 and 57–59). Country Focal Points generally thought that farmers’ organizations, the commercial private sector, and civil society organizations were more involved than did the CAADP Partners, although not significantly so (Figure 11). An average of 26 percent of Focal Points felt that non-state actors had “much influence” while 41 percent felt that they had “some influence.” An average of only 8 percent of CAADP Partners felt that non-state actors had “much influence, while 48 percent felt that they had “some influence.” 4.40 Both country Focal Points and CAADP Partners recommended a number of actions — on both the supply and demand side — to bring about greater involvement and influence of NSAs in national agricultural policy making. On the supply side, many said that NSAs don’t yet have the capacity to be effectively involved. There was a need to strengthen their capacity both substantively in policy analysis and procedurally in consensus-building processes. They needed more information and greater understanding of the sector. Governments also needed to do a better job of providing information and communicating 0% 20% 40% 60% 80% 100% SADC Countries, non-COMESA (n=1) ECCAS Countries (n=2) COMESA Countries (n=10) ECOWAS Countries (n=11) All Countries (n=24) Most of the plan has been funded Much of the plan has been funded (more than 50 percent) Some of the plan has been funded (less than 50 percent) Very little of the plan has been funded 46 Figure 11. To what extent are the following stakeholder groups involved in national agricultural policy making structures in your country — or the country, region, or subregion that you work on? None of the response rates between CAADP Focal Points and Partners are significantly different. with NSAs, and the NSAs with their members. NSAs needed to be well organized and structured at both the national and regional levels. 4.41 On the demand side, NSAs needed incentives to become more involved. When governments failed to implement consensus resolutions, or reversed policy without prior consultation, this reduced the incentives for NSAs to participate. Give NSAs a structural responsibility in the policy making process, such as participatory monitoring and evaluation, rather than simply consultation. Give them bigger roles in the Joint Sector Reviews, such as involvement in the planning process from the beginning. Keep the discussions focused on one or a few initiatives at a time. On the other hand, NSAs shouldn’t only be motivated by “what’s in it for me” without regard to the national interest. 4.42 Some respondents recognized that AUC, NPCA and the RECs have already done some work in this area such as preparing the CAADP Non-State Actor Strategy. AUC and NPCA needed to strengthen their linkages with PAFO (Pan-African Farmers Organization) at the continental level, and the RECs their working relationships with the regional farmers organizations (EAFF, PROPAC, ROPPA, SACAU, and UMAGRI). 4.43 It is reassuring for the CAADP process that most country Focal Points said they were closely connected to the senior policy making functions of their Ministry of Agriculture: 50 percent said that they were “involved with much influence” and another 38 percent were “involved with some influence” with some variation in the extent of involvement and influence among regions (Figure 12). It is less reassuring, but not surprising, that country￾level CAADP processes are less closely connected to national economic policy making and budgeting processes, according to the Focal Points (Figure 13). Only 30 percent said CAADP 0% 20% 40% 60% 80% 100% Civil society organizations Commercial private sector Farmers’ organizations Government agencies CAADP Focal Points (n=33) CAADP Partners (n=36) Percent "Involved with much or some influence" 47 Figure 12. How closely connected or involved are you, as the CAADP Focal Point, with the senior policy making and management functions of your Ministry of Agriculture? (n=32) Figure 13. How closely connected are your country-level CAADP processes to the national economic policy making and budgeting processes in your country? (n=33) 0% 20% 40% 60% 80% 100% SADC Countries, non COMESA (n=3) COMESA Countries (n=12) ECCAS Countries (n=6) ECOWAS Countries (n=12) All Countries (n=33) Involved with much influence Involved with some influence Some involvement but with little influence 0% 20% 40% 60% 80% 100% SADC Countries, non COMESA (n=3) ECCAS Countries (n=6) COMESA Countries (n=12) ECOWAS Countries (n=12) All Countries (n=33) Much connection and influence Some connection and influence Some connection but with little influence Little or no connection 48 processes had “much connection and influence” on national economic policy making, and another 30 percent said “some connection and influence,” again with variation among regions. This finding supports the concerns about the demand-side relevance of the MDTF and CAADP processes expressed earlier in this chapter. ReSAKSS also reports that, for Africa as a whole, the agriculture share of total public expenditure has declined in the last ten years from 3.63 percent in 2003 to 2.93 percent in 2014, again with variation among regions (Badiane and Makombe, 2015, p. 209). On average, ECOWAS countries had the highest agriculture share of public expenditures (4.61 percent) in 2014, followed by COMESA countries (3.59 percent), SADC countries (2.22 percent), and ECCAS countries (2.12 percent). The exhibits a rough correlation with the degree of influence (shown in Figure 13) that CAADP processes have on national policy making and budgeting.11 4.44 The final finding in this intermediate results area concerns country-level monitoring and evaluation. Only 43 percent of country Focal Points and 44 percent of CAADP Partners said that the their country — or the country that they have worked on — had established a functional M&E system that generates national-level reports on core CAADP indicators such as the share of agriculture in public expenditures (Volume 3, p. 57). This finding is consistent with other evidence collected. The ECOWAS CTF project supported a regional workshop, with cooperation from NPCA, in July 2015, among other things, to determine the status and trends in the implementation of countries’ M&E systems. The workshop found that only 5 of 13 countries attending the workshop had established well-functioning M&E systems, and two countries had yet to establish any system.12 4.45 In the CAADP framework the RECs have been entrusted with the mandate to monitor the implementation of the CAADP process in their respective regions in partnership with ReSAKSS. This has been designed to be implemented based on a customized M&E system consisting of a core part at the REC level with nodes at the national level. However, the RECs have reported many obstacles standing in the way of adequate implementation of this effort, including:  The absence of an M&E culture and staffing capacities in the member states as well as in the RECs.  The lack of capacity for data collection in the agriculture sector, and the resulting dearth of adequate and available data in member states.  The complexity of the ReSAKSS system given these institutional capacities and data availability. There were too many indicators for which data could not realistically be generated.  Weak collaboration with the ReSAKSS team in some regions.  Difficulties faced by the RECs and the countries in developing an M&E implementation plan using the special software that had been acquired, despite securing the services of regional M&E experts. 11. The CAADP Partnership Architecture Review (Ogiogio, 2016, p. vii) reports that nine countries (including Burkina Faso, Ethiopia, Guinea-Conakry Malawi, Mali, and Niger) had met the CAADP target of investing 10 percent of their national budgets in agriculture. 12. ECOWAS, Regional Workshop on the Joint Sector Review: Enhancing the implementation of NAIPs/NAFSIPs through alignment to the Sustaining CAADP Momentum Results Framework for accelerated implementation in ECOWAS Member States, Abidjan, July 2015. 49 4.46 Hence, no one — not even ReSAKSS — has been able to determine systematically to what extent CAADP has contributed to the achievement of Level 2 outcomes (in the CAADP Results Framework) at the country level. A follow-on facility needs to take a more strategic and committed approach to addressing the overall results agenda. INTERMEDIATE RESULTS AREA 3: INTERNAL/EXTERNAL COMMUNICATIONS AND OVERALL KNOWLEDGE MANAGEMENT AMONG CAADP STAKEHOLDERS 4.47 The responses to this set of questions exhibited some significant differences between the country Focal Points and the CAADP Partners. An average of 93 percent of CAADP Partners felt that communications and knowledge management among CAADP stakeholders had exhibited much or some improvement across the indicators in the MDTF Results Framework (Figure 14): 34 percent felt things had improved “much” and 59 percent “some”. By contrast, an average of only 19 percent of Focal Points felt that communications and knowledge management had improved “much”, and 53 percent “some”. It would appear that CAADP Partners working at the continental and regional level have enjoyed better access to CAADP communications products than Focal Points working at the country level. Indeed, country Focal Points said in their open-ended comments that the improvements have occurred more at the regional and international levels, and that more work needs to be done to enhance CAADP communication at the national and grass roots levels. Figure 14. To what extent do you feel that the internal/external communications and overall knowledge management among regional and national-level CAADP stakeholders have changed in the last five years in the following ways? ** Indicates significantly different responses between CAADP Focal Points and Partners at the 95% level of confidence, and * at the 90% level of confidence. 0% 20% 40% 60% 80% 100% Through increased diversity of information users in different sectors such as government ministries, media, and civil society ** Through the development of new tools for CAADP communication and information sharing * Through the use of CAADP communications products in agricultural policies and decision making ** Through the regular update and dissemination of the information shared Through more frequent discussions on CAADP at international and continental level fora CAADP Focal Points (n=32) CAADP Partners (n=33) Percent "Much improvement" and "Some improvement" 50 4.48 These findings probably also reflect the well-known “English speaking” bias in the operations of CAADP, which has reduced the access of francophone, lusophone, and Spanish-speaking countries. Indeed, francophone respondents to the survey recommended that greater efforts should be made to ensure that all documents are translated into French and other languages. INTERMEDIATE RESULTS AREA 4: DEVELOPMENT AND MANAGEMENT OF MUTUALLY BENEFICIAL CAADP PARTNERSHIPS 4.49 Partnerships are an important element of the MDTF Results Framework, both for implementing activities and for helping to translate their outputs into results at the country and regional levels. Indeed, the original MDTF program document puts a lot of emphasis on partnerships, stating in its very first paragraph: As a program of the African Union, [CAADP] emanates from and is fully owned and led by African governments and enjoys a broad consensus world wide on objectives, implementing processes, and partnership principles. . . . As part of the NEPAD initiative, it fully reflects NEPAD’s broad principles of mutual review and dialogue, accountability, and partnership. 4.50 The core values and principles of CAADP also put a lot of emphasis on partnerships (Box 4). In the early stages, the MDTF drew upon these values and principles to help define a number of key decisions and processes such as the governance and management of MDTF. Box 4. The Core Values and Principles of CAADP  Partnerships and alliances are fundamental to the CAADP agenda — both as a core component and in acknowledging that agriculture is a cross-cutting sector. These relationships must go beyond conventional inter-sector linkages to include comprehensive interventions with clear collaborative work arrangements, such as inter-ministerial cooperation and public–private partnerships among others. Partnerships and alliances will (i) facilitate alignment and harmonisation of development efforts between national governments and development partners, (ii) raise participation in the policy making process by farmer organisations and other stakeholders, including the private sector and (iii) enable and ease access to greater technical expertise (knowledge and skills).  Dialogue, (peer) review and mutual accountability at the national level open the door to collective responsibility and inclusive participation down to local (grassroots) structures. These principles are expected to stimulate and broaden the practice of benchmarking, mutual learning and harmonisation of national development efforts, while encouraging a greater level of trans-boundary cooperation and regional integration.  Exploitation of regional complementarities and cooperation addresses common and mutual needs and regional comparative advantages. Source: NPCA, Accelerating CAADP Implementation: A Guide for Implementers, 2010, p. 6. 4.51 The present section focuses on the implementing partner organizations of the six CTF organizations at the continental and regional levels that have contributed to MTDF-supported activities. The evaluation has assembled a list of such partners from the self-assessment 51 reports and the site visits to the CTF organizations, which is contained in Annex I, along with their roles or the activities in which they were involved. This list does not include the contributing MDTF donors, country Focal Points, or individual consultants contracted to undertake work for the CTF organizations. 4.52 This list reveals that the principal partner organizations have been international organizations such as FAO, IFPRI/ReSAKSS, UNCCD, and IFAD; regional and subregional research organizations such as FARA, ASARECA, CCARDESA, CORAF, CILSS, and HubRural; and continental and regional farmers’ organizations such as PAFO, EAFF, ESAFF, ROPPA, and UMAGRI. GIZ has also been a major financial and technical partner to several CTF organizations. 4.53 The CAADP Partners responding to our survey felt that there had been some improvement in the development and management of mutually beneficial partnerships in the last five years in relation to the indicators in the MDTF Results Framework (Figure 15). An average of 21 percent of the Partners felt that there had been “much improvement” and another 65 percent “some improvement”. They felt that the least improvements had been with respect to the level of private sector investments into African agriculture and the level of Figure 15. To what extent do you feel that the development and management of mutually beneficial partnerships at the regional and national levels have changed in the last five years (since 2010) in the following ways? 0% 20% 40% 60% 80% 100% The level of future funding sources identified and brokered for CAADP The level of private sector investments into African agriculture More effective CAADP governance through greater coordination and collaboration among continental and regional organizations The number of action points across different stakeholders agreed upon and implemented as tabled at the Partnership Platform The level of alignment of partner support to CAADP processes The number of partnerships established for effective technical support and joint assistance The number of partnership events efficiently and effectively delivered. The number of African agricultural policy position papers jointly agreed at CAADP Governance fora Much improvement Some improvement No change Some deterioration 52 future funding sources identified and brokered for CAADP. In their open-ended comments, they felt that there might have been too many parallel initiatives at the continental level, and that the current structure of partnerships needed to be assessed, revitalized, sharpened, and dovetailed with national governmental systems. 4.54 Thirty percent of the respondents said they have been “very satisfied” working as a partner on CAADP activities, and another 55 percent as “satisfied”. Twenty-two percent rated the overall quality of their organization’s partnership with CAADP activities as “much better” in comparison with the activities of other organizations that they have worked with, and another 51 percent as “better” (Volume 3, pp. 70–73). However, they also expressed frustration at the excessive focus at the continental and regional levels, at the continuing mistrust between the public and private sector partners, and the poor coordination of Development Partners at the national level. They were frustrated with the bureaucratization of everything, while agreeing with the goals. They found the lack of institutional and human resource capacity on the CTF organizational side to be a challenge — also a finding of the site visits to the CTF organizations. 4.55 Twenty-one percent of the respondents felt they have enhanced “to a high degree” the relevance and effectiveness of the CAADP activities that they have worked on, and 43 percent “to a considerable degree”, except with respect to attracting funding for CAADP activities. They felt that the activities they have worked on addressed the most important issues relating to rural development, and have produced results that have been useful for decision-making (Volume 3, pp. 67–69). Many have been involved in capacity building and consensus-building. They found that many of the activities at the continental level have been intended to feed a process; they were not sure what these processes have been adding to country processes where real progress needs to take place. 4.56 The site visits also found the choice of partners by the CTF organizations to be relevant, if not always completely effective. For example, the CAADP team at ECCAS, like those at the other RECs, was expected to work directly with national counterparts, with the help of individual consultants, to jumpstart the CAADP process in member states. However, it rapidly became apparent that the ECCAS team did not have the required institutional capacity to undertake these tasks. Hence, it was decided to enlist the help of partner organizations that could work directly with the countries. ECCAS enlisted the help of FAO to support the preparation of the CAADP Compacts and NAIPs, HubRural to support the design of the regional agricultural policy and investment plan, and IFPRI for analytical studies on growth and poverty reduction in support of the NAIP preparation. Our assessment concluded that this approach was the best one in terms of efficacy and efficiency. It also alleviated the burden on ECCAS to manage consultants directly in every country, and allowed ECCAS to focus on its area of comparative advantage, which was the coordination and harmonization of CAADP processes across Central Africa. FAO’s support proved to be key for NAIP preparation and IFPRI’s modeling work for the design of the NAIPs, although IFPRI faced many obstacles in providing the studies in time. 53 Likely Sustainability of Outputs and Outcomes Achieved 4.57 Sustainability has two major dimensions: (a) the sustainability of the CAADP structures that the MDTF has helped to put in place at the continental, regional, and national levels; and (b) the sustainability of the benefits derived therefrom. Both of these depend crucially on the sustainability of the institutional and human resources capacity that has been built at the three levels and the willingness of Development Partners to continue supporting the CAADP process. 4.58 The MDTF has aimed to strengthen the capacity of the six continental and regional organizations in three dimensions:  Individual human capacity: Strengthening the skills of individual staff members to analyze development needs; to design and implement strategies, policies, and programs; and to monitor results.  Organizational capacity: Strengthening the six organizations’ internal structures, processes, systems, staffing, and other resources to achieve each organization’s goals.  Institutional or system-level capacity: Strengthening the institutional context within which the six organizations operate. That is, improving the performance of public sector institutions like AUC, DREA, and the RECs involves more than simply improving organizational processes, hiring new staff, and strengthening individual skills — although these things are important — but also strengthening the institutional environment within which the six organizations operate as intergovernmental organizations in the context of sovereign states. 4.59 The independent Mid Term Review in 2011 found that the CTF projects had enabled the organizations to buy-in significant additional staff time, but that the sustainability of this approach was problematic. It would have been helpful if the project designs had included, say, five to ten year organizational development strategies for each organization, which would have provided exit strategies from donor support. The Mid Term Review also found the CTF organizations needed staff with excellent facilitation skills to perform the challenging role of catalysts for change. 4.60 The ECDPM Review in 2014 found that the MDTF had played a key role in building the capacity of the six organizations to improve continental and regional coordination around CAADP, but that there should be (a) increased focus on technical capacity building, and more systematic planning and monitoring of it; (b) stronger and more targeted institutional strengthening activities; and (c) more efforts on knowledge management at all levels. The ECDPM Review also concluded that the CTFs should have devoted a larger share of their support to address demands from national stakeholders and to sustain implementation progress in-country after the NAIPs were launched. This Review felt that the MDTF could have done more to equip countries with tools to move from NAIP preparation to actual implementation. 4.61 At the individual level, the present evaluation confirms, based on the site visits and on an extensive documentary review of aide-memoires, that the CTF projects enabled the 54 organizations to recruit new staff to implement their activities, although the recruitment processes were often protracted. Staff were often assigned multiple roles when key positions were not filled, and staff recruited for one purpose, such as monitoring and evaluation, were often pulled into other, deemed higher priority, tasks due to overall staff shortages relative to the demands on the organizations. Individual training focused primarily on improving organizational processes such as financial management and procurement with positive results. The training of operational staff consisted primarily of mentoring and on-the-job experience. Now that the CTF projects have closed, some of the staff who were recruited have been terminated. 4.62 At the organizational level, the Bank’s implementation support missions to the six CTF organizations have focused broadly on enhancing the alignment between their organizational objectives and their visions, and between their organizational priorities and their objectives. The missions have focused more narrowly on ensuring acceptable standards in the CTF organizations for annual work planning, financial management, procurement, staffing and reporting, while highlighting actions and recommendations intended to improve the overall implementation of the CTF projects. The World Bank also provided ongoing backstopping in between implementation support missions by reviewing terms of reference, work plans and procurement plans. 4.63 At the system level, judging by the results, the CTF projects clearly enhanced the capacity of AUC and NPCA to lead continental processes, and the capacity of COMESA and ECCAS to support country and regional CAADP processes, in the case of ECCAS in partnership with FAO, IFPRI, and HubRural. The MDTF, the CTF projects, and the pre- and post-Malabo activities have improved the institutional setting for these four organizations, and thereby contributed to the institutional sustainability of CAADP. The organizations now have the opportunity to build on the past to become even stronger and more relevant. The long-term vision of sustainability for the CAADP organizations and the RECs is to be funded by their member states. Improving the planning, financial management, procurement, staffing, and reporting mechanisms has been a necessary step towards building the financial confidence of the member states to systematically support the organizations. 4.64 Failing to sustain the institutional, organizational, and system-level capacity that has been built would be unfortunate. Seventy-three percent of the country Focal Points responding to the survey felt that the capacity of AUC-DREA and NPCA had been sustainably strengthened — to much or some degree — since 2010 to lead/enable/encourage national governments to establish CAADP processes, and 84 percent felt that the capacity of their RECs had been so strengthened. Seventy-eight percent of the country Focal Points said they had been “satisfied” or “very satisfied” working with AUC-DREA and NPCA during the last five years, and 79 percent “satisfied” or “very satisfied” working with their RECs (Volume 3, pp. 21–23). They wanted the continental and regional organizations to continue their work in a large number of areas such as strengthening country-level capacity, advocating with non-state actors for CAADP principles and processes, and mobilizing resources, while also improving their communications skills. 4.65 The end of the MDTF has demonstrated how vulnerable some of the organizations (AUC-DREA, NPCA, COMESA, and ECCAS) are to the vagaries of external support. In the 55 absence of a follow-on Malabo facility, some Development Partners such as the EC, GIZ and USAID will no doubt continue to fund some staff positions and activities in each of the organizations, which will enable them to sustain some or many of their activities. However, this is unlikely to represent the kind of five to ten-year capacity development strategy that the independent Mid Term Review recommended. 4.66 A comparative review of 17 global and regional partnership programs in 2011 concluded that the three major threats to the sustainability of such initiatives have been:  Failure to keep the initiative’s objectives and design relevant in a changing global and regional context  Failures of governance and management  Failure to demonstrate results (IEG, 20111, p. 44). The first does not appear to be a major threat to the sustainability of CAADP given the renewed commitment of African leaders to the principles and values of the CAADP process in the Malabo Declaration. But the second and third remain significant threats, as will be discussed in Chapter 6. 56 5. Cross-Cutting Issues 5.1 This chapter briefly covers selected cross-cutting issues in the design and implementation of the CTF projects — inclusion, gender, and climate change, the science agenda, and the agribusiness agenda. These issues featured most prominently in the second component of the MDTF to support the development of continental and regional frameworks for each of the four CAADP Pillars. 5.2 The CAADP Pillars had been identified as key priorities for achieving agricultural growth, poverty reduction, and sustainability/resilience in the original CAADP document in 2003. Between 2005 and 2008, during discussions on how to operationalize the CAADP framework, it was decided to develop more detailed vision documents for each Pillar. Then Pillar Lead Institutions were identified to lead expert groups and communities of practice to develop the pillar framework documents. Following the successful launching of these pillar framework documents (which were supported at their end stages by the MDTF), the mandates of PLIs were expanded around 2009–10. However, as discussed in Chapter 3, there was some discomfort in 2010–11 about whether this was the best approach, which led to some re-thinking on how best to develop communities of practice to support what came to be called the KIS agenda. Only one CTF project was approved in component (2) for 1½ years, namely for the CMA/WCA — the PLI for Pillar II on “improving rural infrastructure and trade-related capacities for market access.” 5.3 These cross-cutting issues did not feature as prominently in the original designs of the six CTF projects in the first component of the MDTF. Their initial focus was more on determining if this financial instrument could be made to work to support country and regional CAADP processes. Nonetheless, these issues have grown in importance during the life of the CTF projects. Therefore, the evaluation agreed in the Inception Report to review the coverage of some of these issues in the CTF projects, primarily for the purpose of drawing lessons, if any, for their incorporation into the objectives and design of a potential Malabo facility. The evaluation has not attempted to assess the achievements of the CTF projects in these cross-cutting areas. Inclusion, Gender and Climate Smart Agriculture 5.4 In 2014, the Africa Region Agriculture Global Practice of the World Bank conducted a thematic review of the coverage of these three issues in the MDTF and CTF documents in order to make recommendations for greater efforts to incorporate these issues in a potential MDTF2 (Jonasova, 2104). The report reviewed (a) project documents, including their Results Frameworks, for the parent MDTF, the original CTF projects, and the restructured CTF projects; (b) selected CAADP Compacts and NAIPs; and (c) other CAADP documents (non MDTF). 5.5 Inclusion refers to the poverty focus of CAADP processes and interventions, recognizing that the vast majority of the rural poor in Africa are smallholders, who face many constraints, such as access to modern technologies, capital investments, and supportive research; lack of participation in decision-making; and vulnerability to ecological shocks. A Malabo facility might incorporate technological, institutional and policy approaches for 57 creating more and better farm and off-farm opportunities for smallholder farmers in the face of changing market structures, the globalization of food market chains, large farm competition, increasing pressures on land and water resources, and the growing importance of non-state agencies in supporting their needs. 5.6 Gender refers to the fact that the majority of smallholder farmers in Africa are women. There is growing recognition that reducing gender disparities in the access, control, and use over agricultural assets, production, and incomes is a critical issue in agriculture and rural development. Involving women in natural resources management, in formal and informal markets, and in policy-making processes can enhance environmental sustainability, women’s incomes, and public expenditure allocations that favor investments in social infrastructure such as water supply and schools. 5.7 Climate change refers to the likely impacts on African agriculture due to global warming, and in particular on the viability of strategies for sustainable intensification in the use of land and water resources in African agriculture. There are needs to analyze the most promising climate smart agriculture (CSA) investment options, and to outline the investments needed to transform ongoing and planned programs, activities, and projects into proper CSA interventions, while also identifying corresponding public and private financing sources. 5.8 The Thematic Review found the following coverage of these three issues in the MDTF and CTF project documents (pp. iii–iv):  The Results Framework in the original CAADP MDTF Program Document included these three topics in the Development Objectives.  The original and restructured CTF Project Papers for the AUC did not include coverage of these topics at all.  The original CTF project documents for NPCA and SADC incorporated a focus on these topics. The NPCA project document made references to the three topics in the text, but not in the Results Framework. The SADC Project Concept Note included reference to Pillar 1 of its Regional Agriculture Policy addressing land tenure policy (relevant to all three topics), and to SADC protocols on shared water resources, forestry, and fisheries.  The restructured CTF project documents for COMESA and ECCAS incorporated a focus on the three topics. The restructured COMESA Project Paper included the three topics in the revised Results Framework. The original ECCAS Project Paper made references to the three topics in the text of the document, but not in the Results Framework. The restructured ECCAS Project Paper included the three topics in the key results and activities.  The original CTF Project Paper for ECOWAS included coverage of climate change. 58 5.9 Then the Thematic Review analyzed 15 national CAADP Investment Plans and 12 Compacts (where Investment Plans were not available). The Thematic Review found the following (p. iv):  Almost all Investment Plans and Compacts covered all three topics; however, most of only had a reference to the topics in the text of the documents, rather than specific indicators either in the text or the Results Frameworks.  All but one of the 27 Investment Plans and Compacts had coverage of poverty, with eight specific mentions of vulnerable groups and nine mentions of smallholders. Only two Compacts and one Investment Plan had references to social protection.  Almost all (23 out of 27) Investment Plans and Compacts had coverage of sustainable or natural resource management, with eight having explicit climate change coverage.  Only 15 out of 27 Investment Plans and Compacts covered gender issues. Only three Investment Plans (Malawi, Rwanda, and Togo) and one Compact (Mozambique) elaborated further on the coverage of gender issues.  With respect to illustrating good practices, only Ethiopia and Malawi had robust Results Frameworks in their Investment Plans and Rwanda was noteworthy for gender empowerment in the rural space. 5.10 The Thematic Review recommended — at a time when there was serious consideration of preparing an MDTF2 similar to the original MDTF — that the MDTF2 program document and Results Framework should include a section on each topic. The Review recommended that indicators in the Results Frameworks should be provided for each topic with baseline values and targets. Agreement should also be reached on a minimum set of core indicators for each topic which would be included in country-level documents such as the NAIPs. Definitions and key considerations should also be formulated for each topic, which would inform outcome indicators and quantification of impacts. 5.11 These recommendations are reinforced by the fact that these topics are very much in evidence, and given almost inordinate prominence in the 2014 Year of Agriculture, the Malabo Declaration, the Implementation Strategy and Roadmap (IS&R) and the Program of Work (PoW). Both the Science Agenda and the Agribusiness Strategy also give prominence to these topics, perhaps even when it is not entirely necessary or appropriate. These recommendations are tempered only by the findings of this evaluation with respect to long￾term nature of the difficult task to establish effective monitoring, evaluation and reporting systems at the national level. 5.12 Halving poverty by 2025 through inclusive agricultural growth and transformation is one of the 9 commitments made at the Malabo summit. The Declaration also gives some prominence to ending hunger as well as promoting the roles of women and youth. The first strategic action in the IS&R — on measures to increase sustainable agricultural production and productivity — also highlights the need for inclusion. The PoW talks of strengthening 59 the position of farmers, women and youth in regionally integrated value chains. The practice areas under the PoW include embracing gender, youth, and climate change resilience. 5.13 Climate Change and resilience feature most prominently in these documents. It is one of 9 thematic work streams in the Year of Agriculture and one of only 5 thematic papers tabled for discussion at the Ministerial and Heads of State meeting that endorsed the Malabo Declaration. Enhancing resilience of livelihoods and production systems to climate variability and related risks is one of the 9 Malabo Commitments and one of the 4 Strategic Action Areas in the IS&R. Increased resilience of livelihoods and systems through coping and adaptation mechanisms at the production level and by promoting risk and shock￾reduction measures through the functioning of markets is one of the 4 strategic priorities in the Program of Work. 5.14 During the implementation of the ECCAS CTF project, poverty reduction received the highest attention among cross-cutting issues in Central Africa. The analyses conducted by IFPRI focused on both growth and poverty reduction. Through the micro-simulation models (when household surveys were available) or through studies based on the elasticity of poverty to growth, IFPRI analyzed the impact of various growth strategies on poverty and income distribution. Although these studies were not always available at a sufficiently early stage, all the NAIPs focused heavily on poverty issues. The Gabon NAIP, for example, planned to reduce poverty from 33 percent to 16.7 percent in 2020. 5.15 The ECOWAS CTF project document covered climate change. In key function 2 for example, the ECOWAS Department of Agriculture and Rural Development was supposed to “support analysis at regional and/or national level on public expenditure reviews (PER) and emerging issues such as climate change, support to agricultural inputs, and to the adoption of improved technologies.” The document also raised the need to “intensify the project’s focus on important topics that were not emphasized in the initial CAADP framework documents and processes such as climate smart agriculture.” The recent ECOWAP+10 conference in November 2015 also concluded that there was a need to adopt sustainable intensification models to adapt to climate change. The Science Agenda for Agriculture in Africa (S3A) 5.16 As mentioned in Chapter 3, MDTF resources supported a workshop in Dublin on June 30 and July 1, 2011, to bring African institutions and the CGIAR together to develop a science agenda for African agriculture (FARA, 2014). The subsequent process of formulating the Agenda has been African-owned and African-led, culminating in its adoption by African Heads of State and Government in July 2014 as part of the celebration of the 2014 Year of Agriculture and Food Security. An Expert Panel composed predominantly of African professionals was entrusted with writing and peer-reviewing the document. Then the FARA Secretariat and its constituent SRO partners and national stakeholders spearheaded the implementation process of broad stakeholder consultation for articulating the final document. 5.17 The Science Agenda — whose vision is “By 2030 Africa is food secure, a global scientific player, and the world’s breadbasket” — has been conceived as a vehicle to support the implementation of CAADP and has identified a suite of issues and options for increasing 60 and deepening the contribution of science to African agriculture. The Agenda recognizes low productivity as the overarching agricultural challenge for science in Africa along with a “lack of coherent and conducive policies; poor incentives; poor access to input and output markets; predominant rain fed agriculture; inadequate agricultural R&D spending; heavily degraded and depleted soils; problematic land tenure systems; inadequate levels of mechanisation; many pests, diseases and weeds; and climate change.” The strengths of African agriculture are given as the diversity of agro systems, expanding domestic markets, efficient smallholders, a large youthful population, and growing economies with increased investment in education, infrastructure and policy frameworks. The Agenda recognizes 5 important I’s — institutions, inputs, infrastructure, incentives and information — as being important to driving the transformation of African agriculture. 5.18 The Agenda has 7 strategic thrusts: (a) an enduring vision (b) CAADP as a short term priority (c) research themes that connect institutions and policies with producers, consumers and entrepreneurs (d) strengthening solidarity and partnerships at national, regional and international levels (e) sustainable financing of science and technology (f) creating a favorable policy environment for science (g) establishing a special fund for the Science Agenda — ASATI, the Agricultural Science for Agricultural Transformation Initiative. 5.19 The 85-page document gives an extensive menu of things that need to be done. Some might argue that the comprehensive list covers everything, and perhaps too much. The Agenda is a lot less clear on where, by whom and how things will get done. In the section on moving forward, language such as “identify common challenges,” “complete needs assessment.” “plan for establishment of more…,” “examine and expand facilities,” “develop guidelines,” “work to actualize,” “formulate approaches,” “design and plan,” and “strengthen systemic capabilities” suggests the reality that the Agenda is more an aspirational list than an action plan that stakeholders and funders have bought into. The preamble by the IFAD President who was Chairman and Patron of the Expert Panel is instructive. He points out the need for domestication of the agenda into national strategies, and a next stage of actions that will require much political, financial and stakeholder support. The foreword by the AU Commissioner, NEPAD CEO and the FARA Executive Board Chair “implores all stakeholders to give priority to the operationalization of the Agenda,” calls on countries to increase domestic investment in science, and urges member states to “adopt and adapt the agenda at national level and implement it without delay or hesitation.” 5.20 The AUC-DREA CTF project supported 3 years of the salary costs of the CAADP Advisor for Agricultural Science, Technology Dissemination and Adoption. Box 5 at the end of this chapter is a non-exhaustive listing of his activities in support of the development of the Science Agenda. The officer was also involved in M&E as well as logistic and administrative activities at the AUC and is now designated as a special assistant to the Commissioner. Most of his Science Agenda missions were funded by FARA except where the AUC sent a whole delegation. His missions were not funded by the CTF project, not due 61 to lack of funds, but due to short notices and internal constraints on getting approval and funds for travel from the AUC. 5.21 The AUC has benefited from its engagement in the process. The AUC was able to guide the alignment of the Science Agenda with the vision of the next decade of CAADP implementation. The Science Agenda was also an important input into the Malabo Declaration and in setting the clear agricultural productivity targets seen in the Declaration. The AUC will continue to play a role, funding permitting, in guiding the Research and Knowledge institutions in operationalizing the Science Agenda in the context of the implementation of the NAIPs (Policies, Coordination, Financing, etc.), and in guiding the biennial review cycle on the issues related to the contribution of Science, Technology and Innovations to agricultural transformation in Africa. 5.22 Supply-side relevance. The Science Agenda was very consistent with current global and international challenges and debates around agricultural research. The entire CGIAR system was going through the Dublin Process of making itself more relevant to the needs of the developing world and of Africa. Developing an African Science agenda was a well taken opportunistic effort to contribute to global debates. 5.23 Demand-side relevance. Things are a little less clear around the demand for the process from country governments. The final document does not indicate that the Science Agenda was a felt need among member states or their agricultural research institutions, although it is clear that the agricultural science community at the national level welcomes this document as a tool to enhance fund-raising. This is one continental policy document that national-level stakeholders are aware of. The national agricultural research institutions continue to suffer from under funding, management and staffing constraints and a need to better convince national level policy makers of their importance. They depend largely on Development Partner funding for non-salary program and research funding. National level processes might have been their priority had they been given a chance to make a choice and to access the necessary funding. 5.24 Relevance and effectiveness of partnerships. The Science Agenda process was essentially led by FARA. FARA had an even greater interest than the AUC in the development of this document, but included the AUC’s Adviser for Agricultural Science, Technology Dissemination and Adoption in the center of the process. The Science Agenda benefitted from having that AUC adviser on board. It was an excellent partnership driven by a non-MDTF institution with a big stake in its successful completion. (However, FARA is also being supported by a separate multi-donor trust fund administered by the World Bank.) The process also displayed innovations and lessons that a follow-on Malabo facility could learn from. It was a 3-year process. The best brains and experience were brought to bear led by a 20 member Oversight Group, and a very high-level Expert Panel of 11 panelists guiding and quality controlling the process and document. The document does leave disaggregating its messages to specific stakeholders like NARS, SROs, the private sector and policy makers as an unmet need and trusts the AUC and NPCA to provide guidance on how some of the activities can be incorporated into the implementation of the Malabo Declaration. The Agricultural Science for Agricultural transformation Initiative fund, ASATI, remains a wish at the present time. 62 5.25 Sustainability. The Science Agenda features prominently and was carried into Thematic Area 1 of the Year of Agriculture, the Malabo Declaration, and Strategic Action Area 1 of the Implementation Strategy and Roadmap and the Program of Work. But the sustainability of the Science Agenda process remains a question. FARA will continue to engage in the agricultural science agenda as the apex continental organization for African agricultural research. And it is likely that friendly Development Partners will refer to the document as they develop their continental, regional and national research programs. But as the FARA Board Chair noted, all the authors and institutions can do is “implore stakeholders to give priority to operationalizing the Agenda,” and adopt and adapt it at the national level without delay or hesitation. This is really all a continental level process can be expected to do within the loose cooperation and supremacy of member states enshrined in the current AU charter. The Continental Agribusiness Strategy 5.26 The Continental Agribusiness Strategy was developed by AUC and NPCA in September 2015 (AUC, 2015) and succeeds the earlier (2008) Pillar II framework document — Framework for Improvement of Rural Infrastructure and Trade Related Capacities for Market Access — which member states and RECs have been using while formulating and reviewing NAIPs. 5.27 The earlier CAADP framework document had proved inadequate in terms of engaging the private sector. The need to review the strategy came from a series of private sector meetings in 2013 and 2014. The new Agribusiness Strategy lists technologies, knowledge and infrastructure as the key endogenous factors that make it cheaper to import food from outside rather than from within the continent, and gives as constraints to the expansion of inter-African trade non-tariff barriers, an overly segmented, underequipped and underfinanced agricultural supply and marketing system, and subsidized or highly productive foreign producers. 5.28 The vision of the Continental Agribusiness Strategy is “a dynamic, inclusive agribusiness sector that adds value to primary produce, generates employment and income, contributes to economic growth and reduction of food dependency in Africa.” The mission is “to enable the creation of a conducive environment for increased private sector investment and business in Africa’s agriculture.” Seven guiding principles inspired by the Malabo Declaration are (i) recognition of Africa’s interests; (ii) focus on strategic value chains; (iii) support to the indigenous agribusiness sector, women and youth; (iv) domestic resource mobilization; (v) sustainable use of environmental resources; (vi) regional trade; and (vii) subsidiarity, i.e. that countries lead and the continental and regional organizations support. 5.29 Seven strategic pillars lead to 38 key action areas. The seven pillars are (a) enabling policy (11 key action areas) (b) monitoring and reporting progress (4) (c) strategic research and knowledge management (7) (d) facilitating partnerships (7) 63 (e) building systemic capacity (5) (f) innovative financing for agribusiness (4). 5.30 The key action areas are replete with words like foster, facilitate, eliminate, mobilize, support, and promote, but do not say who or how. Important potential interventions are mentioned (along with some that may not be so important), but in the absence of another level of detail and endorsement, the document seems very general and it is not likely that much will happen as a result of it. The section on implementation says it is the responsibility of all actors in Africa’s agribusiness to implement the strategy, but gives the AUC and NPCA the primary implementation role at the continental level, RECs at the regional level, and member states at the national level acting through country level NAIPs. A committee is to be formed to review progress, and national and African Agribusiness Chambers are to be formed. 5.31 Supply and demand-side relevance. The relevance of having a continental agribusiness strategy is not in doubt. However the current document is not likely to be relevant to those thinking seriously about agribusiness in Africa. It is not clear that the ultimate beneficiaries had much of a say, and it is difficult to have a say when the consultative processes leading to the preparation of a document are so limited. The 25-page document does not reflect the main global and regional challenges currently facing the private agribusiness sector. It reflects the world view of the organizations driving its development with a heavy focus on words like foster, facilitate, and support, and gives a central role to AUC and NPCA which really have little comparative advantage in this area. 5.32 Relevance and effectiveness of partnerships. Partnerships were formed with regional farmers’ organizations and a nascent continental organization (PAFO). Plans were set out for country level action through NAIPs and national Agribusiness Chambers. These have not yet transpired. More effective partnerships, driven more by players who are members of, or who can collect views from the private sector, together with more country level consultation could lead to the development of a more relevant and demand-driven draft with more of a focus on removing policy constraints, access to investment finance, value addition and market access, and less of a focus on monitoring and reporting, research, capacity building and partnerships. 5.33 Sustainability. The AU organs — including the AUC and NPCA — do not have the right background or capacity to either develop or drive implementation of an Agribusiness strategy for Africa. Capacity exists on the continent, but it has not been brought to bear in this process. A similar problem exists at national levels where many countries tout the promotion of agribusiness as a policy goal of policy, while agribusinesses continue to complain about how member states make it so difficult for them to do business, and grow their businesses. Agribusiness should be a large part of the transformation of African agriculture but this Strategy is, at best, an early effort by a very limited group of people to put some thoughts together. 64 Lessons 5.34 A follow-on facility could continue to support the development of continental strategies like the Science Agenda and the Agribusiness Strategy, and on selected topics such as small farmer development, incentives to private sector (agribusiness) investments, intra￾African trade, data collection and reporting, since the latter feeds into the proposed and important peer review process. But these need to be organized and managed well to obtain quality products that are implementable in member countries. They should not be developed by a small, relatively closed group of AUC or NPCA staff, like the Continental Agribusiness Strategy was developed. Their staff should be the conveners, not the authors, working with leading organizations in each field to convene the best team to complete the task, like the way in which the Science Agenda was developed. The follow on facility could spend some resources for such continental level activities that are prepared for endorsement by the AU Summit and reported back to the Summit. Each effort would pull the evidence together, present it in the AU hierarchy of meetings, get in adopted by Heads of State and Government, and then feed it into the biennial peer review mechanism. This could also assist in focusing the Malabo agenda on fewer topics. Box 5. Activities of the CAADP Advisor for Agricultural Science, Technology Dissemination and Adoption in Support of the Development of the Science Agenda  September to December 2012 – Preparation of the MoU between AUC and CGIAR  January 2013 – Signing of the MoU between AUC and CGIAR  January 2013 – Technical Advisory Group Meeting of African Agriculture Science Agenda, Accra, Ghana  March 2013 – Oversight Group Meeting Agriculture Science Agenda for Africa Rome, Italy  April 2013 – Development of a Science Agenda for Agriculture in Africa With inputs for CAADP-CGIAR alignment Dublin  April–July 2013 Preparation of MoU between AUC and FARA on the Development of Science Agenda  July 2013 – Participation in the Africa Agriculture Science Week, consultation meeting on the Science Agenda, AUC signed MoU with FARA. The AUC delegation: Commissioner, Commissioners Special Assistant, Director for REA, CAADP Communication Specialist, CAADP Coordinator, CAADP Adviser for Agricultural Science, Technology Dissemination and Adoption.  August – September 2013 – Contribution in the online consultation to develop the Science Agenda  September 2013 – Continental Consultative Workshop – Science Agenda for Agriculture in Africa (S3A), Accra, Ghana  October 2013 – Development of the Proposal of the Africa Agricultural Technology Platform  November 2013 – Briefing meeting at International Fund for Agriculture Development, Rome, Italy  November 2013 – Oversight Group and Workshop on the Science Agenda, Accra, Ghana  February – March 2014 – Coordination of Synthesis Report on the Agricultural education and skills development and the Data, M&E and Mutual Accountability work streams of the Year of Agriculture 65  March 2014 – Organization of the CAADP PP; Convening of the Science Agenda for agriculture, education and skills development and on Data, M&E and Mutual Accountability work streams  March–April 2014 – Coordination of the Synthesis Report on Agricultural Production and Productivity  April 2014 – Organization of the Ministerial Conference – Presentation of the synthesis paper on Agricultural Production and Productivity  May–June 2014 – Preparation of Documentation for the June 2014 AU Summit in Malabo  June 2014 – Participation in the June 2014 Summit in Malabo – Adoption of the Malabo Declaration (specific target of the Science Agenda – Double Agriculture Total factor Productivity by 2025  July 2014 – November 2014 – Development of the Malabo Declaration Implementation Strategy and Roadmap, specific responsibility for the Strategic Action Area on Production and Productivity (Write shops in South Africa, September and Addis Ababa in November 2014)  November 2014 – Validation of the Malabo Implementation Strategy and Roadmap in Addis Ababa  December 2014 – 15 Anniversary of FARA – Presentation of Malabo Declaration and the Implementation Strategy and roadmap at the Special Event on Delivering Africa’s Future through Science led agricultural Transformation (Operationalization of the Science Agenda in the Context of Malabo) – Launch of the Science Agenda. The AUC delegation: Commissioner, Commissioners Special Assistant, Director for REA, CAADP Communication Specialist and CAADP Adviser for Agricultural Science, technology Dissemination and Adoption  February 2015 – Participation in the High Level Planning Meeting on Scaling Agricultural Innovations In Africa, 23–24 February – Presentation of the Implementation of Malabo Declaration and the linkages with the need for innovations.  March 2015 – Organization of the CAADP PP  May 2015 – Appointment to FARA Board as Representative of AUC. 16th FARA Board meeting  May–June 2015 – Prep Steering Committee on the MoU between AUC and CGIAR.  September–November 2015 on going establishment of a Task Team on Agriculture Science, Technology and Innovations, preparation of the TORs and ongoing discussion on the work program for joint activity to implement the Science Agenda in the context of the Malabo implementation between AUC, FARA and CGIAR.  November 2015 – Invited at the 17th FARA Board of Director. He could not attend as was engaged with the MDTF review team. Source: AUC-DREA, CAADP Advisor for Agricultural Science, Technology Dissemination and Adoption, November 2015. 66 6. Organizational Effectiveness 6.1 This chapter addresses the second major set of evaluation issues in this evaluation. It investigates the governance and management of the program, the efficiency of the financial and human resources used, the monitoring and reporting of progress and results, and the alignment and coordination among Development Partners contributing to the MDTF. Governance and Management Arrangements 6.2 As explained in Chapter 3, the governance and management of the MDTF has gone through three phases. First, there was an informal Interim Steering Committee consisting of three members and two observers which met regularly from November 2008 to November 2009 while longer-term governance arrangements were being developed with the assistance of a consulting firm. Second, the Partnership Committee met seven times between April 2010 and July 2013 based on an Operations Manual that was endorsed at the 5th CAADP PP meeting in Abuja in November 2009. Third, the Partnership Committee adopted a revised structure and mandate in July 2013 in response to the findings and recommendations of the Mid Term Review and met five times between May 2014 and December 2015 based on the revised Operations Manual that was issued in October 2013 (Table 2). 6.3 The original Operations Manual also provided an organogram for the MDTF indicating reporting lines, support structures, and funding channels among the various MDTF bodies (Figure 16). This distinguished the roles of the NPCA and the World Bank. The NPCA as the Secretariat would provide support to funding applicants in the proposal process as well as logistics and technical support to the PC while the World Bank would administer the MDTF and coordinate reporting to the PC. The original Operations Manual (and the organogram) also indicated that the PC reported to the CAADP PP, and that the CAADP PP Figure 16. Organogram of the MTDF, April 2010 Source: CAADP MDTF, Operations Manual, Version 2, April 2010. CAADP Partnership Platform CAADP MDTF Partnership Committee Funding Applicants Appointment Reporting Response Applications Development Partner Group Fund Administrator: World Bank Secretariat: NPCA Partnership Committee Support Structures Reporting Funding Logistics Technical review Reporting Proposal support 67 Table 2. Membership and Mandate of the Partnership Committee April 2010 – July 2013 October 2013 – December 2015 Composition Voting Members:  AUC (Chair)  One rotating representative of the four Pillars  One rotating representative of the four RECs  Two representatives of Development Partners  One rotating representative of civil society organizationsa  One rotating representative of private sector firms/ organizations Non-Voting Observers:  NPCA (Secretariat)  World Bank (Fund administrator)  Global Donor Platform for Rural Developmentb Voting Members:  AUC (Chair)  NPCA (Secretariat)  One rotating representative of the four RECs  Two representatives of Development Partners  One rotating representative of civil society organizationsa  One rotating representative of private sector firms/ organizations  One rotating representative of a knowledge institution  World Bank (Fund administrator) Core Functions  Assessing eligible applications for funding  Making funding recommendations  Reviewing the outcomes of financing allocated to granting institutions  Reporting on the MDTF to the CAADP PP  Contributing to the strategic thinking of the MDTF system by communicating key strategic considerations related to the evaluation of applications Providing overall strategic guidance and monitor￾ing the performance of the CAADP MDTF by:  Providing strategic overview, understanding and analysis of trends, priorities, performance and objectives of CAADP  Outlining prioritized funding options and strategic priorities for the MDTF  Making objective, efficient and effective recommendations to the World Bank for the allocation of MDTF funds  Reviewing and monitoring the performance of all major MDTF PC funding decisions  Reviewing and making recommendations in relation to key management systems/tools developed by the MDTF  Informing the RECs and PP of relevant recommendations and issues agreed in meetings a. Civil society organizations encompass farmers’ organizations and NGOs. b. The Global Donor Platform for Rural Development is a network of 38 bilateral and multilateral donors, international financing institutions, intergovernmental organizations, and development agencies that share a common vision that agriculture and rural development are central to poverty reduction. The Platform was created in 2003 (the same year as the Maputo Declaration), following years of relative decline in public investment in agriculture to increase and improve the quality of development assistance to agriculture, rural development and food security. See www.donorplatform.org for more information. Sources: CAADP Multi-Donor Trust Fund: Operations Manuals, April 2010 and October 2013. would elect the private sector and civil society members to the PC. It envisaged that each CAADP PP meeting would devote a separate session to reporting on and discussing the progress of the MDTF and its associated funding allocations. 68 6.4 The principal changes that were adopted in July 2013 were (a) that each organization or group of organizations would now be responsible for selecting their own representatives to the Partnership Committee; (b) that all representatives were now voting members; and (c) that the PC no longer reported to the CAADP PP because the PP was not really functioning as the governing body for the overall CAADP process. Instead, the July 2013 meeting of the PC agreed that the PC was accountable to the relevant senior executive officers of the AUC and NPCA – namely, the Commissioner and the CEO, respectively. The Committee also undertook to adopt a more formal approach to its business. 6.5 However, neither before nor after July 2013 has the Partnership Committee been a typical governing body of a global or regional partnership program (a) that exercises strategic direction and oversight of the program, (b) that is accountable for all the activities supported by the program, and (c) in which its secretariat has signing authority over the expenditures of the program. It has functioned more like an internal management committee. The PC does not have a legal status, either through the AUC, NPCA, or the World Bank. The fundamental legal facts about the MDTF are (a) the Administration Agreements between World Bank and the contributing donors for the receipt of funds, and (b) the grant agreements between the World Bank and the recipient organizations for the disbursement of funds. Both the World Bank and the grant recipients are required to act in accordance these legal agreements. 6.6 Judging by the minutes of the PC, its primary function has been to make recommendations to the World Bank concerning the allocation of MDTF resources. Even here, its recommendations to allocate (or not to allocate) resources to organizations in the form of recipient-executed grants have occurred after extensive preparations, involving World Bank-organized missions to the potential recipient organizations in which the AUC, NPCA, and some contributing donors have also participated. And once the recipient￾executed grants were signed by both parties, the World Bank has organized implementation support missions to ensure that the CTF organizations follow acceptable standards for annual work planning, financial management, procurement, staffing, and reporting. Thus, the grant recipient organizations have been more accountable to the Bank than to the MDTF Partnership Committee. 6.7 The revised (October 2013) Operations Manual is an improvement over the initial (April 2010) Manual. The latter went into much prescriptive detail concerning the formal operation of the PC that was unnecessary and inappropriate for what is essentially a consensus-making body. 6.8 Both the initial and the revised Operations Manual focused on processes and procedures for the approval of recipient-executed CTF grants. Neither established processes and procedures for approving requests for support from the Bank-executed TA Fund. This was a serious omission in the revised Operations Manual, in particular, as the TA Fund came to absorb the lion’s share of the MDTF resources, as explained in Chapter 3. Early on, the Partnership Committee reached an agreement that such requests for support from the TA Fund would be filtered through AUC/NPCA. For requests below $50,000, the NPCA and the Bank could discuss and agree (or not agree) to approve such requests. Requests over $50,000 were to be vetted by the PC, although sometimes, requests up to $100,000 have been approved by AUC/NPCA and the Bank directly, due to pressing need or infrequent PC 69 meetings. Requests over $100,000 have been rare. Once approved — whether for events, meetings, or consultants providing technical assistance — the procurement of goods and services financed by the TA Fund followed World Bank practices,13 and was supervised and handled by a Bank staff member in Washington or country offices — until the Service Agency started handling some of these requests in April 2012. Governance and Management Performance 6.9 Notwithstanding these legal constraints, the PC has attempted to function like a more typical governing body of a partnership program. The original Operations Manual established six principles of good governance — representativeness, Transparency, Accountability, Balance of Interests, African Ownership, and Fact-Based Decisions — derived from the core values and principles of CAADP (Box 4). These therefore represent appropriate criteria with which to assess the performance of the governance and management of the MDTF. This is also consistent with the now widely used approach to assessing the governance and management of partnership programs based on compliance with generally accepted principles of good public sector governance.14 6.10 Legitimacy and representativeness. The PC has been a legitimate governing body composed of the primary CAADP stakeholders, consistent with the CAADP multi￾stakeholder principle. Like most (80 percent) of the partnership programs in which the World Bank is involved, it represents a stakeholder model of governance in which membership is extended to non-contributing stakeholders rather than a shareholder model in membership is limited to contributing donors. But the small size of the PC and the annual rotation of representatives of RECs, Pillars, civil society organizations, and the private sector has reduced the effectiveness of their representation on the Committee. Why couldn’t all four RECs be represented on the Committee or be allowed to attend as observers?15 The REC representatives on the PC found that the agendas and discussions were dominated by the AUC and NPCA, resulting in a lack of visibility of the RECs, a lack of influence in the PC meetings, and little say in decisions at the continental level. They came to view themselves as a grant recipient rather than a full stakeholder in PC meetings. 6.11 The RECs, Pillars, civil society and private sector constituencies have also been insufficiently organized to solicit views and reach agreement beforehand on issues before the Committee and then communicate the decisions back to their constituencies afterwards. Even though there are only four organizations in the REC constituency, they had little experience of working together before the MDTF was established, and represented diverse economic groupings and organizations with different degrees of maturity. 13. See Kingsmill et al., 2011, Annex 3, for a more detailed description of these processes. 14. See Independent Evaluation Group of the World Bank and the OECD/DAC Network on Development Evaluation, 2007, Sourcebook for Evaluating Global and Regional Partnership Programs, Chapter 12. 15. The original Operations Manual was very restrictive with respect to additional observers attending, stating that “Members of the CAADP MDTF PC may occasionally invite a person to attend a CAADP MDTF PC meeting. However, the attending members of the CAADP MDTF PC must approve the visit of the invitee by consensus in the prior meeting” (p. 17). Since July 2013, the PC has been more open to more stakeholders attending as observers, perhaps in response to such a recommendation from the independent Mid Term Review. 70 6.12 Transparency. This concerns the extent to which a program’s decision-making, reporting, and evaluation processes are open and freely available to legitimate stakeholders, subject to the confidentiality requirements of human resource management. The Independent MTR in 2011 found that the PC had operated informally. Papers had not necessarily been circulated in advance and formal proposals for decision making had not routinely been tabled. While the original program document had specified which African organizations would be eligible for recipient-executed CTF grants, it did not specify who was eligible to receive assistance from the TA Fund. It is presumed that any country doing CAADP implementation was eligible, and that countries typically sent requests for support first-of-all to their RECs. But it is not clear how countries found out about their eligibility, other than by word of mouth. It seems doubtful that countries, similarly situated, had equal knowledge of or access to support from the TA Fund. This situation does not appear to have improved much with the establishment of the Service Agency in April 2012 (Box 6). Box 6. The Theory and Practice of the Service Agency (SA) The specified workflow for handling requests was not really followed. As originally planned, a country would send a request to its REC, would which then be submitted to the SA, be reviewed by a Management Committee and NPCA, and then enter into a 6-monthly workplan that would be communicated to the SA. In practice, many (or most) countries contacted NPCA directly and submitted a proposal, which was sent to the SA piece-meal, as received. The work plan only really had meaning for the “overhead” tasks that the NPCA itself controlled. The Management Committee, which was supposed to be central to resource allocation, was never set up. The funds seem to have been allocated on a “first come-first served” basis, with NPCA making the discretionary judgment about the scale of support to be provided. The SA was to support “administrative services in the identification, deployment and management of contracts for expert support to country and regional CAADP implementation processes.” In practice, this included some support for pre-Compact activities, but mostly for preparation of NAIPs, technical reviews of NAIPs, and Business Meetings. It also included support for the Sustaining CAADP Momentum initiative, the CAADP Results Framework, and the 2014 Year of Agriculture activities — for which $1.1 million was earmarked. About 54 percent of the funds ended up going to these “CAADP overhead” tasks, thereby supplementing AUC’s and NPCA’s CTF resources to carry out these tasks. After repeated requests from the World Bank, on behalf of the contributing donors, NPCA has now (in February 2016) produced an information note on “Access to Expert Support for Country CAADP Malabo Implementation”, specifying how countries might access funds for such support through the Service Agency in 2016 — assuming that the contributing donors agree to a no-cost extension of the MDTF beyond June 30, 2016, and the utilization of unspent MDTF funds in this way. Source: Michael Wales and Patrick Tawonezvi, May 2015. “CAADP: Service Agency Review, Final Report,” interviews, and personal communications. 6.13 The MDTF did not establish its own website independent of the CAADP website, www.caadp.net, which is maintained by NPCA. Based on the difficult experience that the evaluation team has had in assembling documentary evidence for the present evaluation, the program should have established a private, “members only” website on which to upload and make more accessible the most important MDTF documents, including the minutes and 71 supporting documents of the PC meetings. Instead, these were maintained on an individual hard drive in NPCA which turned out to be incomplete. 6.14 Accountability. This concerns the extent to which an organization makes, accepts, and fulfills its commitments along the chain of command and control from its governance bodies to its chief executive officer, team leaders, implementers, and, in some cases, to the program beneficiaries. That the CTF grant recipients (some of which were sitting on the PC) were more accountable to the World Bank than to the PC has already been mentioned. The Independent MTR also found that accountability to the PC was also limited by the absence of an overall work plan, and individual work plans and budgets for the Bank-executed components of the MDTF. As a result, the World Bank started preparing six-month work plans for 2012 and reporting on their progress in their semi-annual Stakeholder Status Reports in October 2012. 6.15 The staff of AUC-DREA and NPCA report to their Commissioner and CEO, respectively. But playing the leading roles on the PC, as the Chair and Secretariat of the PC, has been complicated by the fact that they report to two different organizational heads. On the PC also, they have found themselves in more of a collective action situation, requiring mutual accountability, rather than a hierarchical situation. The RECs, which have been responsible for facilitating and supporting country-level CAADP processes, are also independent organizations that report to their own Member States, not to the AU. For this structure to work effectively, those responsible for implementing MDTF-supported activities, whether recipient-executed or Bank-executed, need an individual sense of mutual accountability and shared commitment to the CAADP process. ECCAS, in particular, has complained about unexpected and excessive delays by NPCA in meeting its responsibilities to organize independent technical reviews of NAIPs and RAIPs. 6.16 Balance of Interests. The original Operations Manual recognized the potential conflicts of interest arising from the multiple roles that some PC members played in the MDTF, and the danger that the PC might become a club in which the members allocated the resources among themselves. All PC members were required to sign a Code of Conduct and complete a Declaration of Interest form upon taking office and update these as required. The diverse representation of the PC members was specifically designed to mitigate conflicts of interests, in particular, the presence of (a) Development Partners who had a material interest in seeing outcomes from the funding allocations, and (b) civil society, private sector, and knowledge institution representatives who had less vested interest in the allocation decisions of PC. However, it is not clear to what extent the latter played this role. The civil society and knowledge institution representatives did not attend the last three meetings of the PC and the private sector representative did not attend the last two meetings. 6.17 African Ownership. The contributing donors have commended the strong African ownership of CAADP, now reaffirmed at the highest political level by 2014 Malabo Declaration of African Heads of State and Government. The original Operations Manual viewed strong African ownership of MDTF decisions and outcomes as important to meet the needs of African countries. The World Bank has also endeavored to respect and sustain the African ownership of CAADP and the MDTF without compromising its legal obligations to 72 the contributing donors, while also creating opportunities for African partners to draw upon the Bank’s considerable knowledge, experience, and convening power. 6.18 However, African countries would have preferred the World Bank to have located its task team in a World Bank office in Africa, and for the Service Agency contract to be issued to an African agency.16 The independent MTR found that the physical separation of the secretariat and fund management functions, in NPCA and the World Bank respectively, reduced both the sense of ownership by the African partners and the efficient functioning of the PC, since much of the necessary management information resided with the World Bank as fund manager. 6.19 There has been an “English speaking” bias in the operations of the MDTF which has reduced the sense of ownership among non-English speaking countries, particularly among francophone, lusophone and Spanish-speaking countries in Central and West Africa. Non￾English speaking countries have had a harder time than English-speaking countries in accessing CAADP and MDTF resources — both technical and financial — since many of the documents were only available in English at the beginning. While some efforts were made later on to produce translations in other languages and to provide simultaneous interpretation services at PC meetings, non-English speaking partners and participants have not had equal opportunity to influence the program and receive benefits from the program. It will be important to allocate extra resources for translating documents in order to overcome this bias in the future. 6.20 Fact-Based Decisions. The original Operations Manual stated this principle as “making funding applications and decisions according to pre-defined processes, high quality information and objectively determined facts” (NPCA, 2010b, p. 11). But technical and analytical skills are also necessary to weigh the evidence appropriately. The independent MTR found that the representational approach to membership on the PC did not necessarily provide the skills needed for the Committee’s roles of reviewing and approving requests for funding, which therefore put a lot of responsibility on the technical analysis of the proposals by the World Bank and NPCA. Now that most CTF grants had been approved (as of November 2011), the independent MTR also felt that the Committee needed more skills in results-based management, knowledge management, and monitoring and evaluation in order to monitor and evaluate existing programs and to provide guidance on redirection over time. New programs like the MDTF require skilled governing body members. They typically take time to establish an efficient division of labor between governance and management functions, tend to blend responsibilities between those who govern and those who manage, and call on governing body members to be more involved in management activities than in the case of more mature programs. 6.21 In sum, effectively applying these good governance principles, which drew upon the core values and principles of CAADP, turned out to be a major challenge for the governance and management of the MDTF. 16. The 2015 Service Agency Review found that the international tender process was conducted properly according to World Bank procedures, but that a considerable amount of time elapsed between the request for expression of interest (EOI) in October 2010 and the awarding of the tender to a United Kingdom firm, HTSPE Ltd., in September 2011. 73 Efficiency of the Financial and Human Resources Used 6.22 This section addresses several aspects of the efficiency of the resources used by the program as a whole. It does not address the efficiency or cost-effectiveness of individual activities because of the difficulty — indeed the impossibility — of attributing particular expenditures to particular outputs, even outputs of a quantitative nature such as the number of countries completing Business Meetings or the number of countries that have mobilized resources for their National Agricultural Investment Plans. 6.23 To begin with, the six contributing donors have contributed $65.2 million to the CAADP MDTF between fiscal years 2009 and 2016 (Table 3), of which the top three donors (European Union, USAID, and United Kingdom) contributed 82 percent of the funds. 6.24 Of this amount, 55 percent ($36.0 million) was committed to the seven CTF recipients, of which $28.8 million was disbursed, leaving $7.2 million of these committed funds unspent (Table 4). Another $27.5 million was spent under the Bank-executed components of the MDTF, leaving an additional $1.7 million unspent as of December 31, 2015. 6.25 As explained in Box 1 in Chapter 1, there were actually two parent trust funds supporting the program — one single donor trust fund (SDTF) for contributions from USAID (ultimately amounting to $18.1 million) and a second multi-donor trust fund for contributions from the other five donors (ultimately amounting to $47.1 million). While the Bank’s task team largely managed these contributions as one pooled trust fund, the five-donor MDTF was the more flexible fund with respect to the “basis of commitments,” or BOC, on which commitments could be made to grantees. The BOC in the Administration Agreement for the SDTF was restricted to cash contributions received, while that for the five-donor trust fund was cash contributions received plus promissory notes and future contributions receivable (based on pledges). The SDTF also did not have an established payment schedule, making it difficult to predict when and how much funds could be expected in any given year. 6.26 As a result, the Bank had no choice but to use the SDTF only for shorter-term commitments and expenditures and to use the larger five-donor trust fund for the longer-term commitments to the recipient-executed CTFs (ultimately amounting to $36.0 million). The freedom to make commitments from the five-donor trust fund based not only on cash contributions received but also on promissory notes and donor contributions receivable turned out to be very important at the beginning of the extension phase (2014–15), when the PC approved commitments of $19.0 million to the six CTF organizations, which exceeded the cash contributions that had been received up to that time.17 Making these commitments based on promissory notes and contributions receivable turned out to be the correct, efficient 17. This was an unusual situation, contrary to the Bank’s current (conservative) trust fund guidelines that programs should only make commitments based on donor funds actually paid into the trust fund. With the approval of the line manager at the Director level, programs are allowed, but not advised to make commitments based on promissory notes or donor contributions receivable. Bank management approved the $19.0 million of commitments in FY2014 based on this provision in the Administrative Agreement of the five-donor fund as well as the extensive discussions that had taken place with the contributing donors concerning the extension phase. 74 Table 3. Annual Donor Contributions, Fiscal Years 2009 to 2016 2009 2010 2011 2012 2013 2014 2015 2016 Total EU 3,228.5 3,141.7 324.4 6,690.5 5,708.7 552.5 19,646.3 USAID 3,100.0 1,500.0 2,000.0 2,000.0 4,500.0 3,000.0 2,000.0 18,100.0 UK 11,595.3 3,988.3 15,583.5 Netherlands 650.0 1,300.0 1,300.0 2,600.0 650.0 6,500.0 Ireland 2,184.6 846.3 331.8 337.5 335.5 4,035.7 France 1,290.9 38.8 1,329.7 Total 6,978.5 4,975.5 18,922.1 7,612.7 11,522.3 5,937.5 8,694.2 552.5 65,195.2 Cumulative Contributions 6,978.5 11,954.0 30,876.1 38,488.7 50,011.0 55,948.5 64,642.7 65,195.2 Source: World Bank Group, Unaudited Trust Funds Financial Reports for the single-donor TF071148 (USAID) and the multi-donor TF071150 (other donors). Data for fiscal year 2016 are through December 31, 2015, only. Table 4. Annual Trust Fund Disbursements, FY2009–2016 2009 2010 2011 2012 2013 2014 2015 2016 Total Recipient-Executed Activities NPCA 1,076.5 926.2 1,497.3 750.1 2,509.7 905.5 7,665.2 COMESA 1,720.9 0.0 1,394.2 1,385.0 1,695.5 0.0 6,195.5 AUC-DREA 729.2 286.1 1,411.8 867.3 1,728.0 633.2 5,655.6 ECCAS 270.4 597.5 1,608.6 1,279.5 1,185.0 785.4 5,726.4 SADC 1,903.3 0.0 1,903.3 CMA/WCA 660.9 439.1 0.0 1,100.0 ECOWAS 500.0 0.0 500.0 Subtotal 4,457.8 2,248.9 5,911.8 4,281.9 9,521.5 2,324.0 28,745.9 Bank-Executed Activities Technical Assistance 520.2 3,107.8 3,061.9 1,726.4 2,704.0 2,938.3 2,604.1 188.2 16,850.7 Supervision 199.5 1,518.2 1,429.1 1,606.6 1,746.9 1,147.9 1,148.2 576.1 9,372.5 Agriculture PERs 79.2 66.4 102.2 366.0 116.8 2.6 733.0 Program Management and Administration 106.2 29.7 120.8 190.6 103.2 29.6 580.1 Subtotal 826.0 4,655.7 4,690.9 3,589.9 4,656.2 4,481.7 3,869.1 766.9 27,536.3 Total 826.0 4,655.7 9,148.7 5,838.8 10,568.1 8,763.6 13,390.5 3,090.9 56,282.2 Cumulative Disbursements 826.0 5,481.6 14,630.3 20,469.1 31,037.1 39,800.8 53,191.3 56,282.2 Source: See Table 3. 75 decision. Waiting for the funds to be paid in would have delayed the formal approval of the extensions of the CTF grants to AUC, NPCA, COMESA, and ECCAS. As things turned out, the contributing donors met their payment pledges much in advance of disbursement needs of the CTF organizations. 6.27 However, once the commitments were made to the recipient-executed CTF organizations, they were ring-fenced. The cash contributions that were ultimately received to back up these commitments could not be reallocated for other purposes (i.e. to Bank￾executed activities) unless some of these commitments were formally cancelled, or the CTFs closed. The Bank did initiate discussions with ECOWAS and SADC in 2015 to cancel some of the commitments to their projects when their requests for disbursements lagged behind the initial projections, with the intention of reallocating these funds to potentially support the operations of the six CTF organizations through the TA Fund and the Service Agency, both before and after project closing (December 31, 2015). However, the cancellation of $1.43 million from the SADC grant and $2.8 million from the ECOWAS grant did not take place until August 2015 and November 2015, respectively. 6.28 According to the Bank’s task team, this inability to reallocate ring-fenced commitments was one reason why the MDTF did not contribute resources to the “Catching the Picture of CAADP Support Flows” exercise commissioned by the European Commission in 2014 on behalf of the Development Partners Task Team (DPTT).18 This also explains why the World Bank was reluctant to assume the chair of the DPTT as the MDTF was closing because it didn’t know what would happen after the MDTF formally closed. It could not accept the responsibility and the risk of being the chair and secretariat without knowing that there would be resources to support this. 6.29 At the end of the day, as of December 31, 2015, Bank-executed expenditures were twice as large as originally expected (Table 5). The original program document clearly under-budgeted the resources required in component 3b for trust fund management, administration and supervision. This was realized early on — about a year into the program — when it took longer than expected and more resources to prepare, appraise, and approve the first five CTF grants. The Partnership Committee subsequently reviewed and increased the implementation support budget on an annual basis. 6.30 The World Bank has now undertaken over 60 support missions to the CTF organizations from identification to appraisal to implementation support and to project closing. These have resulted in more than 60 aide-memoires and management letters and more than 25 financial management/procurement plan reviews. The aide-memoires generally record that financial management and procurement were weak at the outset and improved throughout the course of the CTF projects, with the provision of training for financial management and procurement officers, technical support, and mentoring. The final aide￾memoires for AUC, NPCA, COMESA, and ECCAS have rated the financial management of 18. A second reason was that such funding decisions — to use the MDTF to support DPTT activities — had to be submitted to the Partnership Committee, where donors and African partners sat together to review and approve requests. No such formal request was submitted to the Partnership Committee to use MDTF resources to support the “Catching the Picture of CAADP Support Flows” activity. 76 Table 5. Bank-Executed Activities: Actual Expenditures vs. Original Budget (US$ thousands) Initial Budget Actual Expendituresa Percent of Initial Budgetb Technical Assistance 10,000.0 19,456.3 173% CAADP Framework Processes and Thematic Development 12,525.0 Directly Managed 10,845.9 Via Service Agency 1,679.1 Support to National and Regional Processes 6,198.3 Directly Managed 3,080.0 Via Service Agency 3,118.3 Agriculture Public Expenditure Reviews 733.0 CTF Implementation Support 1,919.0 6,665.9 266% Program Governance, Management and Administrationc 581.0 1,414.2 216% Total 12,500.0 27,536.3 196% a. This breakdown of Bank-executed expenditures has been provided by the Bank’s task team. They differ somewhat from those in Table 4 derived from the data on disbursements in Unaudited Trust Funds Financial Reports. b. These percentages have been adjusted for the fact that total program expenditures were $56.282 million compared to the original budget of $50.0 million. c. The original budget category was only for management and administration, not including governance. The budgeted amount for management and administration comes from the Administration Agreements with the contributing donors, as opposed to the program document. three projects as satisfactory and one project as moderately satisfactory, and the procurement experience of one project as satisfactory, two as moderately satisfactory, and one as moderately unsatisfactory. (The ECOWAS and SADC projects have not yet been rated in these respects.) 6.31 Implementation support missions have been relatively expensive because they have required a higher degree of involvement of World Bank staff and greater travel costs than the average for the Bank-executed activities as a whole (Table 6). Indeed, the costs of World Bank staff and the travel costs of staff and consultants have accounted for 92 percent of implementation support expenditures. AUC-DREA has also spent 85 percent of its CTF grant expenditures of $4.87 million on staff and travel costs — 68 percent on travel and daily subsistence allowances, and 17 percent on long-term consultants and freelance staff attached to the unit. By contrast, the Service Agency has only spent 15 percent of its total expenditures on travel. 77 Table 6. Bank-Executed Activities: Components of Total Costs Bank Staff Costsa Travel Costsb Other Costsc Technical Assistance 7% 22% 72% CTF Implementation Support 63% 29% 8% Agriculture Public Expenditure Reviews 3% 38% 59% Program Management and Administration 75% 22% 3% Total 27% 24% 48% a. Includes salaries and benefits of regular staff and extended term consultants in the Bank’s task team, as well as cross￾support from other units. b. Includes the cost of travel, accommodation, and subsistence expenses of Bank staff and consultants c. Includes all other costs, notably the fees of short-term consultants and firms (such as the Service Agency) Source: World Bank data. 6.32 The Bank-executed expenditures from the TA Fund were larger than expected for a number of reasons explained in Chapter 3. While the first five CTF projects were being prepared, the TA Fund was the only component that could meet the demand for support for CAADP processes at any level. Then the AUC and NPCA decided, with the concurrence of the Partnership Committee, to take a different approach than originally planned to component 2 on supporting the development of continental and regional frameworks for the CAADP Pillars, which eventually became the Knowledge, Information and Skills (KIS) initiative. Then the Partnership Committee put in the place the Service Agency to administer the growing demand for technical support from countries and RECs implementing CAADP processes — which was beyond the capacity of either the World Bank’s task team or the NPCA to do so effectively — and as an interim arrangement until the KIS program was operational. The technical support delivered by the Service Agency — which has amounted to about 25 percent of TA Fund expenditures ($4.8 million) — shows up as a Bank-executed activity because the contract was procured and issued by the World Bank directly. But the 2015 Service Agency Review found that virtually all requests for technical services were channeled through NPCA. The NPCA effectively managed these expenditures by controlling the key tasks of formulating TORs, selecting consultants, and quality assurance (Box 6). 6.33 In terms of value for money, the 2015 Service Agency Review found that the Service Agency has proved competitive (by international comparison) in delivering short-term technical assistance, both in terms of daily rates and the level of margin. The Review found the overall margin of 13 percent to be “not excessive” and the all-inclusive average cost per person per day of around $1,200 to be reasonable — including all fees, travel costs, and daily subsistence allowances. However, the Review found little incentive to economize on the margins charged or the fee rates paid because the budgets were prepared on a “cost plus” basis. 6.34 The independent Mid Term Review found that the CTF projects had been a highly appropriate instrument that supported the priority organizations. The present evaluation has found less support for this approach among the MDTF stakeholders going forward. The results of two of the CTF projects that closed on December 31, 2015, have been unsatisfactory. Both ECOWAS and SADC found it difficult to work with this instrument, 78 among other things, for their own bureaucratic and administrative reasons. They found it very difficult to align their own systems with those the World Bank for preparing and implementing recipient-executed activities. ECOWAS, in particular, would have preferred to receive support entirely from the TA Fund, which worked better for itself in terms of preparing its Strategic and Operational Plan and for its member countries in terms of supporting their country-level CAADP processes. 6.35 On the other hand, AUC, NPCA, COMESA and ECCAS found the Bank’s implementation support helpful in improving their financial management and procurement systems. AUC has regarded the CTF project as something of a pilot project in terms of learning how to work with the World Bank, which has contributed in part to the preparation of a much larger capacity building grant from the International Development Association — the Support for Capacity Development of the African Union Commission (AUC) and other African Union (AU) Organs Project for $25 million, approved in May 2014. So the CTF grant instrument turned out to be more of a “one size fits all” approach than originally intended which worked better for some recipient organizations than for others. 6.36 Even for the four CTF projects with satisfactory results, the recipient organizations have found the administrative burden of complying with all World Bank procedures for recipient-executed grants to be high. They have appreciated the organizational strengthening that has occurred through the Bank’s implementation support, but they have questioned the necessity for the Bank to apply the same degree of oversight and compliance for these relatively small grants as for regular Bank-supported investment projects. Clearly capacity has been built in some important areas, but there are also clear limitations as to what a CTF grant can accomplish, particularly in the areas of accountability and sustainability. 6.37 Finally, if a follow-on Malabo facility is to provide direct support to countries to improve their agricultural policies, to access investment finance, and to establish more effective monitoring and reporting systems, then establishing CTF projects in multiple countries is obviously not practical. Some other support mechanisms needs to be considered, such as block grants or competitive grants. Monitoring and Reporting of Progress and Results 6.38 With the exception of the semi-annual Stakeholder Status Report prepared by the World Bank’s task team, this has been one of the weakest areas in the implementation of the MDTF and associated CTFs. As explained in Chapter 3, the parent MDTF and the six CTF projects adopted a revised Results Framework in 2013–14 to address the deficiencies in their original Results Frameworks identified by the Mid Term Review. The new Results Framework aimed to cover the full scope of the MDTF and CTF activities. It had a common Project Development Objective, four common PDO level results indicators, and four common intermediate results areas to facilitate aggregating the achievements of the MDTF and the six CTF projects into a coherent assessment of the overall progress of the MDTF, as in Chapter 4. Then each of the six CTF projects adopted slightly different intermediate results indicators reflecting their own specific circumstances. (See Annex Table B-4.) 79 6.39 The MDTF Partners should be commended for both the process and the substance of revising the Results Framework in light of the first 3–4 years’ implementation experience to address the deficiencies identified at the Mid Term Review. But adopting an improved Results Framework with indicators that had a greater focus on outcomes could only be the first step in putting in place an effective monitoring and reporting system. Other steps would normally include (a) systematic and regular processes for collecting and managing data, and (b) feedback loops from monitoring and reporting to decision-making. Effective oversight of a program or project, and accountability in the use of resources requires good reporting on a regular basis that is also reasonable under the circumstances. 6.40 The new Results Framework was first incorporated into the project papers for the ECOWAS and SADC CTF projects that were approved in August 2013. While AUC, NPCA, COMESA, and ECCAS did not formally adopt this until the conclusion of their restructuring and additional financing processes in January 2014, the Bank and the CTF organizations began the transition to collecting data to measure progress against the new Results Framework before then. The Bank recruited a team of two consultants to assist the organizations in creating and strengthening the architecture for reporting as well as developing the tools to collect data. Then the Bank started integrating the results team into the regular implementation support visits, aiming to simultaneously establish the importance of results reporting without creating undue burdens for the organizations. 6.41 However, the aide-memories have generally flagged the M&E component as one of the major issues faced by each project. Some CTF organizations failed to recruit M&E specialists. Others recruited specialists who were subsequently drawn into other, deemed higher priority, activities. Others recruited specialists who were insufficiently trained for the task. Even those who were recruited and active needed time to understand and apply the new framework. The initial Results Framework had been based more on process and quantitative measures that were relatively easy to monitor; the revised framework was based on more qualitative indicators requiring the subjective judgment of CAADP beneficiaries, which the CTF organizations attempted to obtain through a Survey Monkey questionnaire addressed to the main stakeholders (such as CAADP Focal Points) at the national level. The latter had difficulty understanding the new MDTF Results Framework, in part because of confusion with the CAADP Results Framework (Box 7). Eighteen percent of the Focal Points who responded to our own survey said that they were not at all familiar with the MDTF, and another 18 percent that they were only modestly familiar with the MDTF. Some recipients also had difficulty filling in the Survey Monkey questionnaire due to translation issues. 6.42 Interviews with CTF staff during the site visits to each organization gave the following additional reasons for lack of progress in this area:  The absence of a monitoring and evaluation culture in the organizations.  Limited demand for M&E for use in planning. Planning was based more on seeking financial resources, primarily from donors, than on monitoring past results.  The absence of an initial baseline and the failure to mainstream M&E at the outset.  The difficulty of attributing outcomes to the outputs of activities. 80 Box 7. The Confusing Existence of Two Results Frameworks The appearance of two results frameworks, from different motivations, one to two years apart, in the latter half of the implementation of the MDTF has caused some confusion. The MDTF Results Framework in 2013 was motivated by the MDTF partners in the Partnership Committee to demonstrate results in the context of specific donor-supported projects. The CAADP Results Framework in 2014–15, which emerged from the Sustaining CAADP Momentum exercise, was motivated by the desire of African countries to renew their commitment to the CAADP principles 10 years after the Maputo Declaration. Timely and useful as the MDTF Results Framework has been, it’s time to move on: a follow Malabo Facility should base its objectives and design on the CAADP Results Framework. This is symptomatic of a larger issue. While Development Partners have often expressed good intentions to coordinate and streamline their monitoring and evaluation processes with those of recipient countries, this has often proven more difficult to achieve in practice. Development partners have their own requirements for project-level M&E to satisfy their own domestic constituencies which are not necessarily consistent with the efforts of recipient countries to establish their own more sector-based M&E systems. Development partners also typically have more funds available to implement their project-based systems. There is a need to address these tensions and for Development Partners to put a lot of effort in the post-Malabo phase to align their M&E systems with the CAADP Results Framework. Experience in the health sector, for example, shows that this is more than a short-term exercise. 6.43 In spite of all these difficulties, the World Bank’s task team has succeeded in producing semi-annual Stakeholder Status Reports of the progress of the MDTF, which have provided a good chronological narrative of the MDTF. These have focused on processes such as the approval of new CTF grants or the initiation of new activities. They have provided ongoing financial reports of donor contributions, MDTF commitments, and disbursements. They have focused more on activities and outputs and less on documenting outcomes to which the activities have contributed. The World Bank demonstrated a strong commitment to transparency and accountability in its administration of the MDTF by constantly adjusting and refocusing the Status Reports in response to the requests of stakeholders. The present evaluation would have been much more difficult to conduct without the existence of these regular reports. Donor Coordination 6.44 The contributing donors commissioned two very useful reviews of the MDTF, both managed by DFID — the independent Mid Term Review in 2011 and the second ECDPM Review in 2014. As explained in Chapter 3, the first review, which contributed to the overall Mid Term Review process, was very consequential — supporting the Sustaining CAADP Momentum process that was just getting underway, and contributing to the two-year extension of the MDTF, the revised MDTF Results Framework, and modifications to MDTF governance. The ECDPM Review was less consequential, primarily due to its timing, coming at the end of the two years in which the MDTF had been formally extended, the final two CTF projects approved, and the other four projects restructured with additional financing. However, it has contributed to the present evaluation because it focused on three important topics — capacity building, country-level results, and donor coordination — and its findings 81 are still very relevant for the current discussions regarding the objectives and design of a follow-on Malabo facility. 6.45 The present evaluation generally confirms the finding of the ECDPM Review that the MDTF has considerably increased coordination at different levels, particularly among African lead institutions, between sectors, and among MDTF contributing donors. The CAADP Results Framework includes an integrated reporting structure which, if implemented, will likely improve coordination even more. 6.46 However, common to many partnership programs, the present evaluation has found that the contributing donors have had somewhat different implicit objectives for the MDTF, which has created some tensions among them. Some seemed to be more interested in leveraging their own contributions, others in institutional capacity building, and others in achieving country-level results. Some complained about their lack of influence in the Partnership Committee with respect to the allocation of MDTF resources. As a result, some contributing donors seem more willing than others to contribute to a follow-on Malabo facility. 6.47 One of the key assumptions underlying the MDTF was that establishing an alliance of donors supporting the MDTF would foster a more unified/aligned approach towards the development of CAADP institutions and processes. The counterfactual to establishing the MDTF was not the absence of external assistance from Development Partners, but assistance that would continue to be fragmented in source and focus, each donor with its own set of rules and reporting requirements. The MDTF aimed to provide less fragmented and more focused assistance, in which the CTF organizations would only have to engage with one set of rules for receiving, managing, and reporting on funds. Harmonizing donor support in a single instrument also aimed to reduce the transactions costs to both donors and recipients associated with multiple funding sources, procedures, and reporting requirements. 6.48 The evaluation concludes that this objective was only partially achieved at the level of the CTF recipient organizations. Most of the organizations continued to seek and receive funding from multiple sources for their CAADP work (Table 7). Some of this was from MDTF donors, some of it was core support like that from the MDTF, some of it was project￾based, and some of it was in-kind. Interviews with the financial management officers on the site visits to the CTF organizations indicated that they were still managing multiple funds on multiple donor schedules. The Final Report of the System for “Catching the picture of CAADP support flows” found that AUC-DREA and NPCA were being supported by over 10 different donors (Cathala, 2015, p. 29). AUC confirmed that only 30 percent of their CAADP activities were supported by CTF funds. 82 Table 7. Additional Development Partners Support for CTF Organizations during the Project Period Organization MDTF Commitments Other Donor Support AUC-DREA 6.0 million  $3.0 million from the Bill and Melinda Gates Foundation for 2013–15.  $2.9 million from USAID through IFPRI support several key CAADP staff positions for Pillar II and Pillar III and CAADP activities with a grant of USD 2.9 million  GIZ recruited a CAADP technical adviser and an Agriculture and Climate Change adviser.  FAO Technical Cooperation Program provided in-kind support for recruitment of staff and technical expertise. NPCA 8.0 million  $2.26 million from FAO, GIZ, and IFAD COMESA 6.2 million  USAID through its broader COMESA support program, the Integrated Partnership Assistance Agreement.  FAO strategic partnership. ECCAS 5.9 million  No additional direct support.  $638,000 (in-kind) from FAO for hiring international consultants to support country-level CAADP processes ECOWAS 4.9 million The Strategic and Operational Plan envisaged the following support over five years:  $3.75 million from USAID for equipment and 3 staff positions  $250,000 from EU/African Union for funding a livestock position  400,000 Swiss francs from Switzerland for funding an expert on capacity building of producers’ associations  €400,000 from France for a food security adviser  $5 million from Spain for funding 3 staff positions at the Regional Agency on Agriculture and Food and for technical assistance via an FAO-managed trust fund to support to effective CAADP implementation SADC 3.9 million  USAID and FAO supported the development of the Regional Agricultural Policy and Investment Plan Source: Self-Assessment Reports. 83 7. World Bank Performance in its MDTF Roles 7.1 The World Bank has played multiple roles in the MDTF (Table 8). Several of these roles have already been discussed in previous chapters such as mobilizing technical assistance financed by the TA Fund (role 3), contributing to MDTF governance and management (role 4), and contributing to donor coordination and harmonization (role 5), since these are contributions that the Bank has made collectively with the other MDTF partners. Therefore, this chapter focuses on the Bank’s performance in the three roles for which the Bank has had primary responsibility — establishing and administering the MDTF, supporting the preparation and implementation of the CTF projects, and facilitating linkages as a Development Partner to the Bank’s own country operations and other trust funds supporting African agriculture. Preparation and Implementation Support to CTF Recipient Organizations 7.2 The original program document, approved by the World Bank in September 2008, saw the need for medium-term, core support to continental and regional organizations to enable them to become effective facilitators of country and regional CAADP processes. Establishing a multi-year funding envelope for AUC, NPCA, the RECs, and the PLIs would provide them with predictable financing to strengthen their institutional capacity, build their continental/regional credibility, and deliver results on the ground more effectively. 7.3 The program document envisaged providing this support in the form of recipient￾executed Child Trust Fund grants. Once a grant agreement was signed, the recipient organizations would have direct ownership, control, and administration of the funds provided to support their own roles in the CAADP processes. Eligible expenditures for recipient￾executed CTFs would include all goods, services, training and workshops, operating costs, and their own capacity building activities to carry out their roles in CAADP effectively. 7.4 The original program document anticipated grants of 4–5 years duration for up to 10 organizations in components (1) and (2) — the CAADP Support Platforms and the CAADP Pillar Frameworks. Since these CTF projects were projected to close on December 31, 2013, this meant that they were expected to be up and running by January 1, 2010, to allow for a minimum 4 years of implementation. 7.5 This turned out to be overly optimistic for a number of reasons. The first four CTF grants — for COMESA, CMA/WCA, AUC, and ECCAS — did not become effective until August through November 2010, and the fifth CTF grant for NCPA did not become effective until April 2011, primarily due to uncertainties about NPCA’s legal status as it transitioned to becoming a formal technical agency of the AUC. 7.6 First, the CTF projects had to comply with a number of World Bank policies for recipient-executed grants, which were similar but not identical to those for standard World Bank loans and credits. As laid out in the MDTF program document and as already mentioned in Chapter 3, these requirements included the preparation of 3–5 year strategic and operational plans (Box 2); the managerial, financial, and technical capability to implement these plans; and legally established administrative arrangements between the 84 Table 8. The Multiple Roles of the World Bank in the CAADP MDTF Role Includes 1. Establishing and administering the MDTF  Preparing and approving the MDTF Program Document in consultation with the contributing donors and African organizations  Negotiating the original Administration Agreements with contributing donors  Receiving and investing funds from contributing donors, administering disbursements to CTF organizations, and providing regular financial reports  Amending the Administration Agreements to extend the MDTF in 2013–14 2. Preparation and implementation support to CTF recipients  Identifying, preparing, and appraising CTF projects to ensure quality at entry  Supporting the development of comprehensive strategic and operational plans  Putting in place functional multi-year funding envelopes for the CTF organizations  Supporting effective implementation  Helping to strengthen managerial, administrative, and technical staff capacity and systems  Proactively identifying and resolving threats to the achievement of relevant development outcomes  Ensuring adequate transition arrangements for regular operation of supported activities after project closing. 3. Mobilizing technical assistance activities financed by the TA Fund  Supporting CAADP framework processes and thematic development  Facilitating technical inputs into strategic processes such as the Malabo Declaration, CAADP Results Framework, etc.  Supporting national and regional CAADP processes  Supporting some continent-wide initiatives such as agriculture public expenditure reviews and the Tertiary Education for Agriculture Mechanism (TEAM Africa).  Facilitating productive partnerships (donor to donor, donor to Africa, World Bank to Africa, etc.) in support of CAADP 4. Contributing to MDTF governance and management  Providing support to the Partnership Committee  Monitoring and reporting of MDTF progress and results  Preparing semi-annual Stakeholder Status Reports. 5. Contributing to donor coordination and harmonization  Enhancing the alignment and coordination of donor support to CAADP processes.  Convening meetings such as the Senior Officials Meeting of Development Partners prior to the AU Summit in January 2015 to mobilize support for the Malabo Declaration. 6. Facilitating linkages to Bank’s country operations  As a Development Partner, providing support through its country programs to individual countries to help them achieve CAADP objectives Bank and each recipient. They would have to prepare annual work plans and budgets for their activities, provide semi-annual reports of implementation progress and quarterly unaudited financial reports on the use of grant funds, and procure items financed by the grant in accordance with the Bank’s Guidelines on Procurement and Consultants’ Services. 7.7 Second, the CTF organizations operated in complex institutional environments. As intergovernmental organizations funded by member states and donors, challenges included a 85 demanding political environment, weak institutional capacity, difficult operational procedures some of which were in the process of being changed, very complicated procurement processes for staff recruitment, and much project-based funding from donors. None of the organizations had fully articulated plans at the outset. The amount of time and human resources required to establish each CTF project was greater than anticipated. Most organizations found it difficult to devote staff time to complete the necessary documentation quickly — also given the schedule of CAADP-related events that they were trying to support. 7.8 Third, the World Bank also faced a complex and changing institutional architecture at the time. The Bank’s Executive Board had just approved (in October 2007) a new Trust Fund Administration Framework19 and Bank management was in the process of putting this in place as the CAADP MDTF got underway. So the Bank’s Africa Region was largely on its own, with little support from Central Vice Presidency Units, putting in place arrangements to establish both the parent MDTF and associated CTFs effectively.20 While the World Bank had long experience in preparing agricultural projects in individual African countries, it had less experience preparing a series of regional capacity building projects involving a handful of regional organizations. This was also the first time that the Bank had worked with some organizations such as ECCAS. 7.9 The underlying assumption behind the MDTF was that focusing on continental and regional institutions would enable these institutions to lead and encourage national governments to establish CAADP processes. The challenge of this approach was that it required, in a very short time period, building their institutional capacity and their continental/regional credibility, and delivering results on the ground while respecting (and indeed promoting) the principle of subsidiarity. 7.10 To address these challenges, the Bank held an orientation workshop for potential recipient organizations at the 4th CAADP PP meeting in Pretoria in March 2009 to inform them of the process and documentation needed to establish a CTF grant. Then World Bank teams started conducting identification missions to potential recipients in May 2009, followed by pre-appraisal missions and Joint Technical Reviews, conducted informal institutional and systems analyses of the potential recipients, and arranged for technical support from the TA Fund to assist them in preparing to receive the CTF grants. As mentioned in Chapter 3, only five of the anticipated 10 organizations met the requirements to establish CTFs by 2010. The first meeting of the MDTF Partnership Committee in Johannesburg in April 2010 recommended World Bank approval of these five CTFs. 7.11 Notwithstanding the delays in preparing the first five CTF projects, their elapsed times from concept review to effectiveness compare favorably with those for standard World Bank- 19. World Bank, 2007, “A Management Framework for World Bank-Administered Trust Funds.” This was the framework that established the current operational distinction between (a) Bank-executed trust funds, (b) recipient-executed trust funds, and (c) Financial Intermediary Funds. 20. Indeed, the minutes of the Decision Meeting on June 8, 2010, to authorize the first five CTF projects noted that the CAADP MDTF was “the first programmatic trust fund in the Africa Region to follow the new procedures for the preparation of CTFs under programmatic TFs.” The minutes expressed the hope that it would “be used as a best practice model and showcased on AFTQK’s TF website.” 86 supported agricultural investment projects in Africa. The present evaluation has undertaken a detailed elapsed time analysis of the preparation times of the first five and subsequent two CTF projects (for ECOWAS and SADC) compared with those for other agricultural projects in African countries approved between fiscal years 2006 and 2015. Annex J provides the detailed methodology and results comparing the elapsed time of the CTF projects between four project cycle milestones with five categories of African agricultural projects (Table 9). Table 9. World Bank-Supported Agricultural Projects in Africa: Average Number of Months for Project Preparation Category No. of Projects Average Number of Months Standard Deviation: “Concept Review” to “Effective￾ness” From “Concept Review” to “Begin Appraisal” From “Begin Appraisal” to Bank Approval” From “Bank Approval” to “Effective￾ness” From ”Concept Review” to “Effective￾ness” GFRP projects 15 1.6 2.3 1.8 5.8 4.2 Development Policy Operations 10 13.3 4.6 4.0 21.8 22.6 Standard IBRD/IDA investment projects 58 12.9 3.6 7.8 24.2 12.3 Regional projects 11 11.4 4.8 8.3 24.4 12.7 GEF projects 27 19.2 6.8 7.8 33.9 20.0 First Five CAADP CTF grants 5 6.3 4.0 5.2 15.5 2.6 All Seven CAADP CTF grants 7 10.8 9.9 4.2 24.9 16.4 All Projects 122 12.5 4.4 6.6 23.5 15.9 7.12 The five categories of projects were established based on a priori reasons why some types of projects take less or more time to prepare than standard IBRD/IDA investment projects:  The Global Food Crisis Response Projects (GFRP) projects were expected to take less time to reach effectiveness because these were part of an emergency program of response to the rapid increases in global food prices in 2007–08, and whose preparation mostly followed a standard template.  Development Policy Operations (DPOs) were also expected to take less time to reach effectiveness because these are fast-disbursing operations to provide balance of payments and budget support in response to government commitments to specified policy reforms, and are often part of a series of such projects to a given country, as in the case of four DPOs to Ghana approved in 2008, 2010, 2011, and 2012, respectively. 87  Regional projects were expected to take more time to reach effectiveness than standard IBRD/IDA investment projects because these typically involve reaching agreements with the governments or more than one country.  GEF projects were expected to take longer to reach effectiveness because these involve dual project cycle steps involving approval by the GEF Council and CEO in addition to approval by the Bank. 7.13 As expected, the GFRP projects reached effectiveness in the shortest amount of time – taking an average of only 5.8 months from Concept Review. The DPOs also generally reached effectiveness more quickly than standard IBRD/IDA investment projects except for two outliers (Figure 17). The GEF projects uniformly reached effectiveness more slowly than all the other project types. Somewhat surprisingly, there is no discernable difference in the project preparation times between regional projects and standard IBRD/IDA investment projects — both averaging about 24 months from concept review to effectiveness. Figure 17. Elapsed Time from Concept Review to Effectiveness Source: Annex J. 7.14 The first five CTF projects achieved effectiveness more slowly than GFRP projects and DPOs, but more quickly than standard IBRD/IDA projects – taking an average of 15.5 months from concept review to effectiveness. On average, they took half as much time to begin appraisal (6.3 months compared to 12.9 months), the same amount of time to get approved, and less time to become effective (5.2 months compared to 7.8 months). Once projects are 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 Months GFRP Projects DPOs Standard IBRD/IDA Regional Projects GEF Projects CAADP CTF Grants Percent of Projects 88 approved, “effectiveness” actually depends on lifting the “effectiveness conditions” in the project documents and financing agreements. Once approved, the CTF projects achieved effectiveness more quickly because there were fewer effectiveness conditions in comparison with standard World Bank investment projects, which often require parliamentary ratification, publication in the official gazette, etc. 7.15 But the last two CTF projects in Figure 17 — for ECOWAS and SADC — took 44 and 53 months, respectively, from concept review to effectiveness, which increases the overall average preparation time for the seven projects to 24.9 months, which is about the same as that for standard IBRD/IDA investment projects. Among other factors, as explained in Chapter 3, the ECOWAS and SADC preparation process had to be slowed down in 2013, pending the outcome of the process to extend the MDTF for an additional two years beyond December 2013. At a certain point, there was no point going forward with the preparation of these CTF projects. Further work on their preparation was appropriately suspended until after the two-year MDTF extension was in place. 7.16 Following the effectiveness of the seven CTF projects, the World Bank team has undertaken more than 40 implementation support missions to CTF organizations during the implementation of their projects, resulting in over 40 aide-memoires and 25 financial management/procurement plan reviews. These have focused broadly on whether the core elements of each organization have been sufficient to deliver on their objectives, whether the objectives have been consistent with the vision for each CTF organization, and whether organizational priorities have been correctly focused and aligned. The missions have focused more narrowly on ensuring acceptable standards for annual work planning, financial management, procurement, staffing and reporting in the CTF organizations, while highlighting actions and recommendations intended to improve the overall implementation of the CTF. The World Bank also provided ongoing backstopping in between implementation support missions by reviewing terms of reference, work plans and procurement plans. 7.17 It is also standard practice for the Bank to ensure adequate transition arrangements for the regular operation of supported activities after projects close. The long-term view of the sustainability of the CTF organizations is for them to be funded by African member states. Developing the basic organizational skills of planning, financial management, procurement, staffing, and reporting is viewed as a necessary step towards strengthening the financial confidence of member states to support the CTF organizations. However, there is little evidence so far of African countries creating mechanisms that will enable the CTF organizations to become funded purely or even primarily by member states in the immediate future. Therefore, the Mid Term Review recommended the preparation of an options paper to describe priorities and options for the future funding of CAADP activities. 7.18 At the specific request of the MDTF Partnership Committee, the Bank drafted a concept note outlining the design of a second phase MDTF. As explained in Chapter 3, the Bank submitted this concept note to an internal quality enhancement review in October 2014 and to a stakeholder consultation meeting in Addis Ababa in November 2014. However, the Bank suspended the preparation of a second phase MDTF at the request of several Development Partners in December 2014 because there was insufficient clarity on the future direction and modalities of CAADP following the Malabo Declaration, the Implementation 89 Strategy and Roadmap, and the Program of Work. More time was needed to see how the implementation of CAADP was going to proceed post-Malabo, for which it was not fully clear that an MDTF would be the best instrument to support CAADP going forward. While the independent Mid Term Review of the MDFT had found that the CTF projects had been highly appropriate instruments that supported the priority organizations, the present evaluation found less enthusiasm for proceeding with a second round of CTF grants. 7.19 Nonetheless, the Bank has continued to explore viable options for the future financing of CAADP, post-Malabo, in consultation with the AUC, NPCA, and Development Partners in particular. And the Bank has proposed to provide unspent MDTF funds to the Service Agency to recruit and retain some CTF staff as consultants in 2016. CAADP is a unique process within Africa and within the agriculture sector, with no equivalent in terms of scale and ambition in Africa (or other regions). As a result, the implementation of the MDTF and associated CTFs has been an exercise in learning by doing. Remaining flexible and nimble to the challenges and opportunities presented by the evolution of CAADP has continued to be important. Facilitating Linkages, as a Development Partner, with the Bank’s Country Operations 7.20 One of the reasons why the AUC and Development Partners called upon the World Bank to establish and administer the MDTF was that the Bank was already providing financial assistance to African regional organizations and countries in many areas of CAADP. They viewed the Bank’s management of the MDTF and the Bank’s country-based programs as complementary and reinforcing each other. Therefore, this section assesses the Bank’s achievements in this regard. 7.21 IEG reviews of global and regional partnership programs such as the CAADP MDTF have identified four types of linkages between partnership programs and the World Bank’s country operations, as follows:  Strategic linkages — the degree of alignment between the objectives and strategies of the partnership program and the Bank’s country operations  Operational linkages — the effectiveness and efficiency with which partnership program staff and country operational staff work together  Financial linkages — partnership program financing of country operational activities, or vice versa  Institutional linkages — others arrangements, generally of a non-operational character, between the partnership program and the Bank’s country operations that also contribute to the achievement of mutually shared objectives IEG reviews have also found that being hosted by the World Bank, as for the CAADP MDTF, does not necessarily lead to strong linkages with the Bank’s country operations. 90 STRATEGIC LINKAGES 7.22 The World Bank neither requires nor expects perfect alignment between the objectives and strategies of partnership programs and its own country operations. One of the roles of partnership programs is to be incubators of innovation that, if successful, the Bank might subsequently incorporate into its own work. And the Bank’s sector strategy papers (typically of 10 years duration) are not well synchronized with partnership programs’ strategic planning periods (typically 3-5 years). Circumstances are constantly changing, rendering one or the other less relevant. 7.23 In the case of the CAADP MDTF, the World Bank issued a new corporate agricultural strategy in 2009 entitled, Agriculture Action Plan: FY2010–12, which sought to operationalize the 2008 World Development Report, Agriculture for Development, issued in October 2007. In 2008, the Bank’s Africa Region also re-organized and merged all its agricultural operations into one unit led by Karen Brooks. The new unit needed a new strategy, if nothing else, as a communications device to align the staff to a common agenda and to explain to World Bank Country Directors how individual agricultural interventions fitted into a broader framework. The new strategy had a dual focus on (a) an immediate short-term response to the high food prices that peaked in May 2008, and (b) longer-term support for agricultural growth in alignment with CAADP. The strategy specifically referred to CAADP so that Country Directors would become aware of CAADP, and aligned its product lines with the four CAADP Pillars. Then this regional strategy subsequently contributed to the development of the Bank-wide Agriculture Action Plan in 2009. 7.24 The 2009 Scale-Up Strategy for African agriculture, as it came to be called, came at a propitious time. The 2008 World Development Report had articulated agriculture’s broad role as a catalyst for development and economic growth. In the wake of the 2008 global food price increases, the donor community was seeking to reverse the steep decline in official development assistance to agriculture from the early 1990s through the mid-2000s. But, while the Scale-Up Strategy formed the basis for a number of presentations, both internal and external, it does not appear to have been published as a stand-alone document. Two years later, the Bank’s Africa Region prepared and published an updated strategy entitled, Africa’s Renewed Emphasis on Agriculture and the World Bank’s Support for It. Noting that the World Bank’s annual commitments to African agriculture had doubled during the last two years from $0.5 billion in 2008 to an annual average of $1.2 billion in 2009–10, the 2011 update maintained a strong alignment with the four CAADP Pillars. 7.25 Thus the Bank’s agricultural strategy for Africa has been strongly aligned with CAADP at the corporate and regional levels since at least 2009. To discern the extent to which this has translated into alignment at the country level, the present evaluation has also conducted a keyword search of the World Bank’s Country Partnership Strategies in African countries since 2005 and of project appraisal documents of African agricultural projects since 2006. 7.26 The World Bank has prepared 126 Country Assistance Strategies, Country Partnership Strategies, and Interim Strategy Notes for African countries, including North African countries, since 2005, and 166 Project Appraisal Documents or their equivalent for African projects mapped to the Agriculture and Rural Sector Board since 2006, in addition to the 12 appraisal 91 documents for the CAADP MDTF and associated CTF projects.21 The results of the keyword search for “CAADP”, for the six CTF organizations, and for CAADP processes such as “CAADP Compact”, “NAIP”, “ITR”, and “Business Meeting” are presented in Annex K. Only 16 percent of the Country Partnership Strategies (20 out of 126) and 42 percent of the Project Appraisal Documents (69 out of 166) had references to “CAADP”. References to CAADP processes were even fewer, while references to ECOWAS and SADC were more frequent than references to AUC, NPCA, COMESA, or ECCAS. (See Annex Table K-1.) However, references to “CAADP” in Project Appraisal Documents have been increasing over time from less than 5 percent in 2006 and 2007 documents to an average of more than 60 percent of documents from 2012–2015 (Figure 18). References to “CAADP” in Country Partnership Strategies also increased to more than 40 percent in 2012 before declining thereafter (Figure 19). Figure 18. Occurrences of “CAADP” in Project Appraisal Documents over Time (n=166) Source: Annex K. 21. Each Bank-supported project is supervised by a task team leader who reports to a regional manager who is represented on a Bank-wide sector or thematic board. The project is thereby “mapped” to — or becomes the responsibility of — that particular board. Each project can also identify up to five sectors of the economy supported by the project (such as agriculture, energy, transportation, and the like) and up to five developmental themes (such as environment, social, or urban development) promoted by the project. Selecting projects mapped to the Agriculture and Rural Development Sector Board has proven to be the best measure of the Bank’s agriculture portfolio. Task team leaders reporting to managers represented on other sector boards cannot be expected to know much about CAADP or the alignment of the Bank’s regional agricultural strategy for Africa with CAADP, even if their projects have some agricultural components. 92 7.27 Then the evaluation reviewed the actual textual references to CAADP in the appraisal documents and concluded that five or fewer references to CAADP represented little more than lipservice to CAADP. By this standard, there were only 26 projects (16 percent) that were strongly aligned with CAADP in terms of building on identified CAADP processes at the country or regional levels (Annex Table K-3). All but two of these projects were approved since 2010. Fifteen of the 26 projects (58 percent) were regional projects or development policy operations — the two categories of projects with the most references to CAADP (Annex Table K-2). Seven of the 26 projects were being led by members of the CAADP MDTF team (Wilhelm Janssen, Christian Berger, Melissa Brown, David Neilson, or Tim Robertson). Figure 19. Occurrences of “CAADP” in Country Assistance Strategies over Time (n=126) Source: Annex K. 7.28 These relatively weak strategic linkages between the Bank’s country operations and the CAADP MDTF are not surprising. Notwithstanding the strong alignment of the Bank’s regional agricultural strategy for Africa with CAADP, some task team leaders of Bank￾supported projects have been averse to be associated with CAADP processes in countries where CAADP had yet to establish its credibility. But mostly, the Bank’s country teams and task team leaders tend to regard the Bank’s Country Partnership Strategies as their principal authorizing environment. And the linkages between Country Partnership Strategies and CAADP are relatively low because Country Partnership Strategies reflect the priorities of the government of the country, not just those of the Bank, and for the whole economy, not just the agriculture sector. That is, African governments also have a lot of influence over the 93 strategic linkages between CAADP and the Bank’s country operations in their country. Only 30 percent of the respondents to the Focal Point Survey said their country-level CAADP processes had “much connection and influence” to the national economic policy making and budgeting process in their country, while another 30 percent said country-level CAAAP processes had “some connection and influence” (Volume 3, pp. 31–32). OPERATIONAL AND FINANCIAL LINKAGES 7.29 IEG Reviews of partnership programs have found that operational and financial linkages have been strong when staff members of the partnership team are the task team leaders of Bank projects, and when the partnership program is providing financial resources in support of the Bank’s country operations.22 7.30 The CAADP MDTF program was not designed to provide financial resources to supplement the Bank’s country operations. The only small way in which the MDTF has done so has been to provide $733,000 to supplement the $4.9 million grant from the Bill and Melinda Gates Foundation to prepare agriculture public expenditure reviews in African countries. The only operational linkages have been for country investment projects in Togo and Rwanda being led by members of the Bank’s MDTF team, and for regional projects in support of AFAAS, CCARDESA, and FARA (Annex Table K-3) also being led by the Bank’s MDTF team. 7.31 However, the Global Agricultural and Food Security Program (GAFSP) has been designed to have operational and financial linkages with both CAADP and the Bank’s country programs, thereby providing an indirect institutional link between the MDTF and the Bank’s country operations. INSTITUTIONAL LINKAGES 7.32 GAFSP was founded in April 2010 as a long-term response to the global food price crisis in 2007–08 and donor pledges of $20 billion at the G8 Summit in L’Aquila, Italy, in July 2009. The World Bank is the trustee and the host of GAFSP Secretariat, one of the seven supervising entities for GAFSP projects, and a non-voting member of the GAFSP Steering Committee — similar to its roles in the Global Environment Facility. The other supervising entities are the Asian Development Bank, the African Development Bank, the Food and Agricultural Organization (for technical assistance activities only), the Inter￾American Development Bank, the International Fund for Agricultural Development, and the World Food Program. 22 Two good examples of the latter are the Global Facility for Disaster Reduction and Recovery (GFDRR) and the Global Environment Facility (GEF). GFDRR is one of the two largest technical assistance programs located in the World Bank that is helping recipient countries build their own capacity in relation to disaster preparedness, prevention, and recovery. Most of its technical assistance activities are Bank-executed and closely integrated with the Bank’s country operations because they are directly supporting these country operations. The GEF is one of the three large investment programs in which the World Bank is an implementing agency. It provides financial resources that supplement the Bank’s own lending program for environmental projects and that are often blended with IBRD/IDA finance to achieve global environmental benefits in addition to national benefits. 94 7.33 GAFSP allocates its resources on a competitive basis in response to periodic calls for proposals from IDA-eligible countries followed by a centralized review process by an Independent Technical Advisory Committee of experts. GAFSP has established a specific connection to CAADP by initially requiring that eligible African countries have prepared a National Agricultural Investment Plan that has successfully completed an Independent Technical Review. This was subsequently revised to having successfully completed a CAADP Business Meeting. Supervising entities appraise and supervise projects according to their own policies and procedures, and receive a 5 percent fee for doing so. Countries submitting proposals may choose their own supervising entity with the agreement of that entity. 7.34 As of 2014, GAFSP had allocated $912 million through its public sector window and $77 million through its private sector window, of which $586.4 million (64 percent) and $30.8 million (40 percent), respectively, were allocated to African countries. The World Bank is currently supervising six African public sector projects (Table 10) with GAFSP financing of $208.1 million. The World Bank has also contributed a total of $177.45 million of IDA financing to four of these projects. Table 10. World Bank-Supervised Public Sector GAFSP Projects to Date (US$ millions) Country Project Name Approval Fiscal Year IDA Financing GAFSP Financing Other Donor Financing Ethiopia Agricultural Growth Program 2011 98.5 51.5 103.4 Togo Togo Agricultural Sector Support Project 2011 9.0 19.0 9.0 Rwanda Land Husbandry, Water Harvesting and Hillside Irrigation AF 2014 34.0 50.0 22.0 Burkina Faso Additional Financing for the Agricultural Productivity and Food Security Project 2014 35.95 37.1 Uganda Uganda Multisectoral Food Security and Nutrition Project 2015 27.64 Tanzania Tanzania: Expanding Rice Production 2015 22.9 Total 177.45 208.14 134.4 Sources: www.gfaspfund.org and World Bank data. 7.35 Both the CAADP MDTF and GAFSP have their own governance, management and administration arrangements. In both cases, the Bank has established its trusteeship function in a separate Vice Presidency, now called the Development Finance (DFi) Vice Presidency, in order to shield its trusteeship responsibilities from potentially conflicting interests as a supervising entity. The main differences between the two programs are (a) GAFSP has been established for the long term supported by a Financial Intermediary Fund, and (b) that the GAFSP governance structure is more typical of global and regional partnership programs that are hosted by international organizations. Similar to other FIFs, the GAFSP Steering Committee has essentially complete authority over the allocation of its resources independent 95 of the supervising agencies, and the Bank does not have any oversight responsibilities for projects that are supervised by other supervising agencies. 7.36 The staff of both units are Bank staff, recruited internationally, and enjoying the privileges and immunities accorded to employees of international organizations. They interact collegially in areas of mutual interest. There do not appear to be any conflict of interest issues between the two units. However, Bank staff do enjoy a slight informational advantage in relation to other supervising entities in the case of GAFSP. Bank staff in the MDTF and GAFSP units have access to internal trust fund databases, and DFi staff have access to more information about Bank-supervised GAFSP projects than those of other supervising entities. Establishing and Administering the MDTF 7.37 Trust Fund administration includes negotiating and amending Administration Agreements with contributing donors, receiving and investing funds from contributing donors, negotiating grant agreements with and administering disbursements to CTF organizations, authorizing expenditures from the Bank-executed components, and providing regular financial reports to all concerned consistent with the Bank’s fiduciary role (Box 8). Box 8. The World Bank’s Fiduciary Role The Bank’s Articles of Agreement require that proceeds of any grant are used for purposes for which the grant was intended with due attention to considerations of economy, efficiency and transparency, and in accordance with the procedures specified in the Grant Agreements and other relevant documents. In order to achieve this objective, the Grant recipients are supported by the Bank to put in place the required implementation capacity, including financial management, disbursement and procurement capacity and an acceptable control framework to safeguard assets. The Bank fulfills its fiduciary (financial management, disbursements, procurement) mandate by employing different interventions such as supervision, fiduciary reviews, as well as requiring regular progress and financial reporting, and audits by external auditors. Source: Due Diligence Review of Trust Funds for CAADP, FARA, and AFAAS, November 2014, p. 12. 7.38 The World Bank is by far the largest trustee of global and regional partnership programs. A 2011 IEG evaluation of the World Bank’s Trust Fund portfolio found that donor contributions to Bank-administered trust funds have exceeded donor contributions to IDA since IDA 13 in FY2003–05. At the time of that evaluation, more than 200 donors contributed to 1,075 main trust funds (like the CAADP MDTF) supporting 16 Financial Intermediary Funds and 182 IBRD-IDA managed trust funded programs (IEG, 2011b). Consequently, the World Bank takes the fiduciary administration of donor trust funds very seriously in order to retain its credibility as a trustee and to minimize conflicts of interest arising from the multiple roles that the Bank plays in many trust-funded programs. 7.39 The World Bank commissions an annual single audit of its financial operations, its Internal Audit Vice Presidency conducts periodic audits, among other things, of individual 96 management units and partnership programs, and individual programs like the CAADP MDTF may also commission external audits of their activities. 7.40 In the case of the CAADP MDTF, the Bank’s sector manager to whom the MDTF task team reports requested an external audit of FY2011 expenses due to the high volume of transactions exceptions that had been approved under the Bank-executed part of the program. Subsequently, the next sector manager also commissioned an internal due diligence review of the financial management of the trust funds for CAADP, FARA, and AFAAS at the end of FY2014. 7.41 Both reviews found that the internal control systems were good because the MDTF had applied the standard control procedures that the Bank has established at the corporate level for administering trust funds and grants. The external audit found that the accounting system was satisfactory, but required closer monitoring by the MDTF team. The main issues that arose from these two audits were the high volume of travel costs of Bank staff and consultants, a potential conflict of interest in relation to the Service Agency, document management, and certain ineligible expenditures by some of the CTF organizations. 7.42 The external auditor noted that $1.8 million (39 percent) of the direct costs of the MDTF were disbursed as travel expenses for Bank staff and consultants in FY2011. However, the external auditor issued a “disclaimer of opinion” in relation to these costs due to being an external auditor who did not have access, per Bank policy, to the documents supporting these travel expenditures. Subsequently, the due diligence review investigated this matter further and found no issues raised in the Bank Group audit report for FY2011 with respect to unsupported MDTF expenditures relating to travel. The review also noted that the Bank has strong control systems to ensure that travel costs are accounted for. The high percentage of direct costs associated with travel in FY2011 has subsequently declined largely due to the establishment of the Service Agency. 7.43 The due diligence review identified a potential conflict of interest in relation to the Service Agency if the NPCA were preparing terms of reference and selecting consultants (to be paid for by the Service Agency) for the NPCA’s own benefit. The Bank’s MDTF team has since clarified with both NPCA and HTSPE (the firm managing the Service Agency) that the services HTSPE provides cannot be for the benefit of NPCA, since such benefits should come through its own CTF grant. The team also considered the auditor’s suggestion to transfer this service agency role to an African agency, but concluded that the transactions costs of doing so would be too high, with only 1½ years left in the MDTF, and also in the light of HTSPE’s satisfactory performance to date. 7.44 Finally, the due diligence review did not observe any ineligible expenditures in the Bank-executed trust funds, but did note a number of ineligible expenditures in the recipient￾executed trust funds, most of which the Bank’s task team had already addressed or were already addressing. The external audit found that document management was weak at that stage in the implementation of the MDTF — a finding consistent with that in Chapter 6 above. But overall, the present evaluation finds that the Bank’s MDTF team has administered the MDTF in a satisfactory way in accordance with the Bank’s standard procedures and control systems established at the corporate level. This has included addressing fiduciary issues that have arisen from time to time in a satisfactory way. 97 8. Conclusions and Lessons 8.1 CAADP is a continental framework, primarily for continental and regional initiatives, to help individual African countries reach and sustain a higher path of economic growth through agriculture-led development. As such, it has been a unique process within Africa and within the agriculture sector, with no equivalent in terms of scale or ambition in Africa or other developing regions. 8.2 The MDTF, in turn, has represented collective action by six contributing donors, their African partners, and the World Bank, also primarily at the continental and regional levels, to achieve objectives that each organization could not achieve, or not achieve as efficiently, on its own. As one of the contributing donors has said, the MDTF represented an opportunity to provide support to CAADP implementation at the continental and regional levels, beyond what it was already doing at the country level, and to align this support with a number of other donors. 8.3 The design of MDTF was, at some level, a step into the dark for the donors, African partners, and the World Bank, there being no prior blueprint for the partners to follow to support continental and regional CAADP processes. Therefore, except for the legal constraints associated with (a) the Administration Agreements between the World Bank and the contributing donors for the receipt of funds, and (b) the grant agreements between the World Bank and the recipient organizations for the disbursement of funds, the MDTF adopted a flexible “thousand flowers bloom” approach to its design and implementation. 8.4 The MDTF and related CTFs have contributed to some significant achievements in supporting country and regional CAADP processes; in improving the enabling environment for African agricultural programs and policies; and in strengthening individual, organizational, and system-level capacity. The MDTF has also increased coordination at different levels, particularly among African lead institutions, between sectors, and among MDTF contributing donors. 8.5 Overall, the design of the MDTF worked in terms of fostering country and regional CAADP processes, and, along with the flexible approach to its implementation, produced a second chance — the Malabo Declaration. This does not mean that a follow-on facility should follow the same design as the MDTF. The evaluation draws the following lessons for the consideration of those responsible for designing a follow-on facility in which contributing donors might once again pool their financial resources to support activities, primarily at the continental and regional levels, to achieve agreed objectives over time. Diversity 8.6 A follow-on facility needs to accommodate the diversity among both RECs and countries. ECOWAS and SADC are more coherent economic communities with more established commissions and secretariats, respectively, and with existing agriculture departments. ECOWAS is also a powerful organization with a concrete economic vision and substantial financial resources levied from member states. It opted to provide each member state with $450,000 to organize a Roundtable, sign a Compact, and prepare a NAIP. 98 COMESA is very spread out geographically from Swaziland in the south to Egypt in the North, and to the Comoros, Madagascar, Mauritius, and the Seychelles in the Indian Ocean. COMESA did not have an existing agriculture unit before its CTF project was established, and ECCAS had an agricultural unit with only one staff member. ECCAS member states did not consider ECCAS to be a significant player on agricultural issues before the establishment of its CTF project. 8.7 The CTF grant instrument turned out to be more of a “one size fits all” approach than originally intended which worked better for some recipient organizations like COMESA and ECCAS than for others. Both ECOWAS and SADC found it difficult to work with this instrument, among other things, for their own bureaucratic and administrative reasons. They found it very difficult to align their own systems with those the World Bank for preparing and implementing recipient-executed activities. ECOWAS would have preferred to receive its support entirely from the TA Fund, which worked better for itself in terms of preparing its Strategic and Operational Plan and for its member countries in terms of supporting their country-level CAADP processes. Supply and Demand-Side Relevance 8.8 A follow-on facility needs to support African voices to continually raise awareness and motivate countries to fulfill their Malabo commitments. Both the MDTF program and the CTF projects have been strongly aligned with CAADP principles and broadly defined strategies. Still, the MDTF has been largely a supply-driven program of the continental and regional organizations and the contributing donors. The Sustaining CAADP Momentum exercise, the 2014 Year of Agriculture, the Malabo Declaration, and the Implementation Strategy and Roadmap have also been supply-driven efforts, created and delivered by the organizations made more relevant by the existence of these documents, and the declarations and processes that flow from them. That these documents exist is a step forward, particularly the Malabo Declaration, since it gives national policy constituencies a document endorsed by their head of state or government from which advocate domestic policy reform. These documents have been among the crowning achievements of the AUC’s and NPCA’s involvement in the MDTF. But the existence of these documents alone cannot cause or result in better agricultural policies and investment plans in member states. 8.9 Although African Heads of State and Government have now renewed their commitment at the highest political level to the principles and values of the CAADP process, this does not always reflect uniform demand at the national level or translate into effective action at the national level. The Malabo Declaration views the agriculture sector and agricultural transformation as principal catalysts for economic growth and development. Yet in African countries where extractive natural resources (such as oil, diamonds, or wood) are important in the economy, their governments less frequently consider agriculture to be a strategic sector. Only 30 percent of country-level CAADP Focal Points, responding to their survey, felt that their country’s CAADP processes had “much connection and influence” on national economic policy making and budgeting. Another 30 percent felt their CAADP processes had “some connection and influence,” and the remaining 40 percent “little influence or connection.” The need for articulation around CAADP is still high, preferably by Africa’s own voices. 99 Subsidiarity 8.10 A follow-on facility should continue to focus its support at the continental and regional levels while providing some well-tailored support directly to countries — with the active engagement of the RECs. The evaluation found strong continuing support for the subsidiarity principle. By and large, the AUC, NPCA, and the RECs have played their roles envisaged in the CAADP guidelines, taking into account their capacity at various stages in MDTF implementation. The AUC has been the convener of continent-wide meetings as the political umbrella for CAADP implementation. NPCA has provided strategic guidance and technical backstopping as the technical agency of the AU responsible for agriculture. The RECs have encouraged and facilitated both country and regional CAADP processes. 8.11 However, member states would have preferred to receive direct financial support from the MDTF to undertake CAADP-related activities rather than through the RECs or the TA Fund. ECOWAS was the only REC that took the subsidiarity principle to the next level down by providing each member state with $450,000 from its own resources to organize a Roundtable, sign a Compact and prepare a NAIP. ECOWAS also negotiated funding from donor agencies to support regional technical partners in their work with member states. While recognizing this, there already are many sources of funds for agricultural planning, policy making, and investments at the country level, and an increasing number of agriculture sector donor working groups. What collective action at the continental level can do is more unique. This can support (a) AUC, NPCA, and the RECs in undertaking activities that only they are positioned to do such as developing continental and regional policies and programs, while also (b) providing support for country-level planning, policy making, and accessing investment finance. 8.12 The evaluation finds an even more prominent role for the RECs in the future, given their closer relationship with their member states and the emphasis on increasing intra￾African trade in the Malabo Declaration. To date, they have played an important, albeit largely non-technical role, in getting countries through their CAADP processes from Compacts to Business Meetings. They should continue to be responsible for monitoring and tracking progress at the country level in their member states. But they need more technical capacity to support the growing agricultural trade agenda and for addressing other transnational issues such as managing shared natural resources (e.g. river basins), linking producers to markets, and dealing with droughts and other natural and man-made disasters. A follow-on facility could help strengthen the REC’s technical capacities in these areas while also strengthening the engagement between AUC, NPCA, and the RECs as an essential part of the CAADP architecture. Need for Focus 8.13 A follow-on facility should focus its direct country-level support on helping countries improve their agricultural policies in line with the Malabo commitment to “evidence-based planning, policy efficiency, dialogue, review, and accountability.” The CAADP agenda has expanded considerably since the MDTF was established in 2008 to embrace, among other things, a science agenda, an agribusiness agenda, tertiary agricultural education, women and youth, expanding intra-African trade, enhancing resilience, and 100 climate smart agriculture. This having been said, there is a need to focus future support on specific policy issues of particular relevance to individual countries. The Sustaining CAADP Momentum paper, which was formally endorsed by African Heads of State and Government in early 2013, put more attention on improving agricultural policies, private sector development, and knowledge management. It encouraged organizations supporting CAADP to engage more directly in strategic and technical analysis of key agricultural policies and political economy issues. 8.14 Supporting countries to improve their agricultural policies involves more than evidence-based research on agricultural policies in Africa countries, although this is important. It involves assisting governments in managing the process and the substance of specific policy reforms in individual countries. It is one thing to organize an inclusive stakeholder process such as the NAIP to mobilize more external resources to benefit all stakeholders. It is more difficult to organize such processes to improve policies that may have domestic winners and losers. Implementing such processes requires both financial resources and more engagement of Africa’s professional communities in universities, policy think tanks, and subregional research organizations, etc., to an extent that did not occur under the MDTF. The Pillar Frameworks 8.15 A follow-on facility could continue to support the development of continental strategies on selected topics, but these need to be organized and managed well to obtain quality products that are implementable in member states. The second component of the MDTF program — to support the development of continental and regional frameworks for each of the four CAADP Pillars — was never implemented as originally designed. Only one PLI met the requirements to receive a CTF grant — the Council of Ministers of Agriculture for West and Central African States — although a second PLI — the Forum for Agricultural Research in Africa (FARA) — did receive support from a separate Bank-administered trust fund. The AUC and NPCA decided in 2010, with the concurrence of the MDTF Partnership Committee, to take a different approach to harnessing Africa’s professional communities in support of CAADP, which led to the preparation of the Knowledge, Information, and Skills (KIS) initiative. But this initiative was never implemented due to the focus on the 2014 Year of Agriculture and follow-up activities to the Malabo Declaration. And the Malabo Declaration envisages a new organizational structure for the provision of technical support, which requires yet another rethinking of KIS. Thus, the MDTF missed an opportunity to focus CAADP processes more. 8.16 However, work has taken place on developing continental strategies in some Pillar￾related areas. The development of Science Agenda for Agriculture in Africa in Pillar IV and the Continental Agribusiness Agenda in Pillar II represent contrasting efforts in this regard. A follow-on facility could continue to support the development of continental strategies on topics such as small farmer development, incentives to private sector (agribusiness) investments, intra-African trade, data collection and reporting, since the latter feeds into the proposed and important peer review process. But these should not be developed by a small, relatively closed group of AUC or NPCA staff, like the way in which the Continental Agribusiness Strategy was developed. Their staff should be the conveners, not the authors, 101 working with leading organizations in each field to convene the best team to complete each task, like the way in which the Science Agenda was developed. The follow-on facility could spend some resources for such continental level activities that are prepared for endorsement by the AU Summit and reported back to the Summit. Each effort would pull the evidence together, present it in the AU hierarchy of meetings, get it adopted by Heads of State and Government, and then feed it into the biennial peer review mechanism. This could also assist in focusing the Malabo agenda on fewer topics. Monitoring and Reporting 8.17 A follow-on facility needs to take a more strategic and committed approach to addressing the overall results agenda. The six CTF organizations have been responsible for monitoring progress in relation to their project’s Results Frameworks, and the RECs have also been responsible for monitoring the progress of CAADP implementation in their region in partnership with ReSAKSS. Monitoring and reporting have been the weakest aspects of the implementation of both the CTF projects and CAADP. Some CTF organizations failed to recruit monitoring and evaluation (M&E) specialists. Others recruited specialists who were insufficiently trained for the task, or who got drawn into other, deemed higher priority, tasks. Interviews with CTF staff revealed the absence of a results culture in their organizations, and limited demand for M&E for use in planning, since planning was based more on seeking financial resources, primarily from donors, than on monitoring past results. Member states have also experienced difficulties in recruiting M&E specialists, so that no one — not even ReSAKSS — has been able to determine to what extent CAADP has contributed to the achievement Level 2 outcomes (in the CAADP Results Framework) at the country level. 8.18 The appearance of two separate results frameworks towards the end of the CTF projects has also caused some confusion — one project-based framework for the MDTF in 2013 and the second program-based framework for CAADP in 2014. This is symptomatic of a common issue. While Development Partners have often expressed good intentions to coordinate and streamline their monitoring and evaluation processes with those of recipient countries, this has often proven more difficult to achieve in practice. Development partners have their own requirements for project-level M&E to satisfy their own domestic constituencies which are not necessarily consistent with the efforts of recipient countries to establish their own more sector-based M&E systems. Development partners also typically have more funds available to implement their project-based systems. There is a need to address these tensions and for Development Partners to put a lot of effort in the post-Malabo phase to align their M&E systems with the CAADP Results Framework. Experience in the health sector, for example, shows that this is more than a short-term exercise. Governance and Management 8.19 A follow-on facility needs more transparent and accountable governance and management arrangements. The MDTF has been governed by a small stakeholder body — the Partnership Committee — chaired by the AU and supported by the NPCA as the Secretariat and the World Bank as the Fund Administrator. The four RECs, civil society organizations, the commercial private sector, and knowledge institutions have also had one voting representative each, and the contributing donors two voting representatives. The 102 Partnership Committee was initially accountable to the CAADP Partnership Platform, and subsequently to the AUC–DREA Commissioner and the NPCA Chief Executive Officer. 8.20 However, the Partnership Committee has not been a typical governing body of a partnership program (a) that exercises strategic direction and oversight of the program, (b) that is accountable for all the activities supported by the program, and (c) in which its secretariat has signing authority over the expenditures of the program. It has functioned more like an internal management committee. Its primary function has been to make recommendations to the World Bank concerning the allocation of MDTF resources. 8.21 The Partnership Committee has been a legitimate body composed of the primary CAADP stakeholders, in which the World Bank has endeavored to respect and sustain the African ownership of CAADP and the MDTF without compromising its legal trusteeship obligations to the contributing donors. But the work of the Partnership Committee has suffered from a lack of transparency and mutual accountability. It chose not to establish its own website independent of the CAADP website maintained by NPCA. Based on the difficult experience that this evaluation has had in assembling documentary evidence, the program should have at least established a private, “members only” website to make more accessible the most important MDTF documents, including the minutes and supporting documents of the PC meetings. Institutional Arrangements 8.22 A follow-on Malabo facility needs to be designed for the long term, with institutional arrangements that could last a long time. While there seems to be some willingness among potential donors to support the establishment of a follow-on Malabo facility, there appears to be little support for a repeat of something close to the MDTF. While the independent Mid Term Review in 2011 found that the CTF projects had been highly appropriate instruments that supported the priority organizations, the present evaluation found less enthusiasm for proceeding with second round of CTF grants. 8.23 Even for the four CTF projects with satisfactory outcomes, the recipient organizations have found the administrative burden of complying with all World Bank procedures for recipient-executed grants to be high. They have appreciated the organizational strengthening that has occurred through the Bank’s implementation support, but they have questioned the necessity for the Bank to apply the same degree of oversight and compliance for these relatively small grants as for regular Bank-supported investment projects. Clearly capacity has been built in some important areas, but there are also clear limitations as to what a CTF grant can accomplish, particularly in the areas of accountability and sustainability. 8.24 Finally, if a follow-on Malabo facility were to provide direct support to countries to improve their agricultural policies, to access investment finance, and to establish more effective monitoring and reporting systems, then establishing CTF projects in multiple countries is obviously not practical. Some other support mechanism needs to be explored, such as block grants or competitive grants. 103 8.25 The above lessons imply that a follow-on facility should have several sub-facilities — for overall CAADP coordination, for capacity building at the continental and regional levels, and for assisting countries in improving their agricultural policies and their access to investment finance — while harnessing Africa’s own professional communities for these purposes. Such a facility should not attempt to provide investment finance because this would dilute its focus and compete with already existing facilities such as GAFSP. 8.26 The biggest challenge will be putting in place institutional arrangements at the continental level that provide for legitimate and effective governance and management of the facility, efficient fiduciary trusteeship of the donor funds, transparent allocation of resources among activities, effective supervision of the activities supported, and effective monitoring and reporting of the results. Letting a thousand flowers bloom is a useful approach to begin with, if only to find out where the flowers grow best. But eventually things need to be institutionalized to bring more order and discipline to any process that wants to achieve sustainable long-term results. 104 References African Union. 2015. “Continental Agribusiness Strategy.” Inaugural Conference of the Specialized Committee on Agriculture, Rural Development, Water, and the Environment. September. Aparisi, Alban Mar, Léopold Ghins and Jean Balié. 2015. Initiatives for Agricultural Public Expenditures Monitoring and Analysis in Africa: A Comparative Review and Analysis. FAO, Monitoring and Analysing Food and Agricultural Policies (MAFAP). AUC. 2014. Malabo Declaration on Accelerated Agricultural Growth and Transformation for Shared Prosperity and Improved Livelihoods. AUC. 2015. “Self-Assessment Report on Implementation Support from CAADP Multi Donor Trust Fund, 2011–2015.” AUC and NPCA. 2015a. “The CAADP Results Framework 2015–2025: Going for Results and Impacts.” AUC and NPCA. 2015b. “Implementation Strategy and Roadmap.” AUC and NPCA. 2015c. “The Programme of Work, 2015.” Badiane, Ousmane, T. Makombe, and G. Bahiigwa (eds). 2014. Promoting Agricultural Trade to Enhance Resilience in Africa. ReSAKSS Annual Trends and Outlook Report 2013. International Food Policy Research Institute (IFPRI). Badiane, Ousmane and T. Makombe (eds). 2015. Beyond a Middle Income Africa: Transforming African Economies for Sustained Growth with Rising Employment and Incomes. ReSAKSS Annual Trends and Outlook Report 2014. International Food Policy Research Institute (IFPRI). Cathala, Hubert. 2015. System for “Catching the Picture of CAADP Support Flows: Final Report.” Joint Africa-EU Strategy Support Mechanism. April. COMESA. 2015. “COMESA MDTF Self-Assessment as at 13th October 2015.” Dietvorst, Désirée and Kah Walla. 2016. “CAADP & Country Systems: Summary of Findings & Proposed Way Forward.” Contribution to NPCA Guidelines. January. ECCAS. 2015. “CEEAC — Auto-évaluation du Fonds Fiduciaire Multi Donateurs (FFMD) du Programme Détaillé pour le Développemnt de l’Agriculture Africaine (PDDAA/CAADP).” ECOWAS. 2015. “Independent Evaluation of CAADP Multi-Donor Trust Fund: Self Assessment.” European Centre for Development Policy Management, Laboratoire d'Analyse Régionale et d'Expertise Sociale, and the Economic and Social Research Foundation. 2014. Independent Assessment of the CAADP Multi-Donor Trust Fund. ECDPM Discussion Paper No. 158. February. FARA. 2014. Science Agenda for Agriculture in Africa (S3A): “Connecting Science” to Transform Agriculture in Africa. Accra, Ghana: Forum for Agricultural Research in Africa. 105 Fraser, Erika and Allyson Thirkell. 2013. “Mapping Investment Flows into African Agriculture: The Role of CAADP.” Social Development Impact for the CAADP Multi Donor Trust Fund. June. Global Agriculture and Food Security Program. 2015. Annual Report 2014: Reducing Hunger, Increasing Incomes. Independent Evaluation Group of the World Bank and the OECD/DAC Network on Development Evaluation. 2007. Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Independent Evaluation Group of the World Bank. 2011a. The World Bank’s Involvement in Global and Regional Partnership Programs: An Independent Assessment. Independent Evaluation Group of the World Bank. 2011b. Trust Fund Support for Development: An Evaluation of the World Bank’s Trust Fund Portfolio. Jonasova, Marketa. 2014. “Thematic Review of the CAADP MDTF and CTFs on Gender Considerations, Inclusion and Poverty Focus, and Climate Change Actions,” Input for CAADP MDTF2. July. Kalu, Udo. 2012. Independent Audited Financial Statements (Transactions Review), January. Kingsmill, William, Amdissa Teshome and Stephen Tembo. 2011. Mid Term Review of the Multi Donor Trust Fund Supporting the Comprehensive Africa Agriculture Development Programme. November. Muhoho, Ido. 2014. “Due Diligence Review of Trust Funds for Comprehensive Africa Agriculture Development (CAADP).” November. NPCA. 2010a. Accelerating CAADP Country Implementation: A Guide for Implementors. NPCA. 2010b. “CAADP Multi-Donor Trust Fund: Operations Manual Version 2.” April. NPCA. 2013. “Sustaining CAADP Momentum: 2013-2023. The CAADP Results Framework.” NPCA. 2015. “NEPAD Agency: Self-Assessment of Implementation Support from MDTF.” NPCA and AUC. 2013a. “CAADP MDTF: Operations Manual Version 4.” Draft version. NPCA and AUC. 2013b. “Knowledge, Information and Skills for African Agriculture: A CAADP Programme. Operational Plan: 2013-2016.” Ogiogio, Genevesi. 2016. “CAADP Partnership Architecture Review: Draft Final Report.” African Centre for Institutional Development, February. Pinto, Yvonne, Colin Poulton, Tim Frankenberger and Olu Ajayi. 2014. “African Agriculture: Drivers of Success for CAADP Implementation, Synthesis Report.” March. Poulton, Colin et al. 2014. “The Comprehensive Africa Agriculture Development Programme (CAADP): Political Incentives, Value Added and Ways Forward.” Futures Agriculture Working Paper 77. February. 106 SADC. 2016. “CAADP Child Trust Fund: Self-Assessment Report.” Tawonezvi, Patrick, Elijah Phiri and Augustin Wambo Yamdjeu. 2013. “A Qualitative Review of CAADP in COMESA Member States and the Contributions of the COMESA Secretariat.” Revised Final Draft. November. Thirkell, Allyson, John Sutherland and Mandy Littlewood. 2012. “Review of the CAADP MDTF Results Framework, Volume 1 – Main Report.” Social Development Impact for the World Bank. May. Tisch, Sarah et al. 2015. Final Performance Evaluation Report of the Regional Strategic Analysis and Knowledge Support System (ReSAKSS). Social Impact, Inc., for U.S. Agency for International Development. June. Wales, Michael, Eugene Terry, and Steve Ashley. 2012. “Knowledge, Information and Skills for African Agriculture: A Proposal for Sector Transformation through CAADP.” May 31. Wales, Michael and Patrick Tawonezvi. 2015. “CAADP: Service Agency Review, Final Report,” May. World Bank. 2007. “A Management Framework for World Bank-Administered Trust Funds.” World Bank, 2008. “A Multi-Donor Trust Fund To Support The Comprehensive African Agricultural Development Programme (CAADP),” November 10. World Bank, 2011. “Africa’s Renewed Emphasis on Agriculture and the World Bank’s Support for It: An Update.” Africa-Agriculture Strategy Report. World Bank CAADP MDTF Team. 2012. “Mid Term Review: Trust Fund Status Report.” Washington DC, April. World Bank. 2009–2015. Semi-Annual Stakeholder Status Reports, October 2009 to March 2015. World Bank. 2015. “World Bank Self-Assessment Report on Bank-Executed Activities.”