EVALUATION OF USAID/EAST AFRICA SUPPORT TO THE ASSOCIATION FOR STRENGTHENING AGRICULTURAL RESEARCH IN EASTERN AND CENTRAL AFRICA (ASARECA) REVISED FINAL REPORT AUGUST, 2011 This publication was produced for the United States Agency for International Development. It was prepared by DAI. EVALUATION OF USAID/EAST AFRICA SUPPORT TO THE ASSOCIATION FOR STRENGTHENING AGRICULTURAL RESEARCH IN EASTERN AND CENTRAL AFRICA (ASARECA) REVISED FINAL REPORT Program Title: SFSA Evaluation of the Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA) Sponsoring USAID Office: USAID/EAST AFRICA Contract Number: AID-623-I-10-00003 / AID-623-TO-11-00003 Contractor: DAI Date of Publication: August 2011 Author: Roger Kirkby, Eugene Terry, Tilahun Zeweldu The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS ACKNOWLEDGMENTS ............................................................................................................... V ABBREVIATIONS ...................................................................................................................... VII EXECUTIVE SUMMARY .............................................................................................................IX INTRODUCTION .......................................................................................................................... 1 1. THE REGION OF EASTERN AND CENTRAL AFRICA ........................................................................ 1 2. ASARECA AND USAID ............................................................................................................. 2 3. EVALUATION METHODOLOGY ...................................................................................................... 4 THE FINDINGS: RESPONSES TO QUESTIONS ASKED BY USAID IN NINE KEY ISSUE AREAS ............................................................................................................................ 11 1. REORGANIZATIONAL QUESTIONS .............................................................................................. 11 2. INSTITUTIONAL CAPACITY QUESTIONS ....................................................................................... 16 3. TECHNOLOGY AND INNOVATION ................................................................................................ 19 4. BIOTECHNOLOGY ...................................................................................................................... 23 5. POLICY ..................................................................................................................................... 27 6. REGIONAL SPILLOVERS ............................................................................................................ 30 7. BUILDING CAPACITY OF NATIONAL INSTITUTIONS ...................................................................... 33 8. CAADP ................................................................................................................................... 37 9. FUTURE PROGRAMS ................................................................................................................. 39 STRATEGIC FINDINGS ON THE FIVE OBJECTIVES OF THE RFTOP’S EVALUATION SCOPE ............................................................................................................... 41 ASSESS THE RELEVANCE OF THE USAID-FUNDED PROGRAMS/PROJECTS IN ASARECA TO CLEARLY ARTICULATED OBJECTIVES AND EXPECTED RESULTS ............................................... 41 DETERMINE THE EFFECTIVENESS AND EFFICIENCY OF ASARECA IN IMPLEMENTING THE USAID-FUNDED ACTIVITIES ......................................................................................................... 42 EVALUATE THE IMPACTS OF THE USAID-FUNDED ASARECA PROGRAMS .................................... 44 ASSESS THE SUSTAINABILITY OF THE ASARECA RESEARCH PROGRAM ....................................... 45 DOCUMENT LESSONS LEARNED FOR FUTURE USAID/EAST AFRICA PROGRAMMING ...................... 46 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA i RECOMMENDATIONS AND LESSONS LEARNED ................................................................. 49 KEY ISSUE 1: REORGANIZATION.................................................................................................... 49 KEY ISSUE 2: INSTITUTIONAL CAPACITY BUILDING ........................................................................ 49 KEY ISSUE 3: TECHNOLOGY AND INNOVATION ............................................................................... 50 KEY ISSUE 4: BIOTECHNOLOGY .................................................................................................... 50 KEY ISSUE 5: POLICY ................................................................................................................... 50 KEY ISSUE 6: SPILLOVERS ............................................................................................................ 51 KEY ISSUE 7: BUILDING CAPACITY OF NATIONAL INSTITUTIONS ..................................................... 51 KEY ISSUE 8: CAADP .................................................................................................................. 51 KEY ISSUE 9: FUTURE PROGRAMS ................................................................................................ 52 ANNEX A: SCOPE OF WORK AND EVALUATION METHODOLOGY .................................... 53 ANNEX B. EVALUATION METHODOLOGY AND MODIFICATIONS ...................................... 61 ANNEX C: LIST OF PEOPLE CONTACTED ............................................................................. 69 ANNEX D: REFERENCES AND KEY LITERATURE CONSULTED ......................................... 75 ii EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA TABLES AND FIGURES TABLE 1 Financial Investments by USAID/EA (Grant No. 623-A-00-02-00095-00) in ASARECA, 2002-2006 (extended to 2007) .......................................................................... 2 2 Financial Investments by USAID/EA (Grant No. 623-A-00-06-00082-00) in ASARECA, 2006-2010 (Extended to 2011) ......................................................................... 3 3 Total Number of Project Partners in Each of Six Programs ............................................... 15 4 Ratios of Overhead Cost vs Project Activities for the Biotechnology Program .................. 17 5 The M&E Performance Monitoring Plan ............................................................................. 18 6 Indicators of Progress by Selected Programs Since OP (totals for Staple Crops, Biotech, PAAP, NRM) ........................................................................................................ 12 7 Summary of Technologies and Innovations Generated and Made Available for Uptake ................................................................................................................................ 23 8 Biotechnology Projects Implemented from 2005 to 2007 ................................................... 24 9 Post-Op Projects of Biotechnology Program ...................................................................... 25 10 ASARECA Countries and Their Biotechnology and Biosafety Policy Status, as of 2010 .......................................................................................................................... 27 11 Policy-Related Areas and Progress Achieved ................................................................... 28 12 Policy-Oriented Interventions Developed by ASARECA/PAAP with Selected Partners and Stakeholders ................................................................................................. 29 13 Some Examples of Spillovers ............................................................................................ 32 14 Number of Capacity-Development Recipients by Country and Type of Training, for Biotech and Staple Crops Only ..................................................................................... 34 15 Research Staffing and Total Research Sub-Grants Advanced by ASARECA to Member NARIs ................................................................................................................... 35 16 MSc Students Program Supported by SCARDA ECA ....................................................... 36 17 Number of Participants of ASARECA ‘s Learning and Mentoring Workshops ................... 36 18 Status of CAADP Process in ASARECA Member Countries ............................................. 38 19 Pre-Compact Services to be Delivered by ASARECA ....................................................... 38 20 Post-Compact Services to be Delivered by ASARECA ..................................................... 38 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA iii iv EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA FIGURE 1 Development Domains in Eastern and Central Africa ........................................................ 12 2 Central role of ASARECA for Dissemination of Technologies and Innovations ................. 22 3 Application of Modern Biotechnology and GM Crop Development Cycle .......................... 25 ACKNOWLEDGMENTS We gratefully acknowledge the support of the Executive Director, Deputy Executive Director for Programs, and all the staff of the Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA), and of Dr Peter Ewell and his colleagues at U.S. Agency for International Development/East Africa (USAID/EA). We wish to thank the numerous respondents—researchers, research managers and policy makers, ASARECA Board members, seed trade representatives, and small-scale farmers—that we interviewed during field visits to Ethiopia, Kenya, Rwanda, and Uganda. Their time and insights were valuable inputs to our work. We acknowledge the sharing of insights by the members of the Mid-Term Review (MTR) of the ASARECA Multi-Donor Trust Fund (MDTF), to the extent that this was possible. Our joint debriefing session with ASARECA Secretariat staff after our return from the field was valuable to us, and we hope also to the MTR and ASARECA. For their efficient technical and logistical support and advice throughout this task, we thank Don Humpal, DeWitt Tillett, and others at DAI, as well as George Okwir, our research assistant in Uganda. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA v ABBREVIATIONS AFAAS African Forum for Agricultural Advisory Services AHI African Highlands Initiative ASARECA Association for Strengthening Agricultural Research in Eastern and Central Africa ASTI Agricultural Science and Technology Indicators BARNESA Banana Research Network for Eastern and Central Africa CAADP Comprehensive Africa Agriculture Development Program CGIAR Consultative Group for International Agricultural Research CIAT Centro Internacional de Agricultura Tropical / International Center for Tropical Agriculture CIP International Potato Center COMESA Common Market for Eastern and Southern Africa CORAF/WECARD West and Central African Council for Agricultural Research and Development CRSP Collaborative Research Support Programs DONATA Dissemination of New Agricultural Technologies in Africa DRC Democratic Republic of the Congo EAC East African Community ECA Eastern and Central Africa ECABREN Eastern and Central Africa Bean Research Network ECAPAPA Eastern and Central Africa Programme for Policy Analysis and Advocacy [of ASARECA and now absorbed in PAAP] EARRNET Eastern Africa Rootcrops Research Network ESRF Economic and Social Research Foundation, Tanzania FAAP Framework of African Agricultural Productivity FARA Forum for Agricultural Research in Africa FTE Full-Time Equivalent FTF Feed the Future [initiative of USAID] GFAR Global Forum for Agricultural Research GMO Genetically Modified Organism EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA vii viii EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA IARC Internal Agricultural Research Center ICU Information and Communication Unit [of ASARECA] IFPRI International Food Policy Research Institute IITA International Institute for Tropical Agriculture ILRI International Livestock Research Institute IPTA Integrated Platform for Technology Adoption ISAAA International Service for the Acquisition of Agri-Biotechnology Applications M&E Monitoring and Evaluation MDTF Multi-Donor Trust Fund MEAPU Monitoring, Evaluation and Planning Unit [of ASARECA] MTR Mid-Term Review NaCRRI National Crops Resources Research Institute [Uganda] NARI National Agricultural Research Institute NARS National Agricultural Research System NGO Non-Governmental Organization NRM Natural Resource Management OP Operational Plan PAAP Policy Analysis and Advocacy Program [of ASARECA] PCD Partnerships and Capacity Development [Unit of ASARECA] PRAPACE Programme Régional d'Amélioration de la Pomme de Terre et de la Patate Douce en Afrique Centrale et de l'Est [Regional Potato and Sweetpotato Improvement Network in Eastern and Central Africa] RABESA Regional Approach to Biotechnology and Biosafety Policy in Eastern and Southern Africa RFTOP Request for Task Order Proposals, USAID RUFORUM Regional Universities Forum for Capacity Building in Agriculture SCARDA Strengthening Capacity for Agricultural Research & Development in Africa [former program of FARA] SRO Sub-Regional Organization USKM Up-Scaling and Knowledge Management [Program of ASARECA] USAID United States Agency for International Development EXECUTIVE SUMMARY The Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA) was established in 1994 as a sub-regional nonprofit association of the National Agricultural Research Institutes (NARIs) in 10 countries of Eastern and Central Africa. Its purpose is to catalyze and promote cross-border collaboration in agricultural research that leads to effective and efficient impact across the region. Agriculture contributes at least 30 percent of gross domestic product and employs more than 60 percent of the population, with the largest numbers of the poor living in rural areas. Incomes are limited by low productivity and poor market access. Intraregional trade accounts for only 5 percent of total trade, and the region is not self-sufficient in food. Many trials and projects have demonstrated that improved technologies can double yields and since agro-ecological zones extend across national borders, it is expected that a regional platform can achieve economies of scale. USAID/East Africa has provided ASARECA a total of US$20 million over the 10 years since 2002. This support has covered both a share in core costs of the organization’s Secretariat as well as targeted research and technology dissemination activities. The current evaluation was commissioned to assess the relevance of USAID support, ASARECA’s effectiveness and efficiency in implementation, and the impacts and sustainability of the funded research programs, as well as to document lessons learned for future USAID/East Africa programming. DAI’s evaluation team of three people visited USAID/East Africa; met with management and senior staff of ASARECA Secretariat in Entebbe, Uganda; interviewed selected partners and stakeholders in a sample of four countries (Ethiopia, Kenya, Rwanda, and Uganda) during a period of three weeks in May 2011; and reviewed relevant reports and documentation for additional secondary data and information. This report follows the analysis subsequent to the fieldwork, and an exchange of preliminary findings with the Mid-Term Review (MTR) team examining the performance of ASARECA’s Multi-Donor Trust Fund, and debriefing sessions with ASARECA, USAID/East Africa, and USAID/Washington. The evaluation methodology was modified to some extent because of the assumption that MTR survey results and an assessment of the impacts of donor investments in the period before 2006 would be available by the time the team started work. As the MTR was delayed in its start-up, we spent additional time searching for impact information that we were to obtain from the MTR, and shared some of this with the MTR for its own assessment of the effectiveness of ASARECA’s reorganization, a process that reached its peak activity from 2006 to 2008. As ASARECA’s monitoring and evaluation (M&E) system was unable to provide as much information as we expected, we had to narrow the focus of our searches to targeted USAID research investments in crop commodities improvement, biotechnology, and policy rather than those related to natural resource management. USAID’s 10 years of support to ASARECA governance and to strengthening management has proved critical. The loose association of 17 networks, programs, and projects—largely outsourced to Consultative Group on International Agricultural Research (CGIAR) centers for day-to-day management until 2006— was consolidated with research already within the Secretariat, and other services were strengthened as well. The resultant seven new programs and five units managed by well-qualified scientists have increased coherence in ASARECA’s research portfolio. (The new programs are Staple Crops, Non-Staple Crops, Livestock and Fisheries, Agro-Biodiversity and Biotechnology, Natural Resource Management EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA ix and Forestry, Policy Analysis and Advocacy, Up-Scaling and Knowledge Management). The organization’s capacity for strategic and participatory planning with stakeholders has improved markedly, with a new 10-year strategy and a new five-year Operational Plan developed by 2008. In reorganization, USAID made important inputs to financial management so that a multi-donor trust fund became feasible, and to the M&E Unit, which revised ASARECA’s log-frame, sharpened its indicators, and thereby improved the potential for delivery of tangible outputs along value chains. However, institutional memory within ASARECA is now inadequate, because the organization lost critical M&E functions (apparently related to staff turnover) that just began a gradual revival last year. We recommend that the Board and management take immediate measures to ensure that past data is recovered, retrieval systems are put in place, and the log-frame is applied systematically throughout the organization. Expanding the Board beyond the 10 countries’ NARIs has extended ownership of ASARECA to other research partners (universities, CGIAR centers, and non-governmental organizations) and private sector and farmer representatives. Donors supported these developments in a coordinated manner with a signed Memorandum of Understanding, an expanded donor base of the organization, and a gradual reduction of USAID’s proportion of total support, from about 33 percent to 18 percent. Current focus on a narrower research scope within the new programs, designed to address only higher priority targets and deliver results within a much shorter lifespan, has improved research efficiency, as measured by the overhead costs ascribed by ASARECA to individual programs that are now around 15 percent—similar to the rate charged by the West and Central African Council for Agricultural Research and Development (CORAF/WECARD), ASARECA’s sister sub-regional organization for West Africa; and below those now charged by CGIAR centers (22 to 25 percent) and by U.S. universities in CRSP consortia (20 to 50 percent). However, the above rate does not include all of ASARECA’s much increased Secretariat costs, which can only be justified by viewing its value addition to the region more broadly than just funding research projects. Also, the focus on targeted research challenges has been at the expense of networking among researchers, yet the two approaches are complementary rather than competitive. We recommend that ASARECA and its programs broaden networking capacity and link its current and past projects and collaborators more consistently into ASARECA’s applied scientific family—including with networks and other initiatives that continue independently of ASARECA—to strengthen partnerships, ensure continuity, and enhance the potentials for sharing benefits across the region. USAID support to specific research areas has been highly relevant and focused, with significant impact from delivery of improved crop technologies and methods derived from networks in the earlier half of the decade auguring well for the newer activities. Excellent examples are new bean varieties that yield up to 40 percent more and are being grown by millions of small farmers (most of them women) in the region; new varieties of potato multiplied through improved seed systems being grown in seven countries with benefits to small-scale farmers, seed producers, and food processing firms; orange-fleshed sweet potato varieties for vitamin A nutrition being produced by famers in five countries; cassava and banana disease￾management methods being used in six countries; and quality protein maize being grown in four countries so far. The small number of regional impact studies on past network achievements have been carried out by CGIAR centers with NARS, and ASARECA has significantly under-invested in Impact Assessment. Some of these technologies and innovations are still well below their potential adoption rates. The Up￾Scaling and Knowledge Management Program is contributing further to impact by applying an innovation platform for technology adoption approach, such as for scaling-up soil and water management practices and drought-tolerant varieties for dryland maize production. The current Strategic Plan priorities x EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA benefited from identifying agricultural development domains (with the support of the International Food Policy Research Institute) that generally cross national borders. We recommend that ASARECA programs systematically employ this concept in facilitating the targeted scaling-up of technologies and innovations. Stronger use of the domain map would also strengthen proposal development by making it easier for lead scientists to identify potential project partners beyond their current experience, and encourage closer project collaboration across countries in carrying out joint research. We recommend that all projects with earliest effect include measures to promote spillovers along development domains within and across countries in their design and implementation. In the area of cutting-edge biotechnology, countries in the region are at very different levels of developing usable products and uptake pathways. The two leading countries in research on genetic modification are Kenya and Uganda, with Kenya advancing knowledge on tolerance to drought and Uganda reaching the stage of confined field trials on critical banana and cassava diseases. Additionally, Sudan, as well as Kenya and Uganda, has started field-testing transgenic cotton. Linkages between ASARECA’s and the National Agricultural Research Systems’ (NARS’) biotechnology work is uneven; ASARECA’s programs have focused on capacity building with NARS and biosafety policies. All member states are signatories to the Cartagena Protocol on Bio-diversity, most have a national biosafety framework, but only Ethiopia, Kenya, and Tanzania have passed a bio-safety act. We recommend that ASARECA’s Biotechnology Program update its assessment of biotechnology research status in member countries to better establish priority areas of need, and offer all countries access to low-cost networking, training, and support in partnering with private and/or public technology providers from within and outside the region. Rwanda is making impressive progress in tissue culture application for rapid and mass multiplication of several crops. The great potential for wider use of this relatively low-technology technique for achieving impact would be more appropriately led by ASARECA’s Staple Crops Program. We further recommend that intellectual property rights receive more attention by ASARECA as a whole; within the next year, the Board should empower the organization to develop its own policy on intellectual property rights, including working with NARS to harmonize plant breeders’ rights, to ensure that its members continue to have access to technologies developed with its regional support. Policy research in ASARECA, with the advantages of uninterrupted management and of financial support over the last 10 years — most notably from USAID — has registered important achievements in catalyzing regional trade. Addressing a small number of significant policy constraints through a regional approach in collaboration with well-chosen partners—the majority from outside the NARIs—has been key to the program’s success in harmonizing national laws or regulations affecting seed trade (for which an impact study is available), product quality standards, and crop variety release. While these new or improved national policies are important to the generation of wider impact from other areas of ASARECA’s research, member states have yet to take full advantage of them in reducing the hurdles to cross-border movement and authorization for release of new varieties or other technologies. These barriers limit the impact of proven research products. Therefore, we recommend that the management and Board of ASARECA consider ways in which ASARECA can exert stronger influence for enforcing agreements at the highest regional and national policy levels. Regional support in catalyzing efficient and rapid scaling-up of new technologies and innovations would benefit from further attention by ASARECA to the establishment of harmonized standards or procedures by which technologies of different categories are validated. We also note that, as an African-owned and staffed institution, ASARECA has comparative regional advantage in helping countries to address critical EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA xi but often emotive issues, such as gender analysis in research and the safe application of modern biotechnology methods to crop improvement. ASARECA has been investing in training for capacity development of “smaller, weaker,” and more isolated national systems. Promising outcome stories from postgraduate studies and individual attachments of scientists have been published. However, major challenges remain. The current concentration of projects in Kenya, Uganda, and Tanzania (76 percent of project funds) neither captures adequately the richness of the region’s biophysical and socioeconomic diversity, nor fully addresses the needs of smaller countries to acquire enough practical research experience to adapt technologies to local situations (this often requires more than simple technology transfer from countries that are stronger in this respect). ASARECA needs a diverse, innovative strategy to make sure that all member countries get tangible benefits from regional collective action. We recommend that ASARECA adopt before the end of 2011 a policy that catalyzes mentoring of scientists across the region, to include M.Sc. and Ph.D. dissertation research in all future projects. These measures will increase university participation in regional programs, increase capacity-building opportunities for weaker countries, and enable the latter to participate more fully in future projects without loss of overall quality of regional research. Although ASARECA’s donor support base is good, sustainability and confidence in the organization would be enhanced if member countries were to contribute more. For example, the following public investments were leveraged through ASARECA projects: development of biotechnology platforms in Uganda and Tanzania, and government-funded postgraduate training of critical staff in Rwanda and Sudan as a result of the initial investments from key donors. We recommend that sustainability of a project’s agenda be included as a criterion in the consideration of all future proposals to reduce the need for their longer-term external funding. The Comprehensive Africa Agriculture Development Program (CAADP) is becoming a key focus for guiding agricultural research and development in Africa. ASARECA needs to be more proactive in the wider community of science and technology institutions operating in the region, with a larger and more proactive role in the CAADP agenda related to Pillar IV activities. ASARECA, as one of the major pillar support organizations, must assume leadership in agricultural research, advisory services, and education and training aspects of technology generation, dissemination, and adoption. This has several dimensions, including contributing to technical review of plans, supporting enhanced capacity to provide relevant knowledge and information for the implementation of agriculture sector investment plans, advocating intellectual property rights and regionally harmonized standards at the country level; and catalyzing the application of the principles of the Framework for African Agricultural Productivity. While we noted a general improvement in understanding by respondents in all categories of agricultural R&D in the region, further coordination among them in regional priority setting and action planning would be valuable, especially in the CAADP context. We recommend that ASARECA take on a more pro-active leadership role in catalyzing more effective cooperation among these entities as part of its strategy for adding value to the region. We do not propose any modification to ASARECA’s current structural arrangements. However, relevance and effectiveness would be significantly enhanced if the existing structure were managed in a more interactive manner, seeking value addition across programs, units, and projects as appropriate. Emphasizing the organization’s other value-adding activities in the region rather than its grant management activities alone will enhance ASARECA’s capacity to deliver on its CAADP commitments and to support the value chains prioritized in the USAID Feed the Future initiative. We recommend that this internal reflection—and, where necessary, work plans and budget adjustments—be completed during xii EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA xiii the next nine months, without redoing the strategy and operational plan, and taking advantage of the reduced load of projects as many mature in 2011. Implementation will require further adaptation by program managers and form an important step toward completing the change management process. We conclude that USAID’s investment in ASARECA over the past 10 years has proven well worthwhile, with important outcomes in terms of technologies generated and disseminated, evidence of impact in increased productivity, household incomes and food security, the application of more effective methods and partnerships in scaling-up agricultural technology, harmonization of policies that have proven valuable in supporting market access and intra-regional trade, and strengthened institutional capacity of national institutions. Each of these impact areas represents a significant achievement and, taken together, have established ASARECA as having a unique role in the Eastern and Central Africa region that is complementary to that of other agricultural research and development actors. While the reorganization of ASARECA reduced its trajectory for achieving productivity impact through the former networks, the investment in strengthening its own institutional capacity and that of its members should make it an important partner institution in supporting USAID’s Feed the Future agenda at both regional and national levels. Our evaluation, though, also identified significant deficiencies that need to be addressed. We believe it feasible for these measures to be successfully addressed in the next 6 to 12 months, if both the Board and the management ensure vigorous and rapid action in line with the 17 recommendations made here. INTRODUCTION 1. THE REGION OF EASTERN AND CENTRAL AFRICA As elaborated in the Feed the Future East Africa FY 2010 Implementation Plan, the countries in eastern Africa have some of the highest poverty and hunger rates in the world. In five of the ten member countries of ASARECA (Burundi, the Democratic Republic of Congo [DRC], Eritrea, Kenya, Madagascar, and Rwanda), the proportion of the population living on less than one dollar a day has increased since 1990, despite generally positive national economic growth rates. Of the approximately 200 million people in Kenya, Ethiopia, Uganda, Rwanda, and Tanzania, about 78 million, or 40 percent, were classified in 2008 as poor and are chronically food insecure. In addition, acute food emergencies requiring food aid and other kinds of safety nets have been occurring more frequently over the past two decades. Agriculture is a core economic sector for all countries in eastern Africa, contributing at least 30 percent of GDP and employing over 60 percent of the population. The largest numbers of the poor are in rural areas, their incomes limited by low productivity and poor market access (World Bank, 2007). Many poor rural households are characterized by a low asset base, small farm size, depleted soil fertility, limited investment in improved inputs, and very limited access to services and information. Many trials and projects have demonstrated that improved technologies can double yields or do even better. Such innovations include pest-resistant varieties, improved seeds, better soil management, increased use of fertilizers, and improved health care for livestock. Techniques such as conservation farming, low- or no￾till methods, organic farming, agroforestry, and holistic rangeland management improve soil moisture retention, protect soil surface from erosion and improve soil organic matter (Spielman and Pandya-Lorch, 2009). Adapted technologies and best practices must be made widely available for adoption in appropriate agro￾ecological zones throughout the region. Agro-ecological zones cross national borders. A regional platform can achieve economies of scale, identifying multiple partners at the national level, and moving technologies and knowledge to potential users more efficiently than could be done on a country-by￾country basis alone, provided that the policy and regulatory environment permit cross-border movement and do not impose multiyear delays between the entry of technologies and their release for general use. Poor access to markets for agricultural products and inputs, uncertainty about production and harvest prices, and policy and regulatory barriers are major factors that limit investments in improved practices by farmers. Investments by public and private sectors alike in distribution and delivery of improved seeds, fertilizers, and other inputs are inadequate and poorly coordinated, in part because individual country markets are too small. Increasing regional trade and opening up an integrated regional market for staple foods in eastern Africa will have multiple benefits. Variable weather conditions and other factors mean that output in any given country fluctuates, leading to food shortages in certain areas in some years and, simultaneously, surpluses in other areas. Regional food balance sheets show that some countries face food crises in years when there is more than enough food in the region as a whole. For example, in 2009, Kenya, Tanzania, and Ethiopia were facing maize shortages, whereas Uganda and Malawi had surpluses. In recent years, Tanzania and EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 1 Ethiopia have had surpluses, whereas Malawi has been in deficit. Open regional trade and investments in trade infrastructure to move grains would permit countries and areas within countries to take advantage of regional diversity and different harvest periods for the same and/or substitute crops and livestock products by moving staple foods from surplus to deficit areas (Feed the Future, 2010). Despite recent growth in intra-regional agricultural trade, it accounts for only 5 percent of total trade for the Common Market for Eastern and Southern Africa (COMESA) countries. COMESA member states spend a combined total of $19 billion on food imports annually, and yet intraregional trade in all agricultural products is only $3 billion per year. Reliable access to larger, more predictable markets creates incentives for investments in more efficient value chain services such as storage, warehouse receipt systems, market information, agro-dealer and input supply networks, more efficient delivery of improved technologies, and other services that stimulate increased productivity. Yet, regional trade is impeded by various kinds of non-tariff barriers, such as differing national laws on seed trade and quality standards for agricultural products and differing food safety standards. Alliances of public and private partners can help open up markets, increase the reliability of food supplies, reduce dependence on imports from outside the region, and accelerate agricultural growth. This in turn will stimulate and support broader economic growth and decrease poverty. The Comprehensive African Agriculture Development Programme (CAADP) of the African Union’s New Partnerships for African Development has been adopted as the central framework for African agricultural development. CAADP Pillar IV constitutes the strategy for revitalizing, expanding, and reforming Africa’s agricultural research, technology dissemination, and adoption efforts. 2. ASARECA AND USAID ASARECA was established in 1994 as a subregional nonprofit association of the National Agricultural Research Institutes (NARIs) in 10 countries of Eastern and Central Africa in order to catalyze and promote cross-border collaboration in agricultural research that leads to effective and efficient impact across the region. The 10 countries are Burundi, Democratic Republic of Congo, Ethiopia, Eritrea, Kenya, Madagascar, Rwanda, Sudan, Tanzania, and Uganda. ASARECA, as one of the sub-regional organizations (SRO), is a member of the Forum for Agricultural Research in Africa (FARA), which in turn is a member of the Global Forum for Agricultural Research. ASARECA also aimed to strengthen the collaboration between the International Centers of the Consultative Group for International Agricultural Research (CGIAR) and the NARIs in adaptive research, through guidance in regional priorities, providing for a dialogue and delegating the management of some of ASARECA’s regional research programs. More recently, ASARECA has been mandated by COMESA to provide leadership in its sub-region for the implementation of the Pillar IV agenda in Science and Technology. USAID’s support to ASARECA over the Evaluation period comprised two 5-year Cooperative Agreements totaling US$20 million; this built on a lower level (in dollar terms) of earlier support throughout ASARECA’s three formative 3-year Operational Plan periods between 1994 and 2002. The first 5-year grant within this evaluation’s scope totaled USD 10.45 million (Table 1), and included support for Secretariat activities including governance, developing capacity in administration and finance, program management and partnerships, and the development of policy guidelines for ASARECA. A Monitoring, Evaluation, and Planning Unit (MEAPU) was also supported. These investments assisted the development of ASARECA as a regional platform and regional institution. Six of ASARECA’s research themes were also supported in this grant period: programs for Biotechnology and for Policy (ECAPAPA), both centrally managed at the Secretariat; and networks on cassava (Eastern Africa Rootcrops Research 2 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Network), beans (Eastern and Central Africa Bean Research Network [ECABREN]), potato and sweet potato (PRAPACE) and agroforestry through sub-grants to the International Institute for Tropical Agriculture, the International Center for Tropical Agriculture (CIAT), the International Potato Center (CIP), and the World Agroforestry Center respectively. TABLE 1: FINANCIAL INVESTMENTS BY USAID/EA (GRANT NO. 623-A-00-02-00095-00) IN ASARECA, 2002-2006 (EXTENDED TO 2007) Grant obligated US$ 1. Secretariat Core Activities 2,415,600 2. MEAPU 1,049,300 3. Biotechnology (1) 1,451,600 4. ECAPAPA 2,262,000 5. Networks on Cassava, Beans, and Potato/Sweet Potato 3,271,500 Total Grant Budget 10,450,000 Note (1): Biotechnology was embedded in the sum for MEAPU and has been extracted here for clarity Source: USAID/East Africa From 2006, the networks and programs that had been coordinated largely in a decentralized manner were consolidated at the Secretariat in Entebbe. A 10-year Strategic Plan was developed, and a new 5-year Operational Plan (OP) for the period 2008–2014; these set forth new structures, systems and directions necessary for the conduct and management of ASARECA’S mandate activities. This re-planning and its subsequent implementation were supported by USAID through a second Cooperative Agreement (Table 2). USAID invested in the development of ASARECA as a regional platform and regional institution, including a share of governance and capacity building at the Secretariat, the Monitoring and Evaluation (M&E) Unit, and programs in policy, biotechnology, staple crops, and technology uptake. USAID made a choice to invest in the platform, rather than in specific research activities designed to produce specific outputs, on the hypothesis that there would be clear value-added to regional collective action in research and technology transfer. TABLE 2: FINANCIAL INVESTMENTS BY USAID/EA (GRANT NO. 623-A-00-06-00082-00) IN ASARECA, 2006-2010 (EXTENDED TO 2011 Grant obligated US$ 1. Governance and Secretariat Management 1,322,969 2. Program Management (Planning, Monitoring and Evaluation) 945,279 3. Knowledge Management and Communication 437,406 4. Operationalization of Programs 1,813,321 5. Research Projects 4,714,591 6. Capacity for Emergency Response 113,270 Total Grant Budget 9,346,836 Source: USAID/East Africa All of ASARECA’s Development Partners agreed to cooperate in their support, and signed a memorandum of understanding that defined a basis for cooperation. Three of the development partners contribute to a multi-donor trust fund managed by the World Bank. This evaluation was designed taking into account that a mid-term review of the Trust Fund would also be carried out. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 3 3. EVALUATION METHODOLOGY PHASE I: EVALUATION SCOPE FINALIZATION Sample Countries. The task order included four sample countries: Kenya, Rwanda, Uganda, and Tanzania. Discussions with the COTR and USAID/EA/REGI staff resulted in a revision to the program of country visits to replace Tanzania with Ethiopia. It was concluded that a selection of an Anglophone country outside of the East African Community would provide greater insights into the effectiveness of past investments in capacity building and program support. A conference call with ASARECA leadership on May 13 confirmed this choice. Evaluation Methodology and Coordination with the MDTF-Mid-Term Review (MTR). DAI requested that USAID provide an update on the status of contracting for the independent Multi-Donor Trust Mid-Term Review. USAID’s SFSA Request for Task Order Proposals (RFTOP) SOL-623-11- 000015 incorporated the MDTF-MTR Terms of Reference. Those terms of reference indicated that focus group discussions, beneficiary surveys, and most importantly, impact assessment on USAID-supported interventions prior to re-organization were to be completed by the MDTF-MTR by the end of April 2011. DAI’s proposed methodology and budget was built on the assumption that the MDTF-MTR work had proceeded as indicated. DAI learned that the MDTF-MTR Terms of Reference had not been completed only after we had been awarded the task order. Discussions with USAID/EA/REGI and ASARECA indicated that the work of the MDTF-MTR consultants would not start until after our scheduled fieldwork had been completed. The final TOR for the MDTF-MTR did not include any assessment of USAID supported activities prior to the OP period, or indeed any reference to USAID supported interventions as such. The effect of this change and the delay was to deprive our evaluation team of the data essential to use for the quantitative comparisons that we would have carried out had the information been available. The Evaluation Team discussions with the USAID/EA/REGI indicated 1) that the ASARECA Evaluation Team would have access to the three prior ASARECA Multi Donor Trust Fund review missions, which provide sufficient, if not always "gold standard" evidence to apply to the core issue of whether the investment in building the regional research platform has been worth it and whether more investment is advisable; 2) that as long as the report was open and transparent about the data and information issues encountered, the team should be able to provide responses to most of the questions asked in a way that would be accepted by USAID, the Multi-Donor Trust Fund (MDTF) parties, and ASARECA itself. Discussions with the USAID/EA/REGI staff and the COTR were held in Nairobi on coordination with the MDTF-MTR team. Arrangements were made to put the two team leaders in contact to discuss times and places where exchange of information could take place once the MDTF-MTR’s team work began. We had a first conference call with the MTR team on May 24 and a first meeting was held on June 7, followed with email and telephone conversations. The Team submitted a draft report to USAID on June 22, a date modified with Contracting Officer approval upon recommendation by the COTR to permit greater interaction with the MDTF-MTR team. The two teams held a joint feedback session with ASARECA staff on June 17, we shared the 7-page draft summary of key findings with the MDTR-MTR on June 24, and informal consultations continued thereafter. The final report was shared with the MTR team leader after delivery to USAID on July 22 with comments received and incorporated in this revised final version of the evaluation report -- with COTR email approval of inclusion of the modifications on August 8.. 4 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA USAID/Washington De-Briefing. The COTR requested and DAI agreed that Dr. Eugene Terry would provide a de-briefing to interested USAID/W staff on or about June 6, 2011. Dr. Terry gave the briefing as scheduled. USAID/EA Debriefing. Dr. Roger Kirkby presented preliminary findings and issues encountered to the COTR and REGI staff on June 8, 2011. ASARECA and Country Coordination. The USAID COTR kindly made initial contact with ASARECA leadership and USAID representatives in the four (4) sample countries to supply them with the Evaluation Team’s scope of work and to transmit proposed working schedules. The Evaluation Team held consultations with ASARECA headquarters management and staff on May 16 and 17 to discuss the scope, start interviews, and to start data collection with program managers, financial managers, and M&E staff. Impact Analyses. Discussions with ASARECA staff revealed only two impact analyses performed on ASARECA-supported programs, one in the policy area and the other in biotechnology. Our original scope indicated that the team could not perform impact analysis, but would use already available impact analyses to compare the ASARECA program with CGIAR-benchmarked standards. We attempted to locate impact analyses specific to ASARECA programs from each country, from funding agencies, from CGIAR centers and in the literature, with limited success. CGIAR centers and/or NARS scientists identified impact studies related to the bean, cassava and potato networks, and carried out by NARS and/or CGIAR scientists. Several success stories in the making were described in a recent ASARECA Newsletter, but do not constitute formal impact studies. The scarcity of impact analysis of the ASARECA programs made the comparison of impact performance with CGIAR measures a moot point. Evaluation Methodology. Evaluation methods used are described below and, with modifications and changes, are contained in Annex B. PHASE II: FIELDWORK In the course of this evaluation, team members met with nearly 180 persons in individual and group settings. Their institutional affiliations, names, and roles are provided in the contact Annex C. PHASE III: ANALYSIS AND REPORTING: QUESTIONS ASKED AND METHODS APPLIED. Ordering of Key Issues. We placed the reorganizational questions in first position as the reorganization has been the primary event in the institutional evolution of ASARECA over the evaluation period. 1. Reorganizational Questions From the reorganization process begun in 2006, there were five questions to consider. The Evaluation team interviewed the Deputy Executive Director of ASARECA and his Program and Unit Managers, three Board members representing NARIs and the CGIAR Centers, and four other senior NARS leaders and also other scientists and stakeholders across four countries. First, “what evidence is there of value added as the institution has moved from a loose association of commodity networks managed separately by the CGIAR centers to consolidated regional mechanisms?” The MDTF-MTR did not provide post-reorganization intermediate outcome indicators within the timeframe of this evaluation, and we could not find within ASARECA direct documentation of the pre￾reorganization intermediate outcome indictors. Therefore, our interviews became focused on EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 5 programmatic outcomes and impacts of pre-reorganization programs (cassava mosaic disease tolerance, policy influence, etc), and we broadened the planned number of network-related interviews. Second, “would the re-organization have been successful with or without the support from USAID/East Africa?” Interviews with program managers and researchers were used to address this question (including some respondents who spanned the pre- and post-Operational Plan periods). Third, “has ASARECA’s performance been disappointing in any areas over the years?” We answered this question through the same structured interviews using the results framework as a prompt and asking for significant differences in performance, positive or negative, between the CGIAR-led commodity networks and the larger and broader regional programs. The remaining two questions relate to the governance reforms and ASARECA’s performance. “Have these changes helped to broaden ASARECA’s effective partnerships beyond the 10 NARIs to the broader agenda of CAADP Pillar IV?” The evidence base for the interviews around this question were the partnership listings for the individual programs, set against the partnership profiles of selected CGIAR, CRSPs, and other regional research programs. The total number of project partnerships in the program areas was researched directly. The time required to do this work directly, rather than use the MTR figures, displaced the time that we would have spent researching similar program profiles with other international and regional research programs. Last, “have the changes had any negative impacts?” We planned to answer this question using the results of the MTR on governance and operational indicators to guide structured interviews to compare pre-OP and post-OP status; however, as noted earlier, the final, contracted TOR for the MTR did not include any assessment of activities prior to the OP, i.e. the period upon which it was asked to focus. We sought and interviewed organizations and individuals who spanned the two periods, but quantitative data points to compare pre-and post-reorganization changes were not available. 2. Institutional Capacity Questions “Has coordinated support from the donors to ASARECA’s single OP proved to be an effective strategy for building institutional capacity?” USAID has asked for a cost and benefit approach focused on ASARECA’s performance in the use of USAID funds as an organizer of national institutional and partner (international institutions, extra-regional universities, private companies, and NGOs) capacities to support agricultural technology development. We proposed that the primary quantitative measure be the following: overhead and general administrative costs (including capacity building at the secretariat) of ASARECA compared to the funds mobilized through ASARECA to support research and national agricultural research capacity development. Counterfactual cases would primarily be CGIAR regional programs and CRSPs with regional presence, where comparable costs and uses of funds can be identified. This measure can then be qualified using the answers from key issues 3-7 on the impact side of research expenditure. However, comparable data for ASARECA’s performance pre-OP and post-OP was unfortunately unavailable and represents a key weakness in the institutional memory of ASARECA and its M&E system. While the team compared overhead rates with institutions in the region, little relevant data that would facilitate comparison of ASARECA’s performance to that of other institutions operating regionally (counterfactuals) was available. The responses obtained were based largely on qualitative analysis of this issue and responses to structured interview questions with ASARECA Management, three Board members and Program Managers, and stakeholders listed as contacts in Annex C. 6 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 3. Technologies and Innovations USAID’s hypothesis is: “Regional USAID investments in science and technology through ASARECA have proved themselves to be an effective mechanism for making widely applicable technologies available in multiple countries.” Our evaluation team intended to use the MTDF-MTR assessment of existing studies for the pre-reorganization period, combined with other published impact studies focused on technological innovation. A useful benchmark for the counterfactual question exercise has been developed by the CGIAR Science Council Secretariat using a meta-analysis approach. These two sources were to be used to address the hypothesis and the question: “What evidence is available [to compare USAID support to ASARECA in terms of technological impact against other organizational alternatives] and what are the lessons learned?” We obtained ASARECA presentations and reports, and interviewed staff with a focus on Staple Crops as the largest program and having an agenda supported by USAID over the entire 10-year period. Since quantitative documentation from ASARECA’s project performance monitoring and evaluation was largely unavailable until 2010 (due to the M&E deficiencies noted in the report under the Institutional Capacity issue, and the exclusion of pre-OP impact assessment from the final TOR for the MDTF-MTR), these issues were addressed through more extensive stakeholder interviews and with the Partnerships and with M&E program managers. Projects performance reports were received from program managers who were asked to complete a summary table designed by reviewers to show project title; start and end dates, source and amount of funding; countries and institutions; expected outputs; actual results; current status and remarks. These questions were designed to assess regionality, resources allocation, delivery of results, and status of the projects. We also increased to 15 the number of NARI leaders of relevant crop programs interviewed across the four visited countries, and interviewed six ASARECA project leaders in four NARIs and two universities. Information was also obtained from interviews and reports received from the former ASARECA network coordinators of BARNESA and PRAPACE, and regional leaders of the international centers Bioversity and CIP. 4. Biotechnology There are three questions. First, “how effectively has ASARECA’s biotechnology program contributed to helping national research institutions in the region to participate in cutting-edge science?” Second, “to what extent has the program opened up pathways for the utilization of biotechnology?” The two categories here are utilization for research and utilization in commercial/public space. Using the two categories defined for the first question, we planned to determine utilization for research via review of ASARECA reports and stakeholder interviews. Utilization in commercial/public space of transgenic organisms depends upon the legal and operational development of a biosafety framework and a biosafety law and associated regulations. Uganda and Kenya are the two countries with biosafety systems functioning in pre-commercial space. Third, “what value has it added to other programs working in this area at the regional and national levels?” We reviewed a presentation and reports from the Biotechnology Program, and interviewed program staff . Biotechnology facilities were inspected in Kenya, Ethiopia and Uganda, including confined field tests in Uganda, and their managers and institutional directors were interviewed also in Rwanda. We interviewed a senior university scientist and ASARECA project leader in this field in Kenya, and the director for Africa of the International Service for the Acquisition of Agri-Biotech Applications (ISAAA). A scheduled meeting with a private sector entrepreneur in this field unfortunately failed to take place. One Evaluation team member (Zeweldu) brought detailed and up-to-date knowledge of this sector across the region as a result of his association with a U.S. university project. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 7 5. Policy We proposed to use the CGIAR approach for Policy Oriented Research POR) Impact Assessment to address the question: “What is the comparative advantage of the ASARECA Policy Program compared to other organizations working on regional policy reform with COMESA and EAC?” This approach uses the causal model of inputs leading to immediate policy research outputs that are disseminated (uptake) via direct and indirect pathways to influence policy decisions whose implementation generates impact. However, the comparative advantage assessment depends upon the existence of the policy impact studies that have completed the cost-benefit analysis of the policy change. We proposed to only compare other research organizations working on the same policy area to ensure comparison of like with like. As we could find only a policy impact study on seed law harmonization that would form the core of the analysis, we shifted to a qualitative assessment of the policy programs based upon study of secondary sources and interviews. The Evaluation team considered the program’s initial presentation, reports, publications, and responses to requests for results and outcomes information around the above questions; and interviewed the Program Manager. At ASARECA the head of the gender mainstreaming initiative was interviewed, as well as one NARI focal point in gender analysis in research, as this topic falls under the program’s remit. We interviewed seed trade association and company representatives in Ethiopia and Uganda, but were unsuccessful in arranging a meeting with the Ugandan agency piloting the work on product standards. We interviewed directors of the African Center of ISAAA to assess biosafety work, and of the International Livestock Research Institute on divisions of effort among institutions. We gave less emphasis to assessing dryland resource management, as a compromise in the face of non-availability of some information expected in advance from the MTR. 6. Spillovers International research support focuses on selecting research problems with what should be explicit consideration of the potential spillover benefits in similar agroecological and socioeconomic environments. The team proposed to focus on the three main types of spillover effects that are generally evaluated in research programs: across location, across commodity, and price. The ASARECA M&E system did not contain enough information to categorize spillover effects in the expected three categories. As spillover impact studies were not available, spillover value was primarily based on expert opinion and, in some cases, end user perception. In addition to assessing presentations by and interviews with managers of USKM, the Information and Communication Unit and ASARECA programs that support technology and policy generation, NARS leaders of three ASARECA scaling-up projects were interviewed in two countries. We met the regional farmers’ association representing groups across the region; and in Ethiopia 32 farmer-collaborators (13 of them women) from 2 communities involved in ASARECA projects were questioned informally about their knowledge, acceptance of and perceptions of limitations in the technologies. Interviews with senior staff of two CGIAR Centers and former ASARECA networks coordinators, and institutional websites, were used to identify information available on adoption and impact. 7. Capacity Building Of the two questions “What has been the balance of benefits to the larger, stronger NARIs and the smaller ones?” and, “How effective has ASARECA been at building capacity of the NARIs and other national institutions?” the latter is much more complex because of the attribution issues, the volatility of support to NARIs over the period to be evaluated, and the need to track the evolution of NARI capacity using acceptable benchmarks. We used published Agricultural Science and Technology Indicators (ASTI) for 8 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 9 the countries visited to explore the effectiveness and the attribution issues through key participant interviews. Ethiopia was substituted for Tanzania in the sample of countries. ASTI indicators were used to establish the research FTEs, but we could not find data that would permit attribution of research expenditure to ASARECA intervention other than on human resource development. ASARECA’s Partnerships and Capacity Development Unit was a key respondent and provider of data on the key questions on this issue. We also interviewed directors general of three NARIs and senior leaders in three universities, and assessed the capacity of individual national programs and scientists through their research, presentations, reports and quality of interactions with the team. 8. The CAADP Agenda First, “how effectively has ASARECA provided technical support to the CAADP Process?” Second, “how clearly are the outputs and expected outcomes of ASARECA-supported activities aligned with the objectives and implementation plans for CAADP?” Answering this broader effectiveness question requires a timeline of inputs and outputs provided by ASARECA programs to plot: a) the development of Pillar 4 elements as the designated SRO under the FARA working within the context of the FAAP, and b) the CAADP Compacts, the Country Investment Plans, and the implementation of the Investment Plans as part of the Regional Economic Community. The CAADP Pillar 4 strategy document provided the framework for the team’s assessment. The head of the PCD Unit as ASARECA’s focal point for CAADP was a key interview for this issue. ASARECA provided their qualitative information on the types of services that they delivered or are prepared to deliver at each stage of the Compact implementation. The team also questioned the ASARECA and national focal points on their knowledge of and interaction with CAADP activities in each of the countries visited, and interviewed representatives of five donor missions met across the countries visited. 9. Future Programs and Feed the Future “Does ASARECA have the capacity and the flexibility to organize responses to specific demands for technologies and knowledge to overcome constraints in those (FTF) value chains, improving their efficiency and competitiveness?” We used the publicly available FTF country implementation plans in the four countries visited, and the regional FTF implementation plan provided by USAID/EA, to identify the staple value chains that are specifically named. The team then compared and contrasted these to the pipeline of ASARECA programs to look at their alignment. We examined whether and how the research pipeline commitments and the programming approval process constrains or permits change, for example through halting one commodity program to replace it with another, or abandoning crop breeding to focus on crop management research. . THE FINDINGS: RESPONSES TO QUESTIONS ASKED BY USAID IN NINE KEY ISSUE AREAS 1. REORGANIZATIONAL QUESTIONS Four questions were posed for this key issue: • What evidence is there of value added as the institution has moved from a loose association of commodity networks managed separately by the CGIAR centers to consolidated regional mechanisms? • Would the reorganization have been successful with or without the support from USAID/East Africa? • Has ASARECA’s performance been disappointing in any areas over the years? • With regard to governance reforms and ASARECA’s performance—the question is posed: Have these changes helped to broaden ASARECA’s effective partnerships beyond the 10 NARIs to the broader agenda of CAADP Pillar IV? The Evaluation team interviewed the Deputy Executive Director of ASARECA and his Program Managers, three Board members representing NARIs and the CGIAR Centers, and four other senior National Agricultural Research System (NARS) leaders and also other scientists and stakeholders across four countries. The following are summary responses to the above questions following a largely qualitative assessment of the degree of improvement and the magnitude of the increase in performance in six key/strategic areas of ASARECA’s mandate activities, namely institutional capacity building; technology and innovations; biotechnology; policy; regional spillovers; and capacity building of national institutions. ASARECA’s reorganization was designed to establish the institutional arrangements required to deliver the organizations results and create impact. As the institution moved from a loose association of commodity networks project, commissioning guidelines were developed and aligned to the innovations systems premised on the concept of research for development (R4D). The USAID/East Africa investment in the consolidation of the pre-reorganization networks into ASARECA’s current portfolio of programs to a regional mechanism has yielded significant and positive results. Primary evidence of this is the assembly of a well-qualified and experienced group of senior scientists as program managers who have improved the coherence in implementation of ASARECA’s portfolio. Towards the end of the first grant period, USAID took the lead in bringing in the International Food Policy Research Institute (IFPRI) for analytical work preparatory to a systematic priority-setting process that led to the development of the new Strategic Plan. The IFPRI analysis identified development domains, based on agricultural production potential, population density, and market access, that cross national boundaries and that should guide decision-making aimed at adding value to national research and up scaling (Figure 1). IFPRI showed that the largest impact for the greatest number of beneficiaries would be attained through a primary focus on major food crops that are also traded regionally. Research gaps EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 11 were also identified in horticulture, oilseeds, and other commodities with high potential returns to agricultural research, as well as in the area of regional public goods such as regional learning and sharing of extension methods; regional learning on institutional partnerships for integrated agricultural research for development; and policy research. ASARECA prioritizes its opportunities based on identification of relevance of a problem to more than two countries in the region, with likelihood of demand by and/or significant benefits for small-scale farmers and other groups of stakeholders. FIGURE 1: DEVELOPMENT DOMAINS IN EASTERN AND CENTRAL AFRICA Source: Omamo et al., (2006), IFPRI and ASARECA The new operational plan (OP) focused on four pillars of changes designed to enhance delivery on its mandate as follows: establishing sound governance; establishing sound management; addressing challenges and opportunities of subregional significance; and measuring performance and ensuring accountability. 12 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA The aim is to bring together an appropriate mix of partners and stakeholders (from both public and private sectors, users and development agencies) in projects that apply agricultural innovation system approaches and Integrated Platforms for Technology Adoption (IPTA) principles. Dissemination of innovations falls under the responsibility of the Up-Scaling and Knowledge Management Program (USKM) as well as the Information and Communication Unit (ICU), with coordination of innovation generation being organized by commodity, thematic and policy programs. Governance of the networks was principally through Regional Steering Committees, which brought in member country representatives to oversee progress and influence the content and direction of the annual work programs. This funding component provided the primary source of support for managing the coordination units of the networks and programs responsible for general administration, financial management, capacity development, communications, steering committee facilitation, and research direction. This financing also contributed to technical backstopping for these networks by the CGIAR centers, and facilitated access for most networks, programs, and projects to a wide range of services (including training, genetic materials, technical advice, use of databases and publication facilities, good connectivity and communications). USAID financing also supported a Competitive Grants System that became a primary instrument of ASARECA for producing regional public goods, ensuring that the research addressed regional priorities, and improved the rigor and quality of research proposals. Information services were strengthened, as evidenced by the growing volume of reports available through the ASARECA website—for example, the African Highlands Initiative (AHI) documented its research findings through 23 working papers, five journal papers, and two methods guides. The reorganization has also broadened ownership of the organization and diverse partnerships beyond its traditional partners in research institutions. These include those partners required by ASARECA in response to one of the major principles embedded in the Framework of African Agricultural Productivity (FAAP), namely plurality in the delivery of agricultural research, extension, and training services. They are the broad range of service providers with diverse skills and strength, namely universities, nongovernmental organizations (NGOs), and the public and private sectors who contribute to publicly supported agriculture productivity programs. Their contributions include best practices in scaling up agricultural innovations through agricultural extension and advisory services, farmer empowerment and development of technology uptake pathways. Broadened ownership has also expanded ASARECA’s reach to fulfill its role as one of the CAADP Pillar IV partner institutions. This includes sensitizing NARIs to the CAADP process, reviewing national and regional investment plans, and support to research based on ASARECA member country priorities. It is important to note here that USAID’s support over this total 10-year period provided the foundation for the strategic initiatives required for ASARECA’s transition to new research programs and their aims, as well as the key changes in the research, management and governance structures and systems for delivery of its mandate. USAID contributed 25 percent of ASARECA’s total budget over the 10-year period and was the second largest donor. The reorganization resulted in additional donors coming in to support the organization to the extent that USAID’s proportion of total support gradually reduced from about 33 percent (it was the largest single donor in the critical period up to 2007) to 18 percent more recently. Significantly, this investment has also resulted in leveraging additional financial resources to further support some of the activities primarily managed with USAID funds. ASARECA has found complementary funding from other sources, for example, from the Bill &Melinda Gates Foundation and the Donald Danforth Centre in St. Louis, Missouri, USA, for advanced biotechnology studies on Cassava EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 13 Brown Streak and Cassava Mosaic diseases at the National Crops Resources Research Institute (NaCRRI) in Uganda. In terms of the drivers of ASARECA’s programs post-reorganization, the operational plan proposes that the dominant drivers for the deliverables would be: generation and uptake of demand-driven technologies and innovations, policy options for enhancing the performance of the agricultural sector in the sub-region; and capacity for implementing agricultural research in an integrated research-for-development approach. It had been determined that with the reorganization, ASARECA would implement fewer, larger and more strategic research projects with a wider scope and scale of activities traversing several result areas. It is to be noted that the staple crops program was created out of six former research networks, and that the drivers of the natural resources management and forestry program which cut across sectoral boundaries were agriculture, land, water and biodiversity as well as land tenure and other policy and social issues. Evidence of a stronger sense of ownership of the consolidated programs by the new partners was their consistent responses in interviews in overall favor of ASARECA’s contributions to the region even when they offered critical comments too. Representation on the Board was of great importance to the new partners, as also was emphasis in some projects on national priorities as opposed to the broader scope of past network activities. However, the team’s assessment is that several challenges remain, including the basis for distributing resources across member countries especially in research programs where small country NARIs may miss opportunities to participate because of a comparative lack of research capacity in that particular research area. In striving to maintain a good standard of regional research, this has led to some imbalance in resource allocation with current concentration of projects in Kenya, Uganda, and Tanzania (76 percent of the total project funds allocated). This does not adequately reflect an awareness of the need for special consideration required at the same to respond to and benefit from the richness of the region’s diversity. Switching from competitive calls for network projects to commissioned proposals for consolidated programs post-reorganization, has also contributed to the imbalance in the allocation of resources, in the team’s view. NARS managers and scientists interviewed in countries receiving relatively small allocations of resources feel that one reason for this situation is that program managers tend to have less knowledge of scientists or areas of research advantage in their countries. Responses also showed that the generally stronger scientists in some of the larger Anglophone NARIs know each other better and, hence, find it easier to put together convincing proposals. Stakeholders of those networks (AHI, Banana Research Network for Eastern and Central Africa [BARNESA], ECABREN) that have continued to function outside the new ASARECA consolidated mechanism reported that these continue to serve a useful purpose, and indeed their continued existence is the result of grassroots/steering committee demand (as well as international centers’ preparedness to continue with some of the coordination responsibilities). These respondents regretted the disappearance of other networks because of the loss of networking benefits. Primary among these benefits are shared ideas in addressing some common goals, and the pooling of intellectual resources where the whole was greater than the sum of its parts. Broader geographic networking in terms of the breadth and scope of NARIs participation catalyzed by ASARECA beyond the purview of current specific projects would strengthen the impact potential of current and future regional research and improve continuity in partnerships forged across NARS as short￾term projects phase out. This should be achieved without great increases in program costs, through 14 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA workshops, IT approaches, and, in the case of continuing networks, linkage of events with past and present collaborators in related ASARECA projects. Significant progress has been achieved in rationalizing and harmonizing policies across the region pre￾and post-reorganization. However, the absence of an intellectual property rights policy for regional public goods remains a major impediment to access by all member countries and their small-scale producers and processors to technologies developed through ASARECA. The pivotal role of ASARECA in the implementation of the forthcoming East African Agriculture Productivity Program funded by the World Bank makes action by the Board on this matter imperative and urgent. Legal recognition of ASARECA in each member country would considerably facilitate the organization’s capacity to assist member NARIs with duty-free importation and procurement of key equipment and supplies required for ASARECA-supported projects. With regard to ASARECA’s broader partnership beyond the NARIs in its member countries, the value of the IPTA approach in its research-for-development activities [see later section on Technology and Innovation] is considerably enhanced by the diverse group of relevant stakeholders resulting from its broader partnership after 2008 (Table 3). This has improved its capacity to develop institutional mechanisms that address value chain constraints to better upscale technologies. TABLE 3: TOTAL NUMBER OF PROJECT PARTNERS IN EACH OF SIX PROGRAMS Type of Project Partner 2008 2009 2010 Public Entities Staple Crops 27 27 27 High Value Crops 0 9 9 Biotechnology 24 21 2 Natural Resources Management (NRM) 0 12 12 Policy 28 34 34 Upscaling & Knowledge Management 10 1 3 Private Enterprise Staple Crops 10 10 10 High Value Crops 0 1 1 Biotechnology 3 1 1 NRM 0 7 7 Policy 11 0 0 Upscaling & Knowledge Management 1 0 2 Universities Staple Crops 5 5 5 High Value Crops 0 3 3 Biotechnology 9 8 1 NRM 0 6 0 Policy 4 2 2 Upscaling & Knowledge Management 0 0 0 NGOs Staple Crops 27 33 33 High Value Crops 0 0 0 Biotechnology 3 0 0 NRM 0 7 7 Policy 5 11 11 Upscaling & Knowledge Management 2 1 1 Sources: Compiled from records of respective programs; and Partnerships and Capacity Development Unit EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 15 2. INSTITUTIONAL CAPACITY QUESTIONS This key issue deals primarily with a cost and benefit approach focused on ASARECA’s performance as an organizer of national, institutional and partner (international institutions, extra-regional universities, private companies, and NGOs) capacities to support agricultural technology development, compared to other available options for supporting agricultural technology development. The following question was posed: Has coordinated support from the donors to ASARECA’s single Operational Plan (OP) proved to be an effective strategy for building institutional capacity? It should be noted that this key issue and the response to the key question is predicated on the presumed existence of comparable data for performance measures pre-OP and post-OP performance of ASARECA. Such data was unfortunately unavailable. It should be noted also that, while the team compared overhead rates with institutions in the region, little relevant data that would facilitate comparison of ASARECA’s performance to that of other institutions operating regionally (counterfactuals) was available. The following are summary responses based largely on qualitative analysis of this issue and responses to structured interview questions with ASARECA Management, three Board members and Program Managers, and stakeholders shown in the Annex. USAID’s sustained support to the ASARECA Secretariat from its formation in 2004 and throughout the current evaluation period has been critical in facilitating and guiding its evolution. According to ASARECA’s financial records, in its formative years—the initial three Operational Plans between 1994 and 2001—USAID provided 61 percent of all donor funds. A significant development in the current OP evolution was an increase in funding levels and a move by donors in general toward core-funding, harmonized management, and financial reporting systems. Over the 10-year period under review here, overall contributions from USAID still amounted to 24.7 percent of the total, of which 31 percent supported core secretariat functions including governance, management, planning and information management (Tables 1 and 2). It was clear to us from interviews with Secretariat staff and from reports that the USAID contribution went well beyond the strictly financial, for example, in influencing M&E capacity development and investing in financial management capacity that made the Finance Unit capable of handling the current multi-donor funds. It was expected that these would enhance ASARECA’s capacity for delivery through enhanced and sound governance and management, and its capacity to address challenges and opportunities of sub-regional significance. It was also expected that the organization’s capacity to measure performance and ensure accountability would be increased. Coordination of donor support to ASARECA’S OP has improved collective priority setting, work-plan development, and budgeting for more effective program implementation. During the course of the consolidation there is also evidence, for example over an 8-year period in the Biotechnology Program (Table 4), of performance improvement as measured by the ratios of overhead cost ascribed by ASARECA to an individual program as against funds mobilized through ASARECA to support the program’s research projects. Table 4 also illustrates a particularly long phase in the case of the Biotechnology Program in which overhead costs were excessively high relative to program activities (the period 2003 to 2009). This was due initially to a lag phase occupied by planning workshops until the first calls for proposals were advertised and until the reorganization led to a broadening of the program’s agenda to include agricultural bio-diversity—a move that brought in other donors besides USAID. According to Financial Unit data, the proportion of costs ascribed by ASARECA to all research programs was only 10.5 percent in 2010. However, if a proportional share of the entire governance and administrative operating cost of ASARECA were to be charged against research projects, then overall 16 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA ASARECA efficiency at 37 percent would look much less favorable. By contrast, the overhead rate generally charged by the West and Central African Council for Agricultural Research and Development (CORAF/WECARD), another sub-regional organization, is 15 percent, although whether overhead rates are calculated by the two sub-regional organizations on the same basis is unclear to us. Typical overhead rates now charged by Internal Agricultural Research Centers (IARCs) in the region are higher—currently around 22 percent to 25 percent. Those charged by US universities and the CRSP projects typically range between 20 percent and 55 percent. It must be noted, however, that the operational support for network activities were of a different order from those for the higher priority target projects. The two approaches are complementary and not competitive in any way—this was also the predominant reaction of the national scientists interviewed. The related questions around comparative technical advantage and responsibilities of ASARECA and other providers of technology and innovation are addressed under the next Key Issue (No 3). TABLE 4: RATIOS OF OVERHEAD COST VS PROJECT ACTIVITIES FOR THE BIOTECHNOLOGY PROGRAM Actual Expenditures for the period 2003-2010 Year Admin Costs US$ Project Costs US$ Total Actual US$ Percentage (%) 2003 99,814.95 65,798.17 165,613.12 152% 2004 110,320.16 66,981.69 177,301.85 165% 2005 120,825.57 133,591.73 254,417.30 90% 2006 135,780.29 309,620.82 445,401.11 44% 2007 117,748.49 209,664.37 327,412.86 56% 2008 152,429.55 271,799.36 424,228.91 56% 2009 163,038.72 456,536.45 619,575.17 36% 2010 192,656.66 1,679,344.28 1,872,000.94 11% Total 1,092,614.39 3,193,336.87 4,285,951.26 34% Source: ASARECA Finance Unit The single OP has enhanced the organization’s capacity for strategic and participatory planning among stakeholders. It has also enhanced programmatic consolidation, and significantly improved its potential for delivering tangible outputs along value chains with clear performance targets, measurable indicators, and timeframes. With the development of the ASARECA Operational Plan, the M&E unit has successfully revised ASARECA’s log-frame, and sharpened its indicators, reducing them from 54 to 24. The unit has also successfully developed data collection tools jointly with project implementers. These indicators now have detailed descriptions on definitions, collection and analysis methods (see Table 5). ASARECA’s donors require the organization to report against project log-frames for more consistent self-assessment of the key components of the organization’s activities. ASARECA's programs currently have in-built Performance Monitoring Plans, but weaknesses still exist in operationalizing these in sub-projects. In some countries the M&E workshops designed to more fully engage sub-project Principle Investigators. NGOs and other participants have only recently been conducted. Attempting to marry ASARECA reporting requirements to NARIs’ internal M&E requirements could improve response rates and quality, but doing so would require much more time and effort than is possible in a workshop of a few days with multiple partners. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 17 A key principle underpinning the OP was to establish mechanisms to learn from experience, to identify processes and mechanisms, and to have regular assessments and continuous adaptation to changing environments. In this context, it is confirmed that key performance frameworks are now in place, and plans to roll them out to all the projects are in an advanced stage. There are continuing weaknesses, however, at project partner level with regard to understanding and applying performance monitoring plans. Indeed, due to staff turnover and delay in recruiting a new M&E unit head (the position was vacant for five months in 2009/2010), the unit has struggled to establish systems and procedures that would facilitate effective learning from experiences. Program Managers expressed concerns about staffing support inadequacies (numbers and competencies) to manage grantees’ financial management procedures effectively. Significantly, grantees in some member countries also shared their concerns about financial and administrative procedures. These relate largely to unrealistic turnaround time in financial and technical reporting after receipt of funds. TABLE 5: THE M&E PERFORMANCE MONITORING PLAN Year Output Target Intended User Outcome Output 1 •Capacity of ASARECA and partners to monitor and evaluate their interventions enhanced 2008 • At least one training workshop for ASARECA program managers and project leader • At least one training workshop for NARS Partners • Program managers and project leaders • NARS partners • Program managers and NARS partners better able to design and manage a performance based M&E system. 2009 • • At least two training workshops for NARS partners • Capacity strengthening plan for NARS • ASARECA programs and units • NARS Partners • Program managers and NARS partners better able to design and manage a performance based M&E system. 2010 • At least one evaluation workshop for ASARECA and NARS partners • Capacity strengthening initiative evaluated • ASARECA programs and units • NARS Partners • Donors • Program managers and • NARS partners better able to design and manage a performance based M&E system. 2011 To be determined following the .Mid-term Review 2012 To be determined following the Mid-term Review Output 2 Appropriate M&E systems and structures established 2008 • Performance monitoring plan compacted Performance monitoring plan automated • Program managers and project leaders • NARS partners • AIL program managers using the automated PMP 2009 • Automated M&E system installed in all program management units * • Automated M&E system installed in all project implementation units • Program managers and project leaders • NARS partners • All ASARECA programs and projects tracking and reporting on progress and achievements according to the performance monitoring plan Source: ASARECA M&E Unit ASARECA’s capacity has been strengthened through the single OP to collaborate with its stakeholders in the implementation of research (Table 6) at the national and regional levels. While mindful of the very 18 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA limited timeframe of two to three years from initiation of the OP to the present, it is important to note from this table the generally improving trends in indicators of progress. TABLE 6: INDICATORS OF PROGRESS BY SELECTED PROGRAMS SINCE OP (TOTALS FOR STAPLE CROPS, BIOTECH, POLICY ANALYSIS, AND ADVOCACY PROGRAM [PAAP], NRM) Indicator Program 2008 2009 2010 No. of research projects Staple Crops 1 15 15 High-Value Crops 0 3 3 Biotechnology 10 10 11 NRM 0 5 2 Policy 1 8 9 Upscaling & Knowledge Management 0 3 4 No. of promising demand driven technologies and innovations Staple Crops 0 0 39 High-Value Crops 0 3 3 Biotechnology 0 7 8 NRM 0 1 3 Policy 0 4 6 Upscaling & Knowledge Management 0 6 3 No. of extension projects Staple Crops 0 0 0 High-Value Crops 0 0 0 Biotechnology 0 3 4 NRM 0 0 2 Policy 1 3 2 Upscaling & Knowledge Management 0 0 0 Source: Records from the respective ASARECA programs 3. TECHNOLOGY AND INNOVATION The main evaluation purpose is to test the hypothesis: “Regional USAID investments in science and technology through ASARECA have proved themselves to be an effective mechanism for making widely applicable technologies available in multiple countries.” Two fundamental questions are asked: 1. What evidence is available …to compare USAID support to ASARECA in terms of technological impact against other organizational alternatives … and what are the lessons learned?2. What are the most promising technologies and innovations coming out of programs that USAID has supported, in the context of ASARECA’s broader programs [following reorganization in 2006]? We obtained presentations and reports from the technology-generating programs of ASARECA and interviewed their staff, with a focus on Staple Crops as the largest program and having an agenda supported by USAID over the entire 10-year period. Since quantitative documentation from ASARECA’s project performance M&E was largely unavailable until 2010 (due to the M&E deficiencies noted above under Key Issue Area No 2 - Institutional Capacity), these issues were addressed through more extensive stakeholder interviews, the Partnerships, and M&E program managers. Further material for this key issue was developed through analysis of projects performance reports received from program managers who were asked to complete a summary table designed by reviewers. The table was designed to show project title, start and end dates, source and amount of funding, countries and institutions, expected outputs, actual results, status, and remarks. These questions were designed to assess regionalism, resources EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 19 allocation, delivery of results, and status of the projects. We also increased to 15 the number of NARI leaders of relevant crop programs interviewed across the four visited countries, and interviewed six ASARECA project leaders in four NARIs and two universities. Information was also obtained from interviews and reports received from the former ASARECA network coordinators of BARNESA and PRAPACE, and regional leaders of the international centers Biodiversity and CIP were interviewed. ASARECA is not the sole research and development (R&D) service provider in the region. Other national, regional, and international R&D organizations play complementary roles (Figure 2), with shared higher-level goals, objectives, and ultimate beneficiaries. For example: • The CGIAR centers have a strong presence in the region. Their focus is on strategic research that provides regional access to global sources of technology and innovation; by usually working in long￾term partnerships with NARS with other national and international organizations and by taking a relatively broad approach with local partners (e.g. in a value-chain context), the centers are able to provide closer technical support to NARS than is the case with ASARECA. We found several examples of recent impact that builds on the earlier partnerships with ASARECA networks (e.g. from the introduction in Kenya of positive seed selection to improve small-scale potato seed systems, as well as support to weaker NARS through access to new breeding materials). • The Collaborative Research Support Programs (CRSPs) are communities of U.S. Land Grant Universities working with NARS (especially universities) and a broad range of other institutions. As projects focus on strategic research and the exposure of developing country partners to advanced research facilities at US universities, this form of collaboration provides valuable opportunities for capacity building at MSc and especially PhD levels, often resulting in strong long-term institutional and scientist-level partnerships. No interviewed respondent reported working with a CRSP. • Non-Governmental Organizations (NGOs) as a category include a few (mostly international) that address applied research issues. More generally, NGOs tend to be adept at intensive action-oriented projects and typically work directly with and reach large numbers of farmers for a narrow range of objectives and thereby achieve rapid scaling out of technologies and products from research that may not be of commercial interest to the private sector. The best among them include a concern for sustainability, for example through building the capacity of farmer organizations. • Many other regional and international organizations are active in this region and occupy highly targeted niche roles related to agricultural R&D (e.g. the International Service for the Acquisition of Agri￾Biotechnology Applications (ISAAA). ASARECA as a regional R&D association without its own research facilities rather serves as facilitator and promoter of regional collaborative research and technology generation. With its consolidated structure and operational plan that covers R&D problems in an integrated manner, including strengthening of national capacities, ASARECA has a unique role in the region. Working in close collaboration with other R&D institutions including IARCs, universities and private R&D institutions that have their own research laboratories and scientific staff, and managing research projects that are led by local institutions and scientists, ASARECA is not competing with any of those categories above. Being also a regionally owned organization operating at the optimal level of subsidiarity, ASARECA is now well positioned to play a unique regional R&D coordination and leadership role that cannot be filled in an adequate manner by any other organization. Although we heard from respondents in the different categories above that understanding across the region of the complementary nature of their roles has improved markedly over the past 5 years, further 20 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA coordination among them in regional priority setting and action planning would be valuable (for example in the CAADP context, see below), and ASARECA should move more pro-actively into this leadership role. ASARECA has a complex and difficult regional role to play in addressing diverse expectations of partners including donors, member countries, and international policy and economic community stakeholders. To meet the high expectations, ASARECA needs a highly sophisticated and dynamic operational system, competent management staff at all levels and organizational discipline. From interviews with individual program managers and with them as a group, and from the projects visited, we concluded that programs mostly operate in too independent a manner, i.e. ASARECA is not yet fully capitalizing on having all programs centralized at the Secretariat. On the technology promotion and dissemination front, ASARECA has made significant contributions as shown by the database that USKM collated. This describes 37 “best-bet” proven technologies and innovations tried and tested in pilot sites and ready for up scaling in Eastern and Central Africa (ECA) countries to improve livelihoods. The presentation of the best bets is divided in eight clusters: crop varieties, crop management practices, post-harvest processing and utilization, technology uptake approaches, seed systems, natural resources management, access to credit and market, and policy. This effort is extremely commendable and captures technologies generated over the last 10 years of ASARECA’s networks and programs and partnerships, many with USAID support. Technologies generated through USAID funding that are utilized in several countries include improved potato varieties, high yielding varieties and also orange fleshed varieties of sweet potato, quality protein maize, high￾yielding bean varieties, drought stress tolerant maize, disease-resistant banana, and integrated disease management for cassava. Examples of innovations that befitted from USAID support, and which have already achieved considerable adoption and impact are new bean varieties that were benefiting 37 million people across seven ECA countries by 2005 (Kalyebara and Buruchara, 2008). Others, such as orange-fleshed sweet potato and quality protein maize, are still far below their potential adoption rates. Currently nine quality protein maize varieties and seven orange-fleshed sweet potato varieties, each with related utilization technologies, are at different stages of the uptake pathway in at least four countries (source: USKM Strategic Plan 2009-2014). Technologies and innovations vary greatly in their complexity and hence the ease or difficulty in achieving full adoption and impact potentials. Generally, dissemination and uptake of technologies and innovations is not as high as it should be. This is mainly due to lack of innovative approaches to promote technologies and innovations both at national and regional levels through involvement of different stakeholders in the value chain. Technologies and innovations that are not attractive as money-spinners to the private sector (open-pollinated quality-protein maize having grain appearance similar to nutritionally unimproved varieties, and dryland crop management practices) generally require more effort and take longer to reach large numbers of users. Bean varieties may have been an exception because of clearly distinguishable grain colors and other characteristics, short maturity cycle, initial large-scale distribution in small low-cost but economically produced seed packets, and increasing penetration of regional trade that started to attract seed companies. Others (snap beans from ECABREN, potato and sweet potato processing from PRAPACE) need stronger involvement of the private sector, should be carefully incubated and translated into business so that they can be sustainable. Wider application of business incubation methods could have accelerated technology uptake and commercialization, and created thousands of new job opportunities for the ever-growing number of young university graduates in the EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 21 region. ASARECA and its partners should target not only resource poor farmers but also young educated entrepreneurs to be able to transform agriculture from traditional farming practice into knowledge-based commercial farming. A good example that should be applied more widely is USKM’s involvement in the implementation of the FARA project Dissemination of New Agricultural Technologies in Africa (DONATA), which is applying the IPTA approach. IPTA is a multi-stakeholder platform to access information and knowledge to adopt proven technologies to local settings. It is composed of farmers, agriculture advisory service providers, agribusiness actors, and researchers. This technology dissemination system needs to replace the traditional linear research-extension mode. USKM is now applying this approach in projects on quality protein maize and yellow-fleshed sweet potato, and is involved in dissemination of New Rice for Africa rice technologies in post conflict areas in northern Uganda and Southern Sudan. Application of the IPTA approach promises to increase the potential for adoption significantly. Other major benefits of IPTA application within ASARECA are the enhancement of the skills—among both ASARECA program staff and their NARS partners—for identifying the issues in moving from rhetoric to practice in achieving broad uptake, and feeding back findings into the design of technologies and innovations. ASARECA’s Information and Communication Unit is innovative in disseminating agricultural knowledge and information through online learning and networking opportunities, particularly for the scientific community and those who have access to internet. As a development from the former Regional Agricultural Information and Learning System network, this approach allows a broad range of stakeholders (NARIs, universities, NGOs, the private sector, IARCs, and others) to participate in e-based knowledge sharing and access to agricultural technologies and innovations. Further strengthening of ASARECA’s ICU and USKM could be a particularly powerful strategy to provide goods and services that ASARECA is expected to deliver. These two critical units also need closer integration without adding to the staffing level. FIGURE 2: CENTRAL ROLE OF ASARECA FOR DISSEMINATION OF TECHNOLOGIES AND INNOVATIONS 22 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA NARS members ASARECA’s virtual and physical technology and knowledge sharing platform Regional R&D Organizations Private Sector Farmers NGOs IARC ASARECA defines technologies as “all agricultural technologies and innovations which encompass the production to consumption continuum. Thus technologies include varieties/breeds, post-harvest products/ practices, methodologies, technologies for natural resources management and biodiversity conservation”. Furthermore, ASARECA defines innovation as “successful exploitation of new ideas by scientists” (ASARECA Annual Performance Report 2010). Also according to this Report, out of a planned total of 538 technologies and innovations to be generated, 668 were achieved—apparently showing a 124 percent performance rate (Table 7). This is extraordinary R&D performance by ASARECA. However, the reviewers could not independently verify whether these technologies and innovations have passed any standard or quality test. Technologies and innovations have standards and quality check mechanisms in different countries. For example, crop varieties are released through national variety release committee procedures, post-harvest technologies should follow the national standards and certification process, and innovations that are improvements on existing ideas, methods, or technologies should be scrutinized and registered by appropriate bodies before they are certified and released for public or commercial use. Accepted procedures for scrutiny and registration do not exist for some types of technology or innovations, although a minimum should be documented acceptability to users in pilot areas. We found no evidence in available reports that the 668 technologies and innovations generated by ASARECA projects had gone through such a process at either national or regional level. TABLE 7: SUMMARY OF TECHNOLOGIES AND INNOVATIONS GENERATED AND MADE AVAILABLE FOR UPTAKE Technology category Technologies and innovations generated Technologies and innovations made available Crop/livestock genetic 286 19 Crop/livestock germplasm characterization and conservation 18 0 Seed multiplication 7 14 Disease and pest management 236 29 Crop/livestock production 8 8 Post-harvest and value addition 107 14 Forage management 0 6 NRM 6 0 Total 668 90 Source: Annual Performance Report 2010, ASARECA 4. BIOTECHNOLOGY We asked the following questions. How effectively has ASARECA’s biotechnology program contributed to helping national research institutions in the region to participate in cutting edge science; to what extent has the program opened up pathways for the utilization of biotechnology? And what value has it added to other programs working in this area at the regional and national levels?” We considered a presentation by the Biotechnology Program, interviewed program staff, and reviewed their reports. Biotechnology facilities were inspected in Kenya, Ethiopia, and Uganda, including confined field tests in Uganda, and their managers and institutional directors were interviewed in Rwanda as well. We interviewed a senior university scientist and ASARECA project leader in this field in Kenya, and the EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 23 director for Africa of the International Service for the Acquisition of Agri-Biotech Applications (ISAAA). A scheduled meeting with a private sector entrepreneur in this field unfortunately failed to take place. One Evaluation team member brought detailed and up-to-date knowledge of this sector across the region because of his association with a U.S. university project. The Biotechnology Program was formally constituted and managed within ASARECA in 2003. Before that, it was an initiative of USAID with a focus on genetically modified organism (GMO)-oriented capacity building and research. In the year 2004, USAID allocated 600,000 USD for a competitive grant for biotechnology projects. The Agricultural Biotechnology Support Project II was tasked to lead the competitive grant scheme in collaboration with ASARECA and provide technical backstopping in selecting projects and grant allocation. The best four biotechnology projects were selected and received US$150,000 each, and were implemented between 2005 and 2007 (Table 8). Prior to and during ASARECA’s reorganization, there were excellent examples of cutting-edge science, supported by USAID through ASARECA (e.g. gene discovery, access, and transformation for drought tolerance in maize). These projects, especially when implemented by local universities, also led to strengthening of national programs capacity in six countries, initially at master’s level. Since 2008, this initial capacity development has been being built on by further Biotechnology Program projects using resources from USAID through training at PhD level. An excellent example is the partnership initiated between Uganda’s national cassava program at NaCRRI Namulonge and the advanced biotechnology lab of the Danforth Center in the USA; this has been especially effective in terms of leveraging by initial capacity development within the NARS the access to advanced labs, their procedures, biosafety laws, and scientists’ skills. TABLE 8: BIOTECHNOLOGY PROJECTS IMPLEMENTED FROM 2005 TO 2007 Project Title Participating countries Biotechnologies involved Remark Bringing the benefits of genomics and systems biology to maize fields through gene discovery and access for genetic transformation addressing drought stress tolerance in the ECA. Kenya Ethiopia Tanzania Sudan Gene cloning, Transformation, Embryo culture, PCR technologies Cutting edge science Marker-assisted breeding of the Stay-Green trait of Sorghum to enhance terminal drought tolerance in Eastern Africa. Kenya Sudan Uganda Marker assisted selection, Quantitative trait locus Cutting edge molecular technique, for non￾transgenic plant breeding Application of marker assisted selection (MAS) for the improvement of bean common mosaic necrotic virus resistance in common bean Kenya Uganda Rwanda Marker assisted selection, Quantitative trait locus Cutting edge molecular technique, for non￾transgenic plant breeding Exploiting genetic differences of indigenous East African cattle breeds to enhance vaccination responses Kenya Uganda Tanzania Vaccines, genomic and bio-informatics Cutting edge science applied Source: Records of the Biodiversity and Biotechnology Program However, the drought tolerant maize variety development project (Table 8, first project) has been the only project to be supported in the post op era (Table 9, project 4). This is mainly a capacity building project designed to tackle a practical problem namely drought. Post-operational plan projects of the biotechnology program (Table 9), with the exception of genetic engineering of maize for drought tolerance as the continuation from the past project, are non-competitive awards. The other projects do not have much science content, but rather focus on physical capacity building and training; they would have 24 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA been more appropriately initiated by the staple crops program. The biotechnology program adds little value to other programs or, as evidenced below, to national programs. Some of the tissue culture and up scaling projects could have been initiated and implemented by national systems alone, although this is not to suggest that encouraging the development of the full potential of tissue culture for up scaling is not an important effort—for example, the project developing use of low-cost decentralized tissue culture is innovative in the African context. We also note that most biotechnology program resources were put into either cassava or sweet potato or both, crops in which the staple crops program is also making investment. TABLE 9: POST-OP PROJECTS OF BIOTECHNOLOGY PROGRAM No Project Title Duration Amount of funding (US$) 1 Establishment of a Genetic Transformation Platform for Cassava in the ECA 2008-2010 310,000 2 Applying Tissue Culture to Improve Access to Cassava and Sweet Potato Clean Planting materials for Farmers in the ECA. 2008-2011 309,000 3 Conservation for Sustainable Availability of Sweet Potato and Cassava Germplasm through the Application of Biotechnology in the ECA. 2009-2011 697,000 4 Genetic Engineering of Maize for Drought Tolerance in the ECA 2008-2011 358,000 5 Development of low cost tissue culture incubators 2011-2013 149,000 6 Evaluation and dissemination of Striga Resistant Sorghum 2011-2011 114,000 FIGURE 3: APPLICATION OF MODERN BIOTECHNOLOGY AND GM CROP DEVELOPMENT CYCLE 2. Transformation Regeneration and Evaluation 3. Contained Greenhouse Test (CGT) 4. Confined Field Test (CFT) 5. Open Field Test (OFT) 6. Commercial Release 1. Gene Discovery and Evaluation The GM crop variety development cycle has at least six stages and involves several painstaking rigorous evaluation of each step before moving to the next cycle (as illustrated in Figure 3). Unless cycle six is EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 25 reached, it would be inaccurate to talk about useful biotech products that can be transferred to resource poor farmers. From relevant gene discovery to commercial release of a transgenic product can take more than 15 years of research and development. Reaching the stage of product release therefore requires careful selection of projects at onset and allocation of sustained and sufficient research funds. ASARECA’s Biotechnology Program funds projects for only two to three years and moves to the next set of projects, as with most projects across ASARECA. Generating solid biotechnologies in two or three year project life is difficult if not impossible. The program therefore needs to be prudent about taking on bio-technology generation in such a short period. Referring back to the discussion of Table 7, in the biotechnology context a product can be the result of a combination of several technologies and innovations such as gene cloning methods, transformation protocols, gene constructs, molecule extraction and purification protocols, fermentation process and procedures. As in the case of Technology and Innovation, biotechnology components and innovations also need to be clearly defined, classified and standardized by subject matter experts. For example, by analogy with a computer product that may contain several hundred technologies and innovations, “golden rice” is a product which contains more than 70 patented technologies. Classification and standardization of technologies and innovations is needed within the biotechnology program in order to understand expected results from specific projects. We recommend that ASARECA’s M&E Unit takes the lead role in this process in consultation with Biotechnology (and other) Program Managers. In terms of opening up pathways for the utilization of biotechnology, countries in the ECA region are at different levels of biotechnology and biosafety development (Table 8). The two leading countries in GMO research are Uganda and Kenya. Sudan has started field-testing transgenic cotton. Rwanda is making impressive progress in terms of applied biotechnology to support its vegetatively propagated planting material (seed) delivery system, through focusing on tissue culture application for mass multiplication of disease-free coffee, banana, sweet potato, cassava, potato etc. Rwanda’s biotechnology program is well t organized and implemented as a publicly owned system, and this is being achieved purely through national efforts and without any support from ASARECA’s Biotechnology Program. ASARECA’s Biotechnology program is involved in no transgenic field-testing in the region directly, as evidenced by visits to national programs. Its role in biosafety support is also limited since Policy Analysis and Advocacy Program (PAAP) took over the leadership in this area (see Policy section below). Linkage between ASARECA’s and NARS’ biotechnology programs is uneven: of the four countries visited, the team found two examples of excellent programmatic linkage, and non-existent linkages in two other cases where desperate need exists. The program could have provided more support to national biotechnology programs in terms of biotechnology and biosafety policy, and in advocacy; For example, Ethiopia has a well-equipped but underutilized lab and no biotechnology and biosafety policy advocacy group, and the country’s own considerable investment would benefit greatly from carefully focused funding and especially technical assistance. Influencing national biosafety policy would be especially valuable in supporting the introduction of proven transgenic technologies such as biotech cotton (an excellent entry￾point since BT-cotton has potential to reduce insecticide use and environmental and health hazards with this crop), maize or soybean for field-testing and commercialization. This would inspire countries to formulate more technology-friendly biosafety laws that take into account tangible economic and environmental/health (e.g. the cotton example) benefits of the technologies on offer, in comparison with those now to be found in the member countries of ASARECA (Table 10). There are countries with magnificent physical facilities including state-of-the-art equipment, but with only skeleton qualified staff such that little practical use is currently being made of those facilities. We 26 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA found a range of possible reasons to explain inadequate linkage, including insufficient observance of ASARECA Operations Manual procedures in calls for proposals, and inadequate communication on the sides of both ASARECA and NARS. Some of these comments are also made elsewhere with reference to other programs, TABLE 10: ASARECA COUNTRIES AND THEIR BIOTECHNOLOGY AND BIOSAFETY POLICY STATUS, AS OF 2010 Country Biotech/Biosafety Policy Biosafety Regulatory Regime Remark Burundi Draft biotechnology policy Sectoral legislation with ref. to biotech draft, biosafety bill 2006 DRC Draft national bio-safety policy Draft biosafety bill Eritrea Draft national biosafety policy Draft biosafety bill and guidelines Ethiopia No standalone biotech policy Biosafety proclamation passed into law in 2009 Highly precautionary Does not even allow GMO research Kenya Standalone national biotechnology development policy Biosafety Act 2009, implementing regulations finalized The only country ready for GMO commercialization Madagascar National Biosafety Policy Draft Bio-safety Bill Rwanda Draft national Biosafety Policy Draft Bio-safety Bill 2005 Sudan National Biosafety Policy Draft Biological Safety Bill Tanzania Uganda National Biotechnology and Biosafety Policy 2008 Sectoral legislation with ref. to biotech and draft biosafety bill Source: From: Status of Biotechnology Policies and Bio-safety Legislation in COMESA Region. COMESA, 2010 5. POLICY We asked questions that concerned strengths and weaknesses in ASARECA’s policy programs on seed harmonization, biosafety, harmonized product grades and standards, and Natural Resource Management in the dry lands. Comparative advantage of the program compared to other organizations working on regional policy reform with COMESA and the East African Community (EAC)? Specific policy related outcomes with regional impacts. The Evaluation team considered the program’s initial presentation, reports, publications, and responses to requests for results and outcomes information around the above questions; and interviewed the Program Manager. At ASARECA the head of the gender mainstreaming initiative was interviewed, as well as one NARI focal point in gender analysis in research, as this topic falls under the program’s remit. We interviewed seed trade association and company representatives in Ethiopia and Uganda, but were unsuccessful in arranging a meeting with the Ugandan agency piloting the work on product standards. The director of the African Center of ISAAA was interviewed to assess biosafety work, and senior management of the International Livestock Research Institute on divisions of effort among institutions. We made the decision to give less emphasis to assessing dryland resource management, as a compromise in the face of non-availability of some information expected in advance from the Mid-Term Review (see Methodology annex). ASARECA’s PAAP has made considerable contributions, through support to research and the brokering of national policy changes in such areas as seed law and the quality of processed agricultural products. In EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 27 several studies, real impact has been made or is on the point of being achieved. Small-scale farmers are now better able to penetrate the milk market. Interviewed representatives of the rapidly growing commercial seed sectors in Ethiopia and Uganda expressed enthusiasm and optimism concerning the immediate prospects for more rapid and efficient access to varieties introduced from other countries. Regionality has been central to PAAP’s approach and success. Each of the constraints or opportunities, which are common to several or even all countries, has been addressed in one of two ways. In some cases, such as the harmonization of national seed laws, a collective approach was applied across countries, whereby potential mutual benefits have stimulated countries’ collaboration, sharing of lessons and progress, and eventual harmonization among them. In others, a sequential approach has been followed, as in the harmonization of root and tuber crops quality standards: Uganda was an initial pilot that was extended through subsequent advocacy and support. TABLE 11: POLICY-RELATED AREAS AND PROGRESS ACHIEVED Broad Policy Areas Geographic Scope Constraint/ Problem Identified Research (related) Output New Policy/Practice Adopted Impact Pathway Description Seed Harmonization Burundi, DR Congo, Ethiopia, Kenya, Madagascar, Rwanda, Sudan, Tanzania and Uganda Growth of seed sector constrained by different and often contradictory policies, regulations and procedures Harmonized seed certification, variety release, phytosanitary measures, plant variety protection, and import export Harmonized policies, laws and regulations and procedures Movement of germplasm across countries, private sector involvement in seed production and trade, more varieties available to farmers Informal dairy Burundi, Kenya, Rwanda, Tanzania and Uganda Limited recognition of informal milk sector Evidence supporting policy reforms and guidelines on safe handling of informal milk by players across the value chain Harmonized policies, laws and regulations and procedures Trade in milk products across countries, informal traders these milk products, more market outlets available to small farmers Biosafety Framework 19 COMESA countries Lack of regional mechanism to regulate GMOs Guidelines and policies regulating commercial planting, trade and use of GMOs in emergency aid Harmonized policies, laws and regulations and procedures Access to GMO seeds, lower production costs at farm level, higher returns at sector level and improved access to emergency aid Harmonized Product Grades and Standards 5 EAC states Lack of grades and standards for cassava and potato products Harmonized standards Harmonized policies, laws and regulations and procedures Commercialization of cassava and potato sectors and welfare benefits to producers and consumers along the value chains Natural Resource Management in the Dry Lands Ethiopia, Kenya and Tanzania Dry lands considered to be waste lands Recommendation domains and investment options that improve livelihoods while conserving Harmonized policies, laws and regulations and procedures Improved targeting of investments in dry lands, improved livelihoods and sustainable land management 28 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Broad Policy Areas Geographic Scope Constraint/ Problem Identified Research (related) Output New Policy/Practice Adopted Impact Pathway Description biodiversity in dry lands Source: ASARECA PAAP Table design based on CGIAR Science Council. 2006. Impact Assessment of Policy-Oriented Research in the CGIAR: A Scoping Study Report. CGIAR Science Council Secretariat: Rome, Italy. Careful identification or selection of appropriate partners and stakeholders has also been central. Hence it has been less a matter of the comparative advantage of ASARECA vis-a-vis other regional players, rather a question of ASARECA’s seeking common ground with, and harnessing, the collaborative advantage of appropriate partners based on their respective roles, interests, responsibilities and capacities (Table 12). TABLE 12: POLICY-ORIENTED INTERVENTIONS DEVELOPED BY ASARECA/PAAP WITH SELECTED PARTNERS AND STAKEHOLDERS ASARECA NARS Private sector IFPRI, other CGIAR RECS Other (specify) Seed harmonization Analysis, advocacy for implementation Advocacy for implementation Endorsement for implementation EAC, COMESA and Southern Africa Development Community Harmonized grades and standards Analysis Advocacy for implementation Endorsement for implementation EAC, Uganda Bureau of Standards NRM dry lands Analysis Analysis Endorsement for implementation International Livestock Research Institute (ILRI), International Union for the Conservation of Nature, Economic and Social Research Foundation (ESRF), Tanzania, and EAC Food prices crisis analysis Analysis Advocacy for implementation Analysis Climate change analysis Analysis Advocacy for implementation Analysis East Africa Farmers Federation Biosafety framework (RABESA) Analysis Analysis Endorsement for implementation COMESA Non-tariff barriers Analysis Advocacy for implementation Analysis Policy think tanks: Kenya Institute for Public Policy Research and Analysis; Economic Policy Research Centre, Uganda; and Tanzania (ESRF) Informal dairy projects Analysis Advocacy for implementation Analysis Endorsement for implementation East Africa Dairy Regulatory Authorities Council Source: Assembled from ASARECA PAAP EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 29 Catalyzing broader institutional adoption of research methods that take explicitly into account the existence of gender (and other socioeconomic) differences among intended small-farmers beneficiaries is a new area under PAAP responsibility—and one that cuts across other programs. Following up on earlier collaboration with the Participatory Research and Gender Analysis Program of the CGIAR that concluded with a gender audit in 2009, and with new funding from the Canadian International Development Agency, a Gender Mainstreaming Unit was incorporated in 2010. The gender audit had shown that awareness and action on gender analysis in research to be uneven in ASARECA and among main partners. A more encouraging recent trend was observed in the Cattle Vaccines Project, which envisages 60 percent of beneficiaries from the adoption of its technologies being women, with expected impacts being appropriately disaggregated according to typical gender interests within small-farm families (Kiambi, 2011). A Gender Mainstreaming Strategy for ASARECA is now in a final draft, and capacity building workshops in support of NARIs’ own gender focal points is currently in progress in response to stakeholders’ request in an initial regional workshop. Significant experience with gender issues exists in some NARS, as evidenced by their identification of gender focal points and the extensive training that has taken place in the region with inputs from many agencies. ASARECA is already playing an important role in value-addition by bringing together previously trained individuals across institutions to reinforce local action; this suggests great scope for harnessing regional resource persons in this area. Criteria for gender compliance are to be developed for ASARECA programs and projects in collaboration with the M&E Unit by 2014, although we believe this timetable is conservative in view of background work and the urgent need and opportunity for ASARECA to mainstream gender planning and analysis in its research projects. If this is achieved through considering gender-related differences among intended beneficiaries as one among several socioeconomic factors to be considered in project targeting and outcomes/impact assessment, then this initiative can be expected to raise ASARECA’s effectiveness in contributing to USAID’s concerns with both gender and poverty targeting. In conclusion, PAAP has had the advantages of uninterrupted leadership from within the ASARECA headquarters and uninterrupted financial support most notably from USAID. The program is well focused and organized as evidenced by its reports and publications, and its achievements in catalyzing significant changes across the region have been considerable. With relatively few but well chosen collaborators, the majority from outside the NARIs, a few key problems have been addressed and from a regional perspective. While in some countries the necessary legal changes are not yet complete, a more important factor in limiting practical impact has become compliance with new laws and policies. High-level advocacy for policy change is probably necessary in an area such as application of modern biotechnology methods to crop improvement where powerful and often misinformed opposition exists. In this regard, the political influence in global policy spheres of IFPRI, already an active partner of ASARECA in a few projects, is especially notable and ASARECA might learn lessons from their approach. 6. REGIONAL SPILLOVERS The Evaluation was asked to assess documented evidence of spillovers of knowledge, information, technologies and practices facilitated by ASARECA as a regional research platform. In addition to assessing presentations by and interviews with managers of the USKM Program, the ICU and those ASARECA programs that support technology and policy generation, NARS leaders of three 30 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA ASARECA scaling-up projects were interviewed in two countries. We met the regional farmers’ association representing groups across the region; and in Ethiopia 32 farmer-collaborators (13 of them women) from 2 communities involved in ASARECA projects were questioned informally about their knowledge, acceptance of and perceptions of limitations in the technologies. Interviews with senior staff of two CGIAR Centers and former ASARECA networks coordinators, and institutional websites, were used to identify information available on adoption and impact. Spillovers across geographic boundaries, between commodities and to enhance regional trade are fundamental to regionality in conserving resources, avoiding duplication of effort, capturing economies of scale and enhancing synergies. Many valuable examples already exist, from the older Networks as well as from the new programs and from some of their projects (Table 13 below). The regional spillover of improved potato varieties is one example among many. These potato varieties are grown in seven of ten ASARECA countries; beneficiaries are small scale farmers, seed producers, and food processing firms. Another excellent example is orange flesh sweet potato for improved vitamin A nutrition, currently promoted in five countries with huge potential to spill over to other ASARECA countries where sweet potato is grown or could be introduced. Quality protein maize varieties were first tested, promoted and grown by small scale farmers in Ethiopia, Kenya, Tanzania and Uganda, and are being evaluated for release in Burundi, DRC and Rwanda. Across-commodity spillovers are evident when it comes to crop management technologies. Agronomic practices such as conservation tillage, soil amendments, timely planting and weed control developed to manage drought for growing sorghum and millet have been adapted for maize and other crops. Tie￾ridging technologies and practices developed to manage drought for growing maize is being tested for different dryland crops in Ethiopia. There are a smaller but significant number of documented examples of across-trade technology spillovers. Seed policy harmonization is one that comes out very clearly. Seed trade, agricultural products trade, germplasm exchange and technology transfer in many forms are hampered by different seed laws and regulations of ECA member countries (see the Policy issue No 5 above). The harmonization of seed policies among Kenya, Tanzania and Uganda and now being adopted by other countries as discussed under Policy above, is expected to accelerate trade not only in the ASARECA countries but also throughout the COMESA region. However, there is a lot more room to facilitate spillovers. Adaptive research to improve and customize innovations to suit the needs of diverse user groups is still weak and, from our interviews, not well understood even among technology promoters both at the regional and national levels. These would have required regular technology validation (see comment also in the Technology and Innovation section) and learning from practice through principles of integrated agricultural research for development, the lessons from farming systems research, and the IPTA approach’s mix of stakeholders appropriate to each case (farmers, agronomists, breeders, soil scientists, agricultural engineers, producers, marketers, processors distributors, consumers, etc.). All projects should include promotion of spillovers in their design, implementation and communications. Especially as spillover often happen informally, ASARECA and its partners should invest more effort in monitoring their scope and scale systematically, for both intermediate and final technologies. ASARECA requires that projects be implemented simultaneously in at least three countries. However, in the cases visited, the team did not detect significant collaboration in research subsequent to proposal development, or in sharing of intermediate and final results. Clearer mechanisms are needed, and program EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 31 management action that leads to more consistent use by principal investigators and key partners of best practices in collaborative research and development These mechanisms should include systematic use of the development domain concept (see under Reorganization, Issue No 1 above), linked with suitable partners (typically, private sector and/or NGOs) to achieve spillovers. There are no legal frameworks and procedures that allow member countries and stakeholders to access technologies generated through ASARECA funding if those technologies involve high value intellectual property of interest to the private sector. ASARECA currently lacks the legal means to enforce compliance of policies it has assisted in formulating and its own institutional/project regulations in recipient countries. This has implications in case of breach of project-related agreements, IPs, and the facilitation of regional seed movement and trade. ASARECA needs to be aware, in funding multiple partners including private sector organizations, that some have their own intellectual property policy and technology transfer office. As long as ASARECA does not have such a policy of its own, it is in a weak position to negotiate with these partners on these matters and assure regional access to the results of regionally-supported research. The regional legal and international property rights framework needs more attention by ASARECA as a whole. The USKM Program already promotes up-scaling of technologies through its listing of best-bet technologies on its program website. The program can play a stronger role in developing projects for scaling up to additional countries that focus on technologies and innovations that have shown farmer acceptance and initial impact in other programs (Staple Crops, etc). However this sequencing of projects also requires the participation of other program leaders to ensure involvement of local research capacity in facilitating and monitoring the adaptation of these technologies in the new countries. TABLE 13: SOME EXAMPLES OF SPILLOVERS No Technologies/Innovations Developed For Spillover To Spillover Type 1 Information on Cassava Brown Streak Disease Center for Agriculture and Biosciences International for ASARECA 7 countries in the region geographic 2 Banana Bacterial Wilt control strategy Uganda 5 countries in the region geographic 3 Cassava Brown Streak Disease resistant varieties Tanzania Uganda geographic 4 Bean varieties All 5 EAC counties All 5 EAC members and DRC geographic 5 Orange Fleshed Sweet Potato technologies Kenya and Uganda Ethiopia, Kenya, Uganda, Tanzania geographic 6 Tie-ridge technology Maize drought management Sorghum and millet drought management commodity 7 Seed harmonization policy Kenya, Uganda and Tanzania Broader Eastern Africa and COMESA region Across trade and commerce Source: Compiled from assistance from ASARECA program managers. The contribution of the CGIAR and other science and technology partners operating at the regional level (Alliance for a Green Revolution in Africa, NGOs, International Fund for Agricultural Development, etc.) in developing and providing key technologies and innovations designed at the global level (“global public goods”) has been crucial to ASARECA and its national partners in achieving significant spillovers. Such technologies tend to be amenable to adaptation across barriers, and innovative approaches to research and scaling up challenges developed as global public goods are also contributing to spillover rates. 32 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 33 7. BUILDING CAPACITY OF NATIONAL INSTITUTIONS The following questions are posed for this key issue: 1. What has been the balance of benefits to the larger, stronger NARIs and the smaller ones? 2. How effective has ASARECA been at building capacity of the NARIs and other national institutions? ASARECA’s Partnerships and Capacity Development Unit was a key respondent to the key questions on this issue. We also interviewed directors general of three NARIs and senior leaders in three universities, and assessed the capacity of individual national programs and scientists through their research, presentations, reports and quality of interactions with the team. There is no easy or uniform distinction between “large, stronger” and “smaller, weaker” NARS. Our analysis showed that even large NARIs can have serious weaknesses in a specific area: for example, extremely scarce scientific skills can limit effective utilization of available and state-of-the-art biotechnology physical facilities; inadequate conceptualization or application of best practices can limit agricultural innovation or dissemination and scaling up. ASARECA utilizes different mechanisms including integration of capacity building in its regional projects, and targeted activities specifically designed on the basis of subject matter or audience. Through these activities, ASARECA has achieved an impressive record of capacity building across the member countries, and across its seven programs. Capacity building programs range for example from plant breeding training at the M.Sc. and Ph.D. levels, to farmer training in banana tissue culture methods, to potato production and marketing methods (See Table 14). TABLE 14: NUMBER OF CAPACITY-DEVELOPMENT RECIPIENTS BY COUNTRY AND TYPE OF TRAINING, FOR BIOTECH AND STAPLE CROPS ONLY Programs: Biotech Staple Crops Recipient Countries Projects No. of Recipients Recipient Countries; Projects No. of Recipients Kenya, Eritrea, Tanzania, Uganda, Sudan Striga Project 18 breeders 1 PhD 4 MSc Uganda, Kenya, Ethiopia BBW 1 PhD 872 farmers ECA NARIs Base Project 96 Institutional leaders Rwanda Improved markets & income generation 48 banana processors & farmers Kenya, Eritrea, Tanzania, Uganda and Sudan Taenia Project 50 Stakeholders Uganda, Kenya, Ethiopia potato production and marketing 28 farmers 1 technician 41 extension agents Kenya, Eritrea, Tanzania, Uganda and Ethiopia Transfer of Banana Project 1600 Farmers Ethiopia, Kenya, Eritrea, Sudan, Uganda and Tanzania. Sorghum project 216 stakeholders Striga management 496 stakeholders Decision Support System for Agrotechnology Transfer model and Agricultural Production Systems Simulator 26 scientists Kenya, Tanzania, Uganda and Ethiopia Tissue Culture 98 Stakeholders Uganda, Kenya, Ethiopia lowland rice-legumes 436 stakeholders Conservation 41 Stakeholders Uganda, Kenya, Ethiopia, Rwanda Cassava-CBSD 20 researchers 51 trainers Sudan, Ethiopia, Tanzania, Kenya. Maize 4 PhD Uganda, Kenya, Tanzania QPM maize 506 farmers/extn workers Value addition 152 farmers Platform 3 Technicians Uganda, Tanzania Up scaling of soil and water management 63 farmers Totals 1916 2957 Source: Compiled with assistance of ASARECA Partnerships and Capacity Development (PCD) Unit 34 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA As shown in Table 15 there is significant imbalance in the distribution of sub-projects and levels of resource allocation across member country NARIs. For example, 33 percent of ASARECA research funds were allocated to one NARI, demonstrating an apparent bias in favor of a large country NARI. This probably has its origins in the fact that this particular NARI has comparative advantages in research and/or training that has potential to benefit the region. The bias can also result from the proven track record of a strong NARI, the number, quality, and superior qualifications of agricultural researchers devoted to research in the stronger NARIs (see Table 13). It is evident that there is quite an irregular distribution in the shares of full-time equivalent (FTE) researchers, and that some of the member countries do not have a healthy complement of FTEs to respond to their research needs. TABLE 15: RESEARCH STAFFING AND TOTAL RESEARCH SUB-GRANTS ADVANCED BY ASARECA TO MEMBER NARIS Country (NARI) Total Research Staff PhD Holders MSc Holders BSc Holders Total Sub-grants (USD) % of total ASARECA Sub￾grants Kenya 549 149 284 116 7,446,628 33.9 Uganda 270 67 123 77 5,751,217 26.2 Tanzania 294 53 165 76 3,550,300 16.2 Ethiopia 572 95 233 244 1,754,378 8.0 Rwanda 85 6 34 45 1,016,806 4.6 DRC N/A N/A N/A N/A 874,568 4.0 Burundi 68 2 49 17 562642 2.6 Sudan 614 135 270 209 433,540 2.0 Madagascar 118 20 81 17 343,880 1.6 Eritrea 78 1 22 55 206,104 0.9 Total 21,940,063 100 Sources: Staff numbers from ASTI reports; Sub-grants from ASARECA PCD Unit. The bias also has a negative effect upon research resources available from ASARECA to small countries. The proposition must be accepted that the capacity of small countries, both in terms of work force and research experience must be enhanced if these countries are not to be trapped in the vicious circle of exclusion from resources because they are weak. Small countries will remain small in size—a fact of life, but they argue; they must not be doomed to remain weak. The Strengthening Capacity for Agricultural Research & Development in Africa (SCARDA) program was created as a continental initiative by FARA and the SROs to address the plight of weaker NARIs. The program addressed the weaknesses in human and institutional capacity to initiate, design, implement and manage research. Rwanda, Burundi, and Sudan were identified as initial beneficiaries, but other member countries have also benefitted. The emphasis is on upgrading their NARIs staff to at least master’s degree level. SCARDA was successful in training 34 research staff from the these countries in disciplines which include: Plant and Animal Breeding, Soil Science, Range Management and Agricultural Information and Communications Management (see Table 16). EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 35 TABLE 16: MSC STUDENTS PROGRAM SUPPORTED BY SCARDA ECA University MSc Course Country Number of Students Males Females Total Makerere University Plant Breeding Sudan 3 0 3 Rwanda 1 1 2 Soil Microbiology Rwanda 0 1 1 Agric Extension/Education Rwanda 0 1 1 Soil science Rwanda 1 0 1 Burundi 0 1 1 Egerton University Agricultural Information and Communication Management (AICM) DRC 1 1 2 Uganda 2 0 2 Ethiopia 1 0 1 Kenya 1 0 1 Sudan 0 1 1 Tanzania 0 1 1 Animal breeding Rwanda 1 0 1 Soil science Rwanda 1 0 1 Sokoine University Soil science Burundi 1 1 2 Nairobi University Range management Sudan 1 0 1 Animal science Rwanda 0 1 1 Source: ASARECA PCD Unit In addition to upgrading small country NARIs staff to masters’ degree level, ASARECA conducted special learning and mentorship workshops for both male and female NARIs staff from Burundi, Rwanda and Sudan (See Table 17). TABLE 17: NUMBER OF PARTICIPANTS OF ASARECA ‘S LEARNING AND MENTORING WORKSHOPS Country Activity Participants Male Female TOTAL Sudan Change management and learning workshop (round 1) 40 5 45 Burundi Mentoring orientation and learning workshop in 19 5 24 Change management and learning workshop-(round 2) 34 5 39 Rwanda Mentoring orientation and learning workshop 15 3 18 Mentoring orientation and coaching workshop 36 12 48 Total 144 30 174 Source: ASARECA PCD Unit ASARECA has provided other special assistance to weaker NARIs to strengthen their infrastructural capacity through procurement of laboratory equipment; refurbishing and equipping gene banks for in￾vitro conservation; establishment of temperature-controlled screen-houses. Implementation of some projects may have been constrained by lack of transport service. Institutional arrangements with regard to this input vary across countries, yet ASARECA project regulations appear to be standardized. ASARECA may need to focus more on the results/outputs and less on the processes. This is even more valid where the empowerment of small farmers as part of an innovations system approach and in applying FAAP principles is to be encouraged, and if transportation facilities become a limiting factor. 36 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Systematically including MSc and PhD training within projects would be an effective means for ASARECA to contribute to the achievement of and incentives for quality research, to building scientific capacity and improving the focus of universities on priority problems in the agriculture sector. Identification and mentoring of promising scientists from the smaller systems to participate more effectively in projects (e.g., those led from Kenya or elsewhere) also needs to be promoted more vigorously. Retention of scientific human capital is as essential as its development. While ASARECA could play an important role in creating awareness and in advocating at the highest levels of government for appropriate national policies that couple scientific human capacity development with an enabling environment for their retention, it is the responsibility of its member countries to continue to capture the value and effectiveness of its capacity building programs. It is to be noted that effectively, it is these NARIs that have implemented and will continue to implement the many impressive programs that have resulted in the broad range of technologies and innovations cited in this report (see sections on Reorganization, Technology and Innovation and Spillovers). Clearly, ASARECA did play the very important facilitation role (training, access to genetic material, technical backstopping, use of databases etc), but it must be emphasized that it was the NARIs with improved capacity and effectiveness who implemented the programs. Private sector involvement in technology dissemination and scaling up is still inadequate in some of the countries visited. Sustainability of farmer use of new technologies emanating from many projects will be dependent upon stronger linkages between technology generators and private sector service providers. Technologies that are not of interest to the private sector generally require more proactive action by researchers and research institutions to bring in public sector services, NGOs, and farmer associations. Support from ASARECA’s ICU has been valuable in providing improved access to information. However, some NARS and their corresponding ICUs may need extra support in order to be able to serve their scientific staff adequately. 8. CAADP Questions posed by USAID focused on how effectively ASARECA and its programs have supported the planning and implementation of Pillar IV of the CAADP agenda. How effectively has ASARECA provided technical support to the process? How clearly are the outputs and the key expected outcomes of ASARECA-supported activities aligned with the objectives and implementation plans for CAADP? The CAADP Pillar 4 strategy document provided the framework for the team’s assessment. The head of the PCD Unit as ASARECA’s focal point for CAADP was a key interviewee for this issue. The team also questioned the ASARECA focal point on their knowledge of and interaction with CAADP activities in each of the countries visited, as well as representatives of five donor missions met across the countries visited. The engagement of ASARECA in the development of countries’ CAADP compacts has been less than anticipated. Only Rwanda took advantage of the CAADP Strategy to invite ASARECA’s full support at that stage. FARA, with overall coordination responsibility for Pillar IV, reportedly visited some countries at this stage without involving ASARECA. In some cases COMESA only invited ASARECA to attend the signing ceremony for a country compact (Tables 18 and 19). These failures to harness the capacity of ASARECA can only be overcome as the organization becomes better known within COMESA. Although ASARECA’s PCD Unit held a workshop in 2010 to try and EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 37 bring together the country focal points for ASARECA (usually in the NARI) and for CAADP (often in the Treasury), in some countries visited we found that these individuals still have little or no knowledge of each other nor how they will work together. TABLE 18: STATUS OF CAADP PROCESS IN ASARECA MEMBER COUNTRIES Stage In Process Countries No. of Countries Compact signed Burundi, Ethiopia, Kenya, Rwanda, Tanzania, Uganda 6 Investment Plan completed Ethiopia, Kenya, Rwanda, Uganda 4 Implementation Plan completed Ethiopia, Kenya, Rwanda, Uganda 4 Source: ASARECA Newsletter TABLE 19: PRE-COMPACT SERVICES TO BE DELIVERED BY ASARECA Service/Activity With other implementing institutions Develop tools and guidelines for pre-compact processes FARA, AFAAS, RUFORUM Sensitize stakeholders about role of research, advisory services and education in agricultural development FARA, AFAAS, RUFORUM Sensitize and strengthen country teams and stakeholders on Pillar IV issues and FAAP principles FARA, AFAAS, RUFORUM Supplement evidence based analysis of common priorities FARA, AFAAS, RUFORUM From: CAADP Pillar 4 Strategy and Operational Plan 2011-2013. ASARECA is in the process of organizing itself to build national capacities around the FAAP principles of: Empowerment of end-users; Subsidiarity; Pluralism in the delivery of agricultural research, extension, and training services; Evidence-based approaches; Integration of agricultural research; Sustainability; Improved management information systems; Cost sharing with end users; and Integration of gender. ASARECA is well aligned with the CAADP expectations for its support to the implementation of country compacts (Table 20). The ASARECA OP specifies expected results in terms of the generation and uptake of demand-driven agricultural technologies and innovations; policy options for enhancing the performance of the agricultural sector in ECA countries; strengthened capacity for implementing research in the integrated agricultural research for development approach in the sub-region; and enhanced availability of information on agricultural innovations. In all these aspects, it is clear that ASARECA’s strategy and actions are well aligned with CAADP and the FAAP. TABLE 20: POST-COMPACT SERVICES TO BE DELIVERED BY ASARECA Service/Activity With other implementing institutions Consolidate and utilize a pool of experts FARA, AFAAS, RUFORUM Develop tools and guidelines on agricultural research FARA, AFAAS, RUFORUM Engage stakeholders to effectively participate in country implementation plans FARA, AFAAS, RUFORUM Build capacity of country and regional teams, and expert groups FARA, AFAAS, RUFORUM Establish M&E systems for reporting, tracking, review and learning FARA, AFAAS, RUFORUM From: CAADP Pillar 4 Strategy and Operational Plan 2011-2013. 38 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA The MTR reported, however, that some NARS respondents still do not know what constitutes the FAAP principles, so much more remains to be done. Clearly, there is much to be done now by ASARECA in supporting CAADP post-compact implementation stages. ASARECA’s investment plan for support to the implementation of CAADP compacts, presented at a recent Zurich meeting, reportedly was well received by the donor community. ASARECA’s CAADP focal point (leader of the CPU) should reflect on next steps and, more broadly, ASARECA needs to revisit its strategy as it continues to focus on implementing the OP. This should be done without a major exercise in redoing its strategy and OP, and taking advantage of the reduced load of projects as many of the latter mature in 2011. Greater integration among existing programs on crosscutting issues will pay off in the quality of support to CAADP. Capacity development for partners in M&E, innovation systems, IPTA procedures, and gender analysis procedures will be important. ASARECA needs to be more proactive in the wider community of science and technology institutions operating in the region, with a larger and more proactive role in the CAADP implementation processes. Renewed emphasis on value addition will help counter the impression that ASARECA is little more than a grant-giving body. 9. FUTURE PROGRAMS The focus of this key issue is the Feed the Future (FTF) initiative - a USAID supported country-owned program focusing on a limited number of value chains among those identified as priorities in the CAADP Investment Plans. Does ASARECA have the capacity and the flexibility to organize responses to specific demands for technologies and knowledge to overcome constraints in those value chains, improving their efficiency and competitiveness? The Evaluation team reviewed FTF documents provided by USAID/East Africa and/or by the USAID officers who were interviewed in three of the four countries visited. These plans were compared with ASARECA pipeline programs. FTF is a geographically targeted and ultra-poor-oriented agricultural and food security program that is well aligned with the accelerated achievement of MDG targets that are enshrined in the CAADP framework and Country Compacts. Forty-three of the current ASARECA post-OP projects will terminate in 2011, which opens up the flexibility to shift resources to new TFT priorities. Both the CAADP agricultural sector investment plans and the FTF programs are designed as country-led/country-owned. It should be noted, however, that FTF country programs for the most part have been developed, while most of the CAADP investment plans are still being developed. Also the FTF plans are more commodity-oriented than are the CAADP investment plans. There are no a priori conflicts between FTF and CAADP priorities; current alignment of ASAECA with the latter (see Key Issue Area 8 above) already reinforces the orientation/capacity to support responses to FTF. However, in view of variations in commodity and thematic priority challenges identified for FTF, ASARECA will need to identify crosscutting research themes such as soil/water management, and focus on capacity development and methods that can be applied across value chains. FTF priorities include value chain categories for staples and high value crops, and livestock in regional trade. Strategic partnerships will have to be developed to manage cross-network collaboration on policy, economic and institutional constraints. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 39 40 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA In some cases, no shift in current USAID support to ASARECA would be needed. A staple food crop of the poor, such as cassava, is important in USAID’s current support; similarly, current support to land and water management in dryland areas would be well justified to continue through FTF funding. An example of a new potential funding target for USAID within ASARECA’s current priorities would be beans, given the crop’s importance as a regionally traded food crop of the rural and urban poor that is prioritized in several national FTF plans. USAID/Washington is working with the CGIAR reform process to identify agro-ecological zones/recommendation domains where focused action can set concrete goals for intensification of targeted systems. ASARECA could play a key role in its region in mobilizing national partners around key common goals. STRATEGIC FINDINGS ON THE FIVE OBJECTIVES OF THE RFTOP’S EVALUATION SCOPE The specific objectives of the evaluation were to: ASSESS THE RELEVANCE OF THE USAID-FUNDED PROGRAMS/PROJECTS IN ASARECA TO CLEARLY ARTICULATED OBJECTIVES AND EXPECTED RESULTS ASARECA’s current portfolio of seven programs and five units appears fully appropriate to address the Operational Plan and log-frame in terms of catalyzing the development and scaling up of technologies and innovations (including in the biotechnology area) for prioritized agricultural development domains, while contributing to the elimination of policy barriers to cross-border exchange of research outputs and information. USAID support post-reorganization to research in the areas of staple crops, biotechnology and policy innovation has been highly relevant and focused, albeit reductionist rather than integrated into agricultural production systems managed by farmers having different socio-economic needs and opportunities; support to natural resource management research partially addresses these issues. The support of USAID to broader governance and to strengthening management proved critical to the strengthening of those processes especially in the period 2006 to 2008. In terms of facilitating trade opportunities and value chains development in favor of small farmers, significant progress has been achieved through research and advocacy towards the rationalization and harmonization of policies across the region throughout the 10-year period covered by this Evaluation (refer to Table 11). Broader partnerships beyond the NARIs in its member countries under the new Operational Plan are improving ASARECA’s capacity to develop institutional mechanisms that address impact-oriented research and better overcome the constraints to up scaling of technologies. With broader partnerships, the value the Agricultural Innovation System and the IPTA is considerably enhanced, both based on earlier principles of participatory research; however, their application is still uneven. The investment of USAID and others is improving ASARECA’s capacity for strengthening research on crops and livestock technologies, although three or four years is simply too soon for investments to show results, for example for breeders to release drought tolerance staple food crops or for a major policy change such a biosafety law to be fully implemented. However, the significant impacts from delivery of improved crop technologies derived from earlier periods (refer to Key Finding No. 3 above and Strategic Findings on impact below) augers well for the newer activities. Broader geographic networking would strengthen the impact potential of regional research, enhance opportunities for shared benefits, and improve continuity in partnerships in among NARS and with their non-traditional partners as short-term projects phase out. ASARECA’s improved development of technologies in staple crops, biotechnology methods and policy innovation, along with better governance and management practice, when paired with broadened partnerships, can reasonably be expected to enhance the organization’s capacity to respond to new EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 41 regional and member state demands to support CAADP country compacts and related USAID’s Feed the Future initiatives. We, therefore, do not propose any modification to ASARECA’s current structural arrangements. However, relevance (as well as effectiveness) would be significantly enhanced if the existing structure were to be operated and managed more interactively, seeking value addition across programs, units, and projects as appropriate. Doing this would capitalize on ASARECA’s unique position in the region as a regional agricultural research and development organization that cannot be filled by any other at the moment. DETERMINE THE EFFECTIVENESS AND EFFICIENCY OF ASARECA IN IMPLEMENTING THE USAID-FUNDED ACTIVITIES ASARECA’s reorganization was designed to improve the institutional arrangements required to deliver the organization’s results and create impact. As the institution moved from a loose association of 17 commodity and thematic networks and programs to a consolidated mechanism to implement regional programs, guidelines were developed and aligned to Agricultural Innovations Systems. This approach is premised on the concepts of research for development, defined in ASARECA’s Strategic Plan as “agricultural research …. that aims at broad-based growth and poverty reduction”. The investments by USAID/East Africa and other donors in support of this consolidation into ASARECA’s current portfolio of programs have yielded significant and positive results, with USAID/East Africa having shown leadership in important areas. Primary evidence of this is the assembly of a qualified and experienced group of senior scientists as program managers who have impressive scientific backgrounds and skills in presentation, analysis and debate. Consolidation of programs management in Entebbe has improved the coherence in implementation of ASARECA’s portfolio in comparison with the situation prevailing earlier through decentralized management by CGIAR centers. Because of investment by USAID and other donors, ASARECA increased the range of stakeholders sharing ownership of the organization (as in Board composition) and the diversity of partnerships responsible for research and dissemination beyond its traditional partners in national research institutions. Significantly, this investment has also resulted in ASARECA leveraging additional financial resources to further support activities in programmatic areas that had previously relied primarily upon USAID funds. Major challenges remain in achieving distribution of resources across member countries in ways that take into account the research capacity support needs of “smaller, weaker” and more isolated countries while continuing to improve the overall quality of regional research: weaker NARS also need research experience to adapt technologies to local situations, and the potential for mentoring from stronger programs and individuals has yet to be adequately exploited. Projects are currently concentrated in Kenya, Uganda and Tanzania (76 percent of project funds allocation), and represent very different proportional contributions to NARI research budgets (according to senior research managers, highly significant in Kenya and very small in Rwanda, although it was possible to verify precise sums). The concentration in the original EAC countries also does not adequately capture the richness of the region’s biophysical and socioeconomic diversity. More systematic use of regionally mapped development domains, devised by IFPRI at the instigation of USAID/East Africa under its support for ASARECA, would improve targeting of calls for proposals, extending the identification of potential project partners for leading scientists beyond their current experience, encourage closer project collaboration across countries in carrying out joint research, and help ensure more systematic scaling up of research results. 42 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Current focus on a narrower research scope in the new programs, designed to address only higher priority targets and to deliver results within a much shorter lifespan, has improved research efficiency as measured by overhead costs ascribed by ASARECA to individual programs. The evaluators agree with the predominant reaction of the national scientists interviewed that highly focused programs and broader networking are complementary approaches rather than competitive in any way, and urge programs to link their current and past projects and collaborators more consistently into ASARECA’s scientific family— including with those networks and other initiatives that continue independently of ASARECA. ASARECA’s overall headquarters operation, however, has become expensive and can only be justified by its value addition, rather than mainly as a conduit for the delivery of targeted research funding as at present. Value addition to the region includes: • Joint priority setting and research implementation across countries that identify more robust and adaptable technologies and innovations accompanied by better understanding of the limits to their usefulness; • Greater cost efficiency in national research through eliminating unnecessary duplication of facilities and effort; • Opportunities for “smaller, weaker, more isolated” national systems to collaborate with stronger research partners, using across-region mentoring, training and support services; • Faster and easier access by weaker and stronger countries alike to a richer pool of information and portfolio of technologies for local adaptation; • Faster achievement of scaling up and impact from research; sharing resources and effort to advocate for progressive policies in politically sensitive areas within the agricultural sector; and • Mutual reinforcement and sharing of success stories that raise the profile of agricultural R&D within countries and improve the likelihood of further internal and external investment. These concepts for value addition already exist within ASARECA. By pursuing all of these components of value addition more systematically, the organization’s full potential would be realized; its regional visibility and influence enhanced; and, current levels of overall investment in the Secretariat would become easier to justifiable. This shift will require further change in the ways that programs perceive and manage their activities, more delegation with guidance to project investigators, and nurturing by Management. Under the current Operational Plan (2008–2014) the M&E Unit has successfully revised ASARECA’s log-frame, and sharpened its indicators, reducing them from 54 to 24. The organization’s capacity for strategic and participatory planning with stakeholders has improved markedly. The resulting prioritization has enhanced programmatic consolidation, and significantly improved the potential for delivery of tangible outputs along value chains with clear performance targets, measurable indicators, and timeframes. ASARECA's programs currently have in-built Performance Monitoring Plans, but weaknesses still exist in operationalizing these in sub-projects. In some countries the M&E workshops designed to more fully engage sub-project Principle Investigators, NGOs and other participants have only recently been conducted. ASARECA’s weak institutional memory with regard to past outputs and delay in recruitment of an M&E unit head has meant the Unit has struggled to establish systems and procedures that would facilitate effective learning from experiences. Recent progress in this regard is more promising. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 43 EVALUATE THE IMPACTS OF THE USAID-FUNDED ASARECA PROGRAMS ASARECA’s externally managed networks and centrally managed programs supported by USAID up to 2007 generally performed very well in terms of technology development, dissemination and awareness creation among small-farmer end users. ASARECA’s Up-Scaling and Knowledge Management Program is contributing further to their impact. One excellent example from a formal impact study is that of new bean varieties, yielding up to 40 percent more, being grown by millions of small farmers in the region, most of them women in the case of this crop; these varieties entered intraregional trade with nutritional and/or income benefits that by 2005 had reached an estimated 37 million people across seven countries (Kalyebara and Buruchara, 2008). New varieties of potato are being grown in seven ASARECA countries, benefiting small-scale farmers, seed producers, and food processing firms (Kaguongo et al., 2007). Other technologies such as the orange-fleshed sweet potato for vitamin A nutrition being promoted in five countries, and quality protein maize (producing 15 percent improvement in child growth rates in Ethiopia, according to CIMMYT) being grown in four countries so far, are still well below their potential adoption rates although in selected areas impressive rates have been achieved (for example, 53 percent for orange fleshed sweet potato in Uganda according to HarvestPlus, 2010). Successful research and then dissemination of disease-management methods for the most important cassava diseases in Uganda, with support from the former USAID-supported network EARRNET, showed an internal rate of return of 167%, although according to Bua et al. (2007) not all R& D costs could be accounted for in their impact study – and a subsequent epidemic of cassava brown streak disease demonstrated the need for continued R&D if advances are to be maintained. Promotion and application since 2007 of the innovation platform for technology adoption approach promises significant increases in numerical impact rates as skills grow among ASARECA program staff and a broadened array of national and local partners to identify the issues in moving from rhetoric to practice. In the area of cutting-edge biotechnology, countries in the ECA region are at very different levels of developing usable products and uptake pathways. The two leading countries in GMO research are Uganda and Kenya, and Sudan has started field-testing transgenic cotton. The team saw promising biotechnology work on critical cassava diseases in Uganda, which has reached the stage of confined field trials. Biosafety regulations are being worked on but are in general yet to be fully regionalized. Rwanda is making impressive progress in terms of applied biotechnology to support the multiplication and delivery of vegetative planting material (seed) by focusing on tissue culture application for rapid and mass multiplication of disease-free coffee, banana, sweet potato, cassava, and potato, and there is great potential for wider use of this relatively low-technology technique for achieving impact. These observations support the conclusions of the Program’s impact study that documented institutional outcomes in the areas of information and capacity development and intermediate research products, but not yet impact at farmer productivity level (Kiambi, 2011). ASARECA’s Policy Analysis and Advocacy Program, which has had the advantage of uninterrupted financial support over the last 10 years most notably from USAID, has registered important achievements in catalyzing regional trade. Through support to research and the brokering of national policy changes in such areas as seed law and quality standards for processed agricultural products, real impact has been made or is on the point of being achieved to increase domestic and cross-border trade. Between 2002 and 2008, seed imports into the region almost doubled from 9,000 to about 15,000 MT, and intra-ECA seed imports have more than tripled as seed exports from Kenya and Uganda have gradually increased to more 44 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA than 3,000 MT; moreover, the harmonization of seed policies has seen a general increase in price stability for maize seed in the entire region which benefits commercial farmers (Waithaka et al., 2011). Harmonized standards for cassava and potato processed products were published by the East African Community were a significant outcome from ASARECA-supported research (EAC, 2010). Collaborative research and advocacy with ILRI and others in Kenya have enabled small-scale farmers to penetrate extensive milk markets. Taking a regional approach to addressing a small number of significant policy constraints in collaboration with well-chosen partners, the majority from outside the NARIs, has been to key to PAAP’s success (see also Tables 11 and 12). Projects initiated since ASARECA’s reorganization was completed only three years ago have not yet had time to register impacts, although promising outputs and a few outcomes are evident and encourage us to believe that impact will follow. A good example is the support to soil and water management and drought-tolerant varieties for dryland maize production in Ethiopia, and the related research with farmers there on best practices for adaptation to increasing variability in climate and weather patterns; the farmer￾collaborators we met there with local researchers were enthusiastic and the experimental evidence appeared to reinforce the basis for their enthusiasm. A booklet on success stories, while not an analytical study, provides evidence of uptake and progress towards impact (ASARECA, 2010b). While no impact study was found on ASARECA’s capacity strengthening activities, three examples of success stories as outcomes from recent postgraduate training and professional attachments are described in its newsletter (ASARECA, 2011e).A visit to the Jomo Kenyatta University-based biotechnology project on maize drought tolerance was notable for demonstrating contribution towards direct capacity building in six countries of the region in terms of both research and training; this project also enabled the collaborating regional scientists to participate in cutting-edge science. This approach should be replicated more broadly. ASSESS THE SUSTAINABILITY OF THE ASARECA RESEARCH PROGRAM As an organization originally set up by the NARIs of the 10 member countries to bring added value to their nationally-focused activities and impact-creation, ASARECA’s interventions and products continue to be in increasing demand, and moral support from its member countries is solid. With the extension of the ownership and governance in the past five years, it was clear to us that national and regional African institutions’ support has been strengthened further. For example, all three private seed sector representatives we met in Ethiopia and Uganda expressed genuine appreciation for ASARECA’s activities that have resulted in new and harmonized national seed laws that facilitate regional trade. Increasingly close collaboration with regional bodies such as the EAC and COMESA to support implantation of the CAADP agenda can also be expected to leverage new sources of support, from both political and financial stakeholders. Member NARIs’ commitment to direct investment in ASARECA is set currently at a modest US$5,000 per year—and five out of 10 countries are heavily in arrears (ASARECA Audit Report, 2010). Imminent broadening of the stakeholder platform to bring Ministers in a Patron status may help in raising the profile and hence the financial support from countries—although a more formal Council of Ministers would probably be still more effective in this regard. Several striking examples of public investments leveraged through the strategic support for ASARECA projects by USAID have resulted. For example: the biotechnology facilities at NaCRRI in Uganda and Mikocheni Agricultural Research Institute in Tanzania EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 45 have taken over further development of the platforms, and postgraduate training of critical staff has also benefited from government funding in Rwanda and Sudan as a result of the initial investments from key donors. However, more could have been achieved in terms of sustainability had these projects deliberately included sustainability in their frameworks. An indicator of sustained donor support to ASARECA is that the number of donors has remained more or less constant over the past three years, at around eight donors, with a small number of departures and new arrivals annually. The existence of the MDTF, and of the memorandum of understanding (MOU) among ASARECA’s donors including USAID, tends to create confidence in the organization. ASARECA’s alignment with CAADP, its African ownership, and its role as a CAADP Pillar IV supporting organization make it a natural partner for the future. DOCUMENT LESSONS LEARNED FOR FUTURE USAID/EAST AFRICA PROGRAMMING The goal of ASARECA’s institutional re-organization supported and financed by USAID was to move towards “fewer, larger, more strategic research projects.” An important lesson emerging from the evaluation is that the narrow scope of the priorities and activities, and more especially the manner in which this reprogramming has been interpreted, has contributed to the imbalance in allocation of resources among the member countries’ NARIs to the disadvantage of smaller countries that have fewer opportunities to bring to bear their very few skilled scientists. With suitable and pro-active intervention in devising calls and evaluating proposals, this situation can be alleviated to the benefit of all and without loss of regional research quality. The continued existence of some of the networks that were targets for phasing out (AHI, BARNESA, ECABREN) indicate that much broader networking catalyzed by ASARECA beyond the purview of the current consolidated projects still has demand and potential for impact through the continuity of the partnerships forged across NARIs. As the MDTF activities and the principles and procedures for harmonized support to ASARECA are evaluated, we suggest that the continued value of networking be recognized. Costs could be minimized if closer linkage is established with continuing networks, ASARECA-sponsored scientists participate in CGIAR center workshops and professional association conferences, and internet communication continues after projects end. USAID’s sustained support to the ASARECA Secretariat from its formation in 1994 and throughout the current evaluation period has been critical in facilitating and guiding its evolution—USAID contributed 25 percent of ASARECA’s budget over the 10-year evaluation period. As an African-owned and staffed institution, ASARECA has comparative regional advantage in helping countries to address such critical but often emotive issues as gender analysis in research and the safe application of modern biotechnology methods to crop improvement. As CAADP becomes a focus for guiding agricultural research and development in Africa, ASARECA’s strategy and preparedness for its pivotal role as a regional Pillar IV support institution should be increasingly oriented to increasing access to information, knowledge and analytical skills needed to support evidence-based analysis, planning, and development of quality investment programs being developed by its member countries. It is now well positioned to deliver on that commitment. The FAAP principles are now enshrined in ASARECA’s calls for proposals, and their application will be important. FAAP principles are consistent with CAADP’s Pillar IV priorities. The need for transparent planning processes and road maps to define timeframes for major milestones in ASARECA’s technology delivery mission is currently well appreciated by the organization. It was 46 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 47 evident from the evaluation that fulfilling this mission at an accelerated pace will continue to face inherent technology generation and other product development challenges and obstacles. Gestation periods required also for the unique technologies needed to respond to some of ASARECA’s member country demands will impede rapid delivery. USAID has been a consistent supporter of ASARECA’s agenda and its continued recognition of the long research product lifecycles and the timelines to technology breakthroughs and impacts on poverty will be important if the high rates of return registered globally for sound agricultural research are to be achieved. For example, breeding and adapting a new maize variety for local conditions typically requires at least 10 years to its release to users. In view of the MDTF-MTR scheduling delays with consequent loss of some opportunities for coordination and task sharing between that review and this evaluation, coordination among ASARECA’s development partners could benefit from a reassessment by USAID/East Africa and other donors of relevant clauses in their joint MOU. RECOMMENDATIONS AND LESSONS LEARNED KEY ISSUE 1: REORGANIZATION Recommendation 1. ASARECA adds value to regional agricultural technology generation and delivery in many ways. It provides 1) multi-country priority setting and research implementation that can increase cost-efficiency in national research by eliminating duplication of efforts; 2) opportunities for “smaller, weaker” national systems to benefit from collaboration with stronger research partners and mentors; 3) access to a richer pool of information and portfolio of technologies for local adaptation; 4) faster achievement of scaling up and impact; 4) platforms for sharing resources and efforts to advocate for progressive policies in sensitive areas; and 5) ways to mutually reinforce and share success stories that raise the profile and investment opportunities in agricultural R&D. ASARECA should pursue all of them systematically to realize the organization’s full potential, raise its regional visibility, and fully justify its more costly headquarters operation. This shift will require further attention to change management. Recommendation 2. The current focus on a narrower research scope, designed to address only higher priority targets and to deliver results within a much shorter lifespan, has improved research efficiency as measured by overhead costs ascribed by ASARECA to individual programs. However, NARS scientists miss the opportunities provided by the former networks for broader and more sustained regional contact and sharing in applied science. The two approaches are complementary rather than competitive. We recommend that ASARECA and its programs broaden their networking capacity and link current and past projects and collaborators more consistently into ASARECA’s applied science family—including those networks and initiatives that continue independently of ASARECA—to strengthen partnerships, ensure continuity, and enhance the potentials for sharing benefits across the region. KEY ISSUE 2: INSTITUTIONAL CAPACITY BUILDING Recommendation 3. Evidence points to the conclusion that the Board and management of ASARECA have neglected the M&E area—witness the turnover in staff, failure to rapidly replace M&E leadership staff, an apparent lack of and/or inconsistent application of performance benchmarks, and weak data storage and retrieval systems. We recommend that the Board and management take immediate measures to ensure that data is recovered (at least for the past 10 years, including for outsourced networks), retrieval systems are put in place, and the log-frame is applied systematically throughout the organization. Failure to achieve this would imply a need for ASARECA to declare the year 2010 as its official baseline/benchmark M&E capacity-strengthening year of reference. This was the baseline/benchmark year in which capacity strengthening was systematically reassessed and revived for the key areas of performance indicators, data collection instruments, data storage and retrieval systems, and performance monitoring systems. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 49 KEY ISSUE 3: TECHNOLOGY AND INNOVATION Recommendation 4. A wide range of standards and quality check mechanisms are used in member countries before a new technology or innovation is considered acceptable for registration and wider dissemination. National crop variety release committees are widely used; some countries have agricultural product standards. Clarity regarding validation of a new technology would enhance rapid and well￾targeted action in catalyzing scaling up and regional spillover. We therefore recommend that ASARECA institute and promote appropriate sets of harmonized standards or procedures by which technologies of different categories are validated at the regional level. Establishing these standards should be a transparent and consultative process organized by the M&E Unit in collaboration with the respective ASARECA program and carried out once for each category of technology and innovation, in each case by two or three independent and experienced experts drawn from regional and international sources and to whom reference can be made to resolve rapidly any subsequent issues of application. KEY ISSUE 4: BIOTECHNOLOGY Recommendation 5. The linkage between ASARECA’s and NARS’ biotechnology programs is uneven. We recommend that ASARECA’s program assess, before the end of 2011, the status of biotechnology in its member countries to determine and document research capacity and the policy environment for biotechnology product and tools to better establish priority areas for interventions. As the next step, the program should offer and promote low-cost networking and service provision with interested national programs. Future regional biotechnology interventions should focus on providing unique opportunities to build human capacity, technical advice in developing suitable laboratory and field facilities for the effective utilization of upstream biotechnologies, assistance to national programs to be more proactive in influencing national biosafety policy, and assistance to more national programs to partner with private and/or public technology providers from within or outside the region. The program should always work closely with ASARECA’s commodity programs and identify opportunities for cutting-edge technology that has potential to address those intractable productivity-reducing stresses that cannot be addressed readily through conventional breeding. KEY ISSUE 5: POLICY Recommendation 6. The Policy Analysis and Advocacy Program is an excellent example of what ASARECA can achieve through continuous and experienced program management, and through well￾chosen partnerships with regional and international institutions and individual scientists from beyond the NARIs. The program also demonstrates the outcome value of uninterrupted funding from USAID over an extended period. We recommend that ASARECA replicate these techniques in other programs. Recommendation 7. In harmonizing seed policy across the region, PAAP has encountered obstacles in influencing change to pre-existing national laws. Thus, it has sometimes been less difficult to introduce new ones. More broadly, compliance with ASARECA-catalyzed changes in national policies remains a limitation to moving beyond outcomes to impact. For example, there can be institutional and financial incentives for a national ministry to retain requirements for such practices as mandatory trials before commercial release, even when agro-ecological conditions are identical in bordering countries. Therefore, we recommend that the Board of ASARECA consider at its next meeting ways in which the organization can exert stronger influence at the highest regional and national policy levels to ensure policy compliance of member countries to agreements brokered by ASARECA. 50 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Recommendation 8. The introduction of the gender mainstreaming unit is important for ASARECA’s alignment with and promotion of the FAAP principles. Criteria for gender compliance are urgently needed for ASARECA programs and projects, and we urge this Unit and the M&E Unit to advance the current timetable for their development. If this is achieved through the inclusion of gender-related differences among intended beneficiaries as one among several socioeconomic factors to be considered in project targeting and outcomes/impact assessment, then this initiative can be expected to raise ASARECA’s effectiveness in contributing to both gender and poverty targeting. KEY ISSUE 6: SPILLOVERS Recommendation 9. ASARECA requires that projects be implemented simultaneously in at least three countries. However, significant collaboration in research after proposal development sharing intermediate and final results does not necessarily occur. Program management needs to ensure that projects use best practices in collaborative research and development, including systematic use of the development domain concept linked with the private sector and/or NGOs to achieve spillovers. We recommend that all projects with immediate effect include measures to promote spillovers in their design and implementation. Recommendation 10. ASARECA has no legal framework or procedures to ensure access by member countries and stakeholders to technologies generated with ASARECA regional funding. Intellectual property rights need more attention by ASARECA as a whole, and the Board should ensure that the organization develops its internal policies in this area, works with NARS to harmonize plant breeders’ rights, and develops an appropriate legal framework that ensures sharing of benefits of regional public goods. Recommendation 11. There has been significant informal transfer of technologies within and across countries. ASARECA should invest more effort in monitoring the scope and scale of technology transfer in a systematic manner. KEY ISSUE 7: BUILDING CAPACITY OF NATIONAL INSTITUTIONS Recommendation 12. Weaker countries need research experience to be able to adapt technologies to local conditions, and affirmative action may be needed in the selection of participating countries, their research staff, and suitable mentors. A diverse, innovative strategy is needed to make sure that all member countries get tangible benefits from regional collective action. Given that the SCARDA Program has ended, we recommend that ASARECA adopt, before the end of 2011, a policy that systematically catalyzes the mentoring of scientists across the region and includes M.Sc. and Ph.D. dissertation research in all future projects. These measures will increase university participation in regional programs and increase the benefits to countries weaker in any particular thematic area, without loss of overall quality of regional research projects. KEY ISSUE 8: CAADP Recommendation 13. Whereas CAADP is becoming a focus for guiding agricultural research and development, the publication of a Pillar IV Strategy and Operation Plan does not lead automatically and rapidly to mutual knowledge about the roles, responsibilities, and potential contributions by ASARECA. ASARECA needs to be more proactive in the wider community of science and technology institutions operating in the region, with a larger and more proactive role in the CAADP implementation processes. This has several dimensions: significant contributions to the technical review of CAADP plans as they EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 51 52 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA relate to all Pillar IV activities; enhanced capacity to provide relevant knowledge and information support for the implementation of regional and its member countries’ agriculture sector investment plans; advocacy to make sure that regionally harmonized standards are implemented at the country level; advocacy to overcome legal and regulatory bottlenecks in intellectual property rights, including patents and plant breeders’ rights; and promotion and national capacity building for the application of the FAAP principles. Recommendation 14. Whereas ASARECA programs and units have made progress in working together, further integration is needed to exploit their crosscutting potential fully and to better serve countries in implementing their CAADP compacts. ASARECA should manage its current structure in a more interactive manner so that the whole is stronger than the sum of its parts. Recommendation 15. The lack of clarity that existed around the regional division of labor between CGIAR centers and other science and technology partners operating at the regional level (Alliance for a Green Revolution in Africa, NGOs, International Fund for Agricultural Development, and so on) has declined over the past 5 years. However, further coordination among them in regional priority setting and action planning would be valuable and especially in the CAADP context. We recommend that ASARECA take on a more pro-active leadership role in catalyzing more effective cooperation among these entities as part of its strategy for adding value to the region. KEY ISSUE 9: FUTURE PROGRAMS Recommendation 16. USAID should support the revision of ASARECA’s strategy to reflect a better balance between its value-addition (capacity building, sustainability through spillover design, intellectual property rights policy, M&E, and so on) and its research project grant-giving management. Further adaptation by program managers will be needed, and will constitute an important step toward completing the change management process. The revision should represent a re-prioritization of activities within the current Operational Plan, rather than a fundamental reorientation of ASARECA strategy and operations. If current levels of USAID funding are maintained, the maturation and completion of research projects in 2011 will permit this rebalancing to be executed within nine months, in concert with the MDTF. Recommendation 17. Given the shifts in USAID interests in specific commodities and thematic priority challenges proposed in the FTF plans, we recommend that ASARECA identify crosscutting research themes, such as soil and water management, and focus on capacity development, methods, and partnerships that can be applied across value chains. ANNEX A: SCOPE OF WORK AND EVALUATION METHODOLOGY Annex A presents Section C. the Statement of Work and Evaluation Methodology from Task Order AID￾623-TO-11-00003. SECTION C: STATEMENT OF WORK C.1 OBJECTIVE The main deliverable will be a stand-alone report of findings, conclusions, and recommendations for USAID, following the guidelines in USAID’s new evaluation policy1 . This policy requires that an evaluation be undertaken for any activity implemented for five or more years should be evaluated, to provide a measure of the change in development attributable to USAID’s interventions. The findings of the evaluation shall provide a basis for judgment to improve the effectiveness and/or to inform decisions about current and future programming. The scope of work outlined below is in line with the broad objectives of the USAID evaluation policy. The findings will document the outcomes and impact resulting from the investments made by USAID, as contributions to the larger regional agricultural research for development program implemented by ASARECA, with funding from multiple donors. The work of this team contracted by USAID will be organized in close coordination with a team carrying out a mid-term evaluation of ASARECA’s current 2008-2013 Operational Plan, which is being implemented for the World Bank on behalf of the three Development Partners’ that contribute to the Multi-donor Trust Fund: the European Union, the U.K. Department for International Development, and the Canadian International Development Agency. A third evaluation by the European Union of two phases of their support is also planned. Linking the three evaluations will have many advantages. The teams will share information collected from documents and reports, including the reports of three joint supervision missions carried out in 2009 and 2010. All of the evaluators will be able to discuss and contrast their findings and analytical conclusions, and present them in a coordinated way that will benefit ASARECA’s Board of Directors, management and all of its development partners. The TOR for the evaluation team for the Operational Plan/Multi-donor Trust Fund is provided as an Appendix. C.2 BACKGROUND USAID/East Africa requires an evaluation of the regional development impact of USG investments in the Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA). ASARECA is an African regional institution for agricultural science and technology, linked to Pillar IV of the Comprehensive Africa Agricultural Development Programme (CAADP). For over 10 years, USAID/East Africa has supported activities designed to build the capacity of this regional platform for 1 http://www.usaid.gov/evaluation/USAID_EVALUATION_POLICY.pdf?020911 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 53 research for development—the systematic diffusion of technologies and knowledge among countries. Over this period, ASARECA has been supported by a total of eight development partners as the association has implemented a major reorganization of its management structures and modes of operation. In 2009 these partners signed a MOU in which, among other things, they agreed to coordinate reviews, monitoring, and evaluations rather than burden ASARECA with completely separate, repetitive processes. The team contracted by USAID will make an independent evaluation, following the key issues in this Scope of Work. The activity to be evaluated is the Regional Agricultural Research Program in East and Central Africa, grants 623-A-00-02-00095-00 and No: 623-A-00-06-00082-00. USAID/EA has provided a total of US$20 million since 2002 to support the program. Funds were provided as part of the Initiative to End Hunger in Africa (IHEA) and the Global Hunger and Food Security Initiative (GHFSI), which has developed into the Feed the Future Initiative. ASARECA was established in 1994 as a subregional nonprofit association of the National Agricultural Research Institutes (NARIs) in 10 countries of Eastern and Central Africa. The 10 countries include: Burundi, DR Congo, Ethiopia, Eritrea, Kenya, Madagascar, Rwanda, Sudan, Tanzania and Uganda. By 2002 it was coordinating 17 commodity networks and programs, many of which were managed by centers of the Consultative Group on International Agricultural Research (CGIAR). USAID/East Africa supported the following activities: • Policy Analysis project • Agro-forestry research network • Biotechnology project • Cassava network • Bean network • Potato and sweet potato network • Planning, monitoring and evaluation unit In 2006, ASARECA initiated a re-organization process, consolidating the 17 networks and projects into 7 regional research programs managed by an expanded secretariat. The Committee of Directors of the 10 members NARIs was expanded into a 16 member Board of Directors drawn from research, universities, farmers’ organizations, the private sector, NGOs, and COMESA. The secretariat is headed by an Executive Director. An analysis carried out in collaborations with IFPRI and a systematic priority-setting process led to the development of a 10-year Strategic Plan and a 5-year Operational Plan supported by the donors as a group. The secretariat, located in Entebbe, Uganda, coordinates a competitive funding process and oversees regional projects implemented by scientists from national institutions from at least three countries. ASARECA is a member organization of the Forum for Agricultural Research in Africa (FARA) whose programs are guided by the Framework for African Agricultural Productivity (FAAP). ASARECA has been mandated by COMESA to provide leadership in the implementation of the NEPAD CAADP Pillar IV agenda. All of ASARECA’s Development Partners (DPs) have agreed to cooperate in their support, and have signed a memorandum of understanding (MOU) that guides their cooperation. Three of the DPs contribute to a multi-donor trust fund managed by the World Bank. 54 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA The overall objectives of ASARECA are: • Establishment and operation of performance based governance systems and management structures of agricultural research; • Facilitate the generation and uptake of demand driven agricultural technologies and innovations; • Facilitate and promote policy options for enhancing the performance of the agricultural sector in the sub-region; • Strengthen the capacity for implementing agricultural research for development in Eastern and Central Africa • Enhance the availability of information and knowledge on agricultural innovations. USAID/East Africa currently supports a sub-set of ASARECA’s activities, principally the following: • A share of the governance and management functions of the secretariat • Policy Analysis and Advocacy Program (PAAP) • Biotechnology and biodiversity program • Staple crops program • Monitoring and evaluation unit • Capacity building at the secretariat The specific objectives of the evaluation are to: • Assess the relevance of the USAID funded programs/projects in ASARECA to clearly articulated objectives and expected results • Determine the effectiveness and efficiency of ASARECA in implementing the USAID-funded activities • Evaluate the impacts of the USAID funded ASARECA programs • Assess the sustainability of ASARECA research program • Document lessons learned for future USAID/East Africa programming KEY ISSUES TO BE INVESTIGATED The USAID evaluation team will concentrate specifically on the following key issues: 1. Institutional Capacity-building USAID/East Africa in collaboration with other ASARECA donors, made a strategic decision to invest in building the capacity of ASARECA as a lead African regional institutional platform for agricultural research, pulling together scientific expertise and capacity from national institutions and linking them with partners including international institutions, private companies, universities, and non-governmental organizations. The team should evaluate the costs and benefits of this approach, compared to other available options for supporting agricultural technology development. Has coordinated support from the EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 55 donors to ASARECA’s single Operational Plan proved to be an effective strategy for building institutional capacity? 2. Reorganization In 2006, ASARECA initiated an institutional re-organization to enable it to implement its regional research agenda as set out in the new strategy. The goal was to move towards “fewer, larger, more strategic research projects” that respond to clearly defined regional needs and demands. The team should identify those areas where the reorganization of ASARECA’s management system has led to clear progress over the 10 years. What evidence is there of value added as the institution has moved from a loose association of commodity networks managed separately by the CGIAR centers to consolidated regional mechanisms? Would the re-organization have been successful with or without the support from USAID/East Africa? Has ASARECA’s performance been disappointing in any areas over the years? Following the re-organization, an Operational Plan was prepared and it is being used to guide the implementation of activities under the current strategy. The evaluation team should review the current operational plan of ASARECA and determine whether it is sufficient in guiding the delivery of the regional research agenda. The team should identify those areas where the reforms of ASARECA’s governance—the expanded Board of Directors, the creation of the General Assembly, Patron Ministers, and the expanded secretariat—have had positive effects. Have these changes helped to broaden ASARECA’s effective partnerships beyond the 10 NARIs to the broader agenda of CAADP Pillar IV? Have the changes had any negative impacts? 3. Technologies and Innovations What are the most promising technologies and innovations coming out of programs that USAID has supported, in the context of ASARECA’s broader programs? The team should look specifically at the output indicators in the annual work plans and reports. They should then chose a sample of technologies and probe how effectively they have been linked to the private sector, extension agencies, and NGO’s to move them efficiently to wide scale adoption. The objective is to test the following hypothesis: “Regional USAID investments in science and technology through ASARECA have proved themselves to be an effective mechanism for making widely applicable technologies and knowledge available in multiple countries.” Alternatives might have been greater support to the Consultative Group for International Agricultural Research (CGIAR) and other international centers, to US universities through the Collaborative Research Support Programs (CRSPs), direct support to national research institutes and universities, and greater cooperation with multinational companies. What evidence is available and what are the lessons learned? 4. Biotechnology How effectively has ASARECA’s biotechnology program contributed to helping national research institutions in the region to participate in cutting edge science? To what extent has the program opened up pathways for the utilization of biotechnology? What value has it added to other programs working in this area at the regional and national levels? 5. Policy Over the past 10 years, USAID/East Africa has invested resources through ASARECA in creating an enabling policy environment that would contribute to increased agricultural productivity within the sub￾56 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA region. Broad policy areas covered included seed policy harmonization, tariffs, grades and standards, and biosafety framework. The evaluation team should assess and document whether there are any strengths and weaknesses to date in ASARECA’s policy programs on seed harmonization, a regional framework for biosafety, harmonized product grades and standards, and Natural Resource Management in the dry lands? What is the comparative advantage of the ASARECA Policy Analysis and Advocacy Program (ECAPAPA/PAAP) compared to other organizations working on regional policy reform with COMESA and the EAC? The evaluation team should identify specific policy related outcomes with regional impacts. 6. Regional Spillovers As a regional research platform, ASARECA was set up to facilitate regional spillovers of knowledge, information, technologies and practices. The team should assess documented evidence of such spillovers. 7. Building Capacity of National Institutions ASARECA was set up to facilitate the strengthening of national member institutions to manage their programs, develop and disseminate technologies and practices efficiently. This capacity development function was done through training programs, mentorship, physical research facilities development, and sharing and cross-border utilization of regional professionals, scientific facilities and equipment by its member institutions and affiliates. How effective has ASARECA been at building capacity of the NARIs and other national institutions? What has been the balance of benefits to the larger, stronger NARIs and the smaller ones? 8. The CAADP Agenda The team should assess how effectively ASARECA and its programs have supported the planning and implementation of Pillar IV of the CAADP agenda in the countries where it is working. How effectively has ASARECA provided technical support to the process? How clearly are the outputs and expected outcomes of ASARECA-supported activities aligned with the objectives and implementation plans for CAADP? 9. Future Programs Looking forward, the Feed the Future initiative is asking the USAID to “focus and concentrate” on a limited number of value chains among those identified as priorities in the CAADP Investment Plans. Does ASARECA have the capacity and the flexibility to organize responses to specific demands for technologies and knowledge to overcome constraints in those value chains, improving their efficiency and competitiveness? METHODOLOGY AND DELIVERABLES The methodology for the evaluation shall include a combination of the following: • Review of documents including past evaluations, impact studies, annual reports, field reports, proceedings of board meetings, supervision mission reports, etc. • Interviews and dialogues with staff members of the Secretariat, key stakeholders—NARS, universities, private firms, NGOs, CGIAR and other agricultural research institutes active in the region, Development Partners, Board Members, etc. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 57 The evaluation of USAID/EA support to ASARECA is from 2002-2011 and uses the guidelines in USAID’s new evaluation policy. The methodology shall be consistent with the timing and scope for the Multi-Donor Trust Fund (MDTF) Mid-Term Review (MTR) of ASARECA’s Operational Plan (OP). DAI will review the MDTF MTR report against the specific objectives of the USAID evaluation, focusing on the ASARECA programs and functions supported by USAID. DAI will consult with USAID on whether the MDTF Framework of Results meets the USAID/EA standard for specificity regarding results targets for this evaluation, with modification for USAID. DAI will develop a mapping of USAID-supported programs over the 2002-2011 periods and relate it to ASARECA’s operating history, including program organization and significant changes in programs. This will be used as the guideline for comparing impact findings with the major context variables that affected the supported research programs. Where USAID has specified hypothetical counterfactuals— such as alternate investments only through the CGIAR or other international centers, U.S. universities through CRSPs, national research institutes and universities, or greater cooperation with multinational companies—we will rely on available literature to determine if stand-alone impact benchmarks exist for similar programs over the same time periods with adjustments for the relative youth of ASARECA In carrying out this assignment DAI will follow the proposed sequence as detailed below: TABLE A1: SEQUENCING OF EVALUATION TASKS Phase Locations, Dates, and Team Members Tasks Phase I: • Nairobi (May 12–13, 2011) Scope finalization with USAID/EA. Using the Evaluation Scope • Kampala (May 16-17, 2011) evaluation issues mapping discussed in our methodology section, the designated team members Finalization Dr. Kirkby, Dr. Terry, Dr. Zeweldu will finalize the scope of work with a focus on: 1) prioritizing efforts across the specific objectives and the questions in the RFTOP; 2) selecting the sample programs; 3) clarifying the modalities for MTR coordination; and 4) identifying USAID information resources and contact points for the sample programs and country visits. Coordination consultation with ASARECA and the MTR team. Using the final scope, the team will establish the evaluation working relationship with ASARECA staff and the MTR team. Phase II: • Uganda: National Agricultural Research • Uganda fieldwork and data collection. Fieldwork Organization, research networks, university, and user/stakeholders (May 18–21, 2011) • Rwanda: ISAR and stakeholders (May 22-24) • Kenya: International Research Campus, Kenya Agricultural Research Institute, universities, and user/stakeholders (May 25–May 28, 2011) • Tanzania: Dar Es Salaam, Morogoro, and user/stakeholders (May 30-June 1, 2011) Dr. Kirkby, Dr. Terry, Dr. Zeweldu, Research Assistant • Rwanda fieldwork and data collection. • Kenya fieldwork and data collection. • Tanzania fieldwork and data collection. • Information gap filling. 58 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 59 Phase Locations, Dates, and Team Members Tasks Phase III: Analysis and Reporting • Nairobi and home-bases (June 27 full time; June 8–July 8 as needed) Dr. Kirkby, Dr. Terry, Dr. Zeweldu, Research Assistant Analysis of data and formulation of findings, coincident with fieldwork and task 6 (June 2-7). USAID debriefing (June 7). First draft report written (June 7–10). First draft report submission (June 14). Draft revision (June 28–30). Final report submission (July 7). ANNEX B. EVALUATION METHODOLOGY AND MODIFICATIONS PHASE I: EVALUATION SCOPE FINALIZATION On March 12 and 13, Drs Kirkby, Terry, and Zeweldu met with the COTR Dr. Peter Ewell and USAID/EA/REGI staff to review the scope of work and information held by USAID, to discuss the evaluation methodology and coordination with the Multi-Donor Trust Fund Mid-Term Review (MDTF￾MTR), and to identify the programs of ASARECA that would be reviewed. Sample Countries. The task order included four sample countries, Kenya, Rwanda, Uganda, and Tanzania. Discussions with the COTR and USAID/EA/REGI staff resulted in a revision to the program of country visits to replace Tanzania with Ethiopia. It was concluded that a selection of an Anglophone country outside of the East African Community would provide greater insights into the effectiveness of past investments in capacity building and program support. A conference call with ASARECA leadership on May 13 confirmed this choice. Evaluation Methodology and Coordination with the MDTF-MTR. DAI requested that USAID provide an update on the status of contracting for the independent Multi-Donor Trust Mid-Term Review. USAID’s SFSA RFTOP SOL-623-11-000015 incorporated the MDTF-MTR Terms of Reference. Those terms of reference indicated that the following tasks were to be completed by the MDTF-MTR by the end of April 2011 for use by the MDTF-MTR team in May 2011: “Review and evaluation methodology: • Desk review of existing documents; • Focused group discussion with ASARECA management, program managers, stakeholders, development partners; • Survey (for randomly selected beneficiaries using questionnaires developed along the Result Framework; • Impact assessment on specific interventions supported USAID in previous years, prior to the setting up of the MDTF; • Site visits and beneficiary consultation with NARS (NARIs, Universities, NGO, Private Sector, Farmer Organizations).” DAI’s proposed methodology and budget were built on the assumption that the MDTF-MTR work had proceeded as indicated. DAI learned that the MDTF-MTR Terms of Reference had not been executed only after we had been awarded and executed the task order. Discussions with USAID/EA/REGI and ASARECA suggested that the work of the MDTF-MTR consultants would not start until after our scheduled fieldwork had been completed. The final TOR for the MDTF-MTR did not include any EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 61 assessment of USAID supported activities prior to the OP period, or indeed any reference to USAID supported interventions as such. The effect of this change in the TOR for the MDTF-MTR and the delay in its development was to deprive our evaluation team of the data essential to use for the quantitative comparisons that we would have carried out had the information been available. The COTR acknowledged this problem and agreed that substitute methods would have to be employed that were more based on interviews and qualitative assessment than had been anticipated. The Evaluation Team discussions with the USAID/EA/REGI indicated 1) that the ASARECA Evaluation Team would have access to the three prior ASARECA Mid-Term Reviews, which provide sufficient, if not always "gold standard" evidence to apply to the core issue of whether the investment in building the regional research platform has been worth it and whether more investment is advisable; 2) that as long as the report was open and transparent about the data and information issues encountered, that the team should be able to provide responses to most of the questions asked in a way that will be accepted by USAID, the MDTF parties, and ASARECA itself. Discussions with the USAID/EA/REGI staff and the COTR were held in Nairobi on coordination with the MDTF-MTR team. Arrangements were made to put the two team leaders in contract to discuss times and places where exchange of information could take place once the MDTF-MTR’s team’s work began. We had a first conference call with the MTR team on May 24 and a first meeting was scheduled for June 7, to be followed with email and telephone conversations. The Team submitted a draft report to USAID on June 22, a date modified with Contracting Officer approval upon recommendation by the COTR to permit greater interaction with the MDTF-MTR team. The two teams held a joint feedback session with ASARECA staff on June 17. We shared the 7-page draft summary of key findings with the MDTR-MTR on June 24 and informal consultations continued thereafter. The final report was shared with the MTR team leader after delivery to USAID on July 22 with comments received and incorporated in this revised final version of the evaluation report -- with COTR email approval of inclusion of the modifications on August 8. USAID/Washington De-Briefing. The COTR requested and DAI agreed that Dr. Eugene Terry would provide a de-briefing to interested USAID/W staff on or about June 6, 2011. Dr. Terry gave the briefing as scheduled. ASARECA and Country Coordination. The USAID COTR kindly made initial contact with ASARECA leadership and USAID representatives in the four (4) sample countries to supply them with the Evaluation Team’s scope of work and to transmit proposed working schedules. The Evaluation Team held consultations with ASARECA headquarters management and staff on May 16 and 17 to discuss the scope, start interviews, and to start data collection with program managers, financial managers, and M&E staff. Impact Analyses. Discussions with ASARECA staff revealed only two impact analyses performed on ASARECA-supported programs, one in the policy area and the other in biotechnology. Our original scope indicated that the team could not perform impact analysis, but would use already available impact analyses to compare the ASARECA program with CGIAR-benchmarked standards. We attempted to locate impact analyses specific to ASARECA programs from each country, from funding agencies, from CGIAR centers and in the literature, with limited success. CGIAR centers and/or NARS scientists identified further impact studies related to the bean, cassava and potato networks, carried out by either NARS or CGIAR scientists. Several success stories in the making were described in a recent ASARECA 62 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Newsletter, but do not constitute formal impact studies. The scarcity of impact analysis of the ASARECA programs made the comparison of impact performance with CGIAR measures a moot point. PHASE II: FIELDWORK In the course of this evaluation, team members met with nearly 180 persons in individual and group settings. Their institutional affiliations, names, and roles are provided in Annex C. PHASE III: ANALYSIS AND REPORTING: QUESTIONS ASKED AND METHODS APPLIED. Ordering of Key Issues. We placed the reorganizational questions in first position as the reorganization has been the primary event in the institutional evolution of ASERECA over the evaluation period. 1. Reorganizational Questions From the reorganization process begun in 2006, there were five questions to consider. The Evaluation team interviewed the Deputy Executive Director of ASARECA and his Program and Unit Managers, three Board members representing NARIs and the CGIAR Centers, and four other senior NARS leaders and also other scientists and stakeholders across four countries. First, “what evidence is there of value added as the institution has moved from a loose association of commodity networks managed separately by the CGIAR centers to consolidated regional mechanisms?” Our suggested measures will be provided by the application of the Intermediate Outcome Indicators for Result 2 (generation and uptake of demand-driven agricultural technologies and innovations facilitated of the ASARECA MTDF framework). The post-organization measures will be provided by the MDTF-MTR consultant. We will attempt to construct a set of comparable pre-reorganization measures and will select a sample of the best-documented commodity network reports to construct a set of roughly comparable indicators for the period before reorganization. Methodological Change: The MDTF-MTR did not provide post-reorganization intermediate outcome indicators within the timeframe of this evaluation. We could not find direct documentation of the pre￾reorganization intermediate outcome indictors. Therefore, our interviews became focused on programmatic outcomes and impacts of pre-reorganization programs (cassava mosaic disease tolerance, policy influence, etc) through interviews. Second, “would the re-organization have been successful with or without the support from USAID/East Africa?” We will answer this question through interviews using the MTR report on result indicators to query stakeholders who span the two periods. Method used: Interviews with program managers and researchers were used to address this question. USAID’s support to the ASARECA Secretariat was consistently as a key determinant of re-organization success. Third, “has ASARECA’s performance been disappointing in any areas over the years?” We will answer this question through the same structured interviews using the results framework as a prompt and asking for significant differences in performance, positive or negative, between the CGIAR-led commodity networks and the larger and broader regional programs.” EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 63 Method used: Interviews with program managers and researchers suggested that a better balancing of commodity network and highly targeted research grants may provide better value-addition. Pre-OP results were not available to use in these interviews. The remaining two questions relate to the governance reforms and ASARECA’s performance. “Have these changes helped to broaden ASARECA’s effective partnerships beyond the 10 NARIs to the broader agenda of CAADP Pillar IV?” The evidence base for the interviews around this question will be the partnership listings for the individual programs as revealed in the MTR and set against the partnership profiles of selected CGIAR, CRSPs, and other regional research programs. We will pay special attention to whether funds are concentrated with the NARIs or are more broadly used. Method used and methodological changes: The total number of project partnerships in the program areas was researched directly. The time required to do this work directly, rather than use the MTR figures, displaced the time that we would have spent researching similar program profiles with other international and regional research programs. Last, “have the changes had any negative impacts?” We will answer this question using the results of the MTR on governance and operational indicators to guide structured interviews to compare pre-OP and post-OP status. This will require interviews with organizations and individuals who span the two periods. Method used: Interviews were used to address this question as proposed, but quantitative data points to compare pre-and post-reorganization changes were not available. 2. Institutional Capacity Questions “Has coordinated support from the donors to ASARECA’s single OP proved to be an effective strategy for building institutional capacity?” USAID has asked for a cost and benefit approach focused on ASARECA’s performance in the use of USAID funds as an organizer of national institutional and partner (international institutions, extra-regional universities, private companies, and NGOs) capacities to support agricultural technology development. This question requires the definition of a performance measure, using it to assess pre-OP and post-OP performance of ASARECA, and then comparing the performance of ASARECA as a regional entity to other institutions/networks that operate regionally. We propose that the primary quantitative measure be the following: overhead and general administrative costs (including capacity building at the secretariat) of ASARECA compared to the funds mobilized through ASARECA to support research and national agricultural research capacity development. Counterfactual cases would primarily be CGIAR regional programs and CRSPs with regional presence, where comparable costs and uses of funds can be identified. This measure can then be qualified using the answers from key issues 3-7 on the impact side of research expenditure. Method Used and Modification: It should be noted that this key issue and the response to the key question is predicated on the presumed existence of comparable data for performance measures pre-OP and post￾OP performance of ASARECA. Such data was unfortunately unavailable and represents a key weakness in the institutional memory of ASARECA and its M&E system. It should be noted also that, while the team compared overhead rates with institutions in the region, little relevant data that would facilitate comparison of ASARECA’s performance to that of other institutions operating regionally (counterfactuals) was available. The responses obtained were based largely on qualitative analysis of this issue and responses to structured interview questions with ASARECA Management, three Board members and Program Managers, and stakeholders shown in the Contacts Annex. 64 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 3. Technologies and Innovations USAID’s hypothesis is: “Regional USAID investments in science and technology through ASARECA have proved themselves to be an effective mechanism for making widely applicable technologies available in multiple countries.” Most research impact evaluations use standard economic approaches that assume linearity in causality and hold environmental variables constant or at least assume a constant trend in the environmental variables. The MTDF MTR consultant will most likely use existing studies and the collected survey information that uses the following causal model: problem identification–research– research output–dissemination–uptake/adoption–impact. Our evaluation team will use the MTDF-MTR impact assessment for the pre-reorganization period, combined with other published impact studies that have been carried out on the programs focused on technological innovation. A useful benchmark for the counterfactual question exercise has been developed by the CGIAR Science Council Secretariat using a meta-analysis approach. These will be used to address the hypothesis and the question: “What evidence is available [to compare USAID support to ASERCA in terms of technological impact against other organizational alternatives] and what are the lessons learned?” This analysis will help us sort through the ex ante projections of benefits in the current USAID-supported ASARECA programs to select a sample of promising technologies to discuss with private sector companies, extension agents, and NGOs. We will structure these interviews to seek their viewpoints on likely adoption rates and timeframes to impact to answer the question: “What are the most promising technologies and innovations coming out of programs that USAID has supported, in the context of ASARECA’s broader programs [following reorganization in 2006]?” Method used: We obtained presentations and reports from the technology-generating programs of ASARECA and interviewed their staff, with a focus on Staple Crops as the largest program and having an agenda supported by USAID over the entire 10-year period. Since quantitative documentation from ASARECA’s project performance monitoring and evaluation was largely unavailable until 2010 (due to the M&E deficiencies noted in the report under the Institutional Capacity issue, and the exclusion of impact assessment from the final TOR for the MDTF-MTR), these issues were addressed through more extensive stakeholder interviews and with the Partnerships and with M&E program managers. Further material for this key issue was developed through analysis of projects performance reports received from program managers who were asked to complete a summary table designed by reviewers. The table was designed to show project title; start and end dates, source and amount of funding; countries and institutions; expected outputs; actual results; current status and remarks. These questions were designed to assess regionality, resources allocation, delivery of results, and status of the projects. We also increased to 15 the number of NARI leaders of relevant crop programs interviewed across the four visited countries, and interviewed six ASARECA project leaders in four NARIs and two universities. Information was also obtained from interviews and reports received from the former ASARECA network coordinators of BARNESA and PRAPACE, and regional leaders of the international centers Bioversity and CIP were interviewed. 4. Biotechnology There are three questions. First, “how effectively has ASARECA’s biotechnology program contributed to helping national research institutions in the region to participate in cutting-edge science?” In consultation with USAID, we will agree on a limited set of modern biotechnologies to address this question. We will also need to agree on the weighting of the evaluation effort among modern biotechnologies used to support classical crop breeding, soil, and agronomic investigations and programs involving the development of GMOs. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 65 Second, “to what extent has the program opened up pathways for the utilization of biotechnology?” The two categories here are utilization for research and utilization in commercial/public space. Using the two categories defined for the first question, utilization for research will be determined via review of ASARECA program reports and interviews with biotechnology program stakeholders. Utilization in commercial/public space of transgenic organisms depends upon the legal and operational development of a biosafety framework and a biosafety law and associated regulations. Uganda and Kenya are the two countries with biosafety systems functioning in pre-commercial space. We suggest that we use the case of Bt cotton or transformed bananas. Third, “what value has it added to other programs working in this area at the regional and national levels?” We will answer this question through interviews with other ASARECA program leaders, CGIAR programs, NARIs, and seed and agrichemical companies. Method Used: We solicited and received a presentation by the Biotechnology Program, interviewed program staff and reviewed their reports. Biotechnology facilities were inspected in Kenya, Ethiopia and Uganda, including confined field tests in Uganda, and their managers and institutional directors were interviewed also in Rwanda. We interviewed a senior university scientist and ASARECA project leader in this field in Kenya, and the director for Africa of the International Service for the Acquisition of Agri￾Biotech Applications (ISAAA). A scheduled meeting with a private sector entrepreneur in this field unfortunately failed to take place. One Evaluation team member brought detailed and up-to-date knowledge of this sector across the region as a result of his association with a U.S. university project. 5. Policy We will attempt to use the CGIAR approach for Policy Oriented Research POR) Impact Assessment to address the question: “What is the comparative advantage of the ASARECA Policy Analysis and Advocacy Program compared to other organizations working on regional policy reform with COMESA and EAC?” This approach uses the causal model of inputs leading to immediate policy research outputs that are disseminated (uptake) via direct and indirect pathways to influence policy decisions whose implementation generates impact. However, the comparative advantage assessment depends upon the existence of the policy impact studies that have completed the cost-benefit analysis of the policy change. If the policies have not been changed at a regional level and ex ante or ex post cost benefit analyses do not exist, then the evaluation will have to focus on the relationship between inputs and outputs that can be confirmed using key informant interviews. We will only compare other research organizations working on the same policy area to ensure comparison of like with like. Difficulties in ring-fencing input costs per policy constraint studied may limit the assessment of comparative advantage. Method Used: We could find no policy impact studies, except on seed law harmonization, that would form the core of the analysis. We therefore had to shift to a qualitative assessment of the policy programs based upon study of secondary sources and interviews. The Evaluation team considered the program’s initial presentation, reports, publications, and responses to requests for results and outcomes information around the above questions; and interviewed the Program Manager. At ASARECA the head of the gender mainstreaming initiative was interviewed, as well as one NARI focal point in gender analysis in research, as this topic falls under the program’s remit. We interviewed seed trade association and company representatives in Ethiopia and Uganda, but were unsuccessful in arranging a meeting with the Ugandan agency piloting the work on product standards. The director of the African Center of ISAAA was interviewed to assess biosafety work, and senior management of the International Livestock Research Institute on divisions of effort among institutions. We made the decision to give less emphasis to 66 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA assessing dryland resource management, as a compromise in the face of non-availability of some information expected in advance from the Mid-Term Review. 6. Spillovers International research support focuses on selecting research problems with what should be explicit consideration of the potential spillover benefits in similar agroecological and socioeconomic environments. The team will focus on the three main types of spillover effects that are generally evaluated in research programs: across location, across commodity, and price. The team will elicit documented evidence of these spillovers from ASARECA annual reports and any specific studies or publications on regional spillovers. The team will inventory the documented cases of spillover, assigning the quality of documentation as either subjective (based on expert opinion) or objective (based on quantitative estimation or measurement). Method Used: The ASARECA M&E system did not contain enough information to categorize spillover effects in the expected three categories. Spillover impact studies were not available. Therefore, spillover value is primarily based on expert opinion and, in some cases, end user perception. In addition to assessing presentations by and interviews with managers of USKM, the Information and Communication Unit and those ASARECA programs that support technology and policy generation, NARS leaders of three ASARECA scaling-up projects were interviewed in two countries. We met the regional farmers’ association representing groups across the region; and in Ethiopia 32 farmer-collaborators (13 of them women) from 2 communities involved in ASARECA projects were questioned informally about their knowledge, acceptance of and perceptions of limitations in the technologies. Interviews with senior staff of two CGIAR Centers and former ASARECA networks coordinators, and institutional websites, were used to identify information available on adoption and impact. 7. Capacity Building “What has been the balance of benefits to the larger, stronger NARIs and the smaller ones?” Answering this requires a compilation from ASARECA M&E records of the training programs, mentorships, physical research facilities development, and sharing and cross-border use of research professionals, scientific facilities, and equipment by member institutions across member states. “How effective has ASARECA been at building capacity of the NARIs and other national institutions?” is much more complex because of the attribution issues, the volatility of support to NARIs over the period to be evaluated, and the need to track the evolution of NARI capacity using acceptable benchmarks. We will use Agricultural Science and Technology Indicators, primarily the absolute value indicators of total finance and researcher FTEs (with gender disaggregated data) and the intensity ratios of research spending as a percentage of agricultural gross domestic product and FTE researchers per million farmers as the core capacity indicators for the countries which the team will visit Kenya, Uganda, Tanzania, and Rwanda to explore the effectiveness and the attribution issues through key participant interviews. Method Used: Ethiopia was substituted for Tanzania in the sample of countries. ASTI indicators were used to establish the research FTEs, but we could not find data that would permit attribution of research expenditure to ASARECA intervention other than on human resource development. ASARECA’s Partnerships and Capacity Development Unit was a key respondent and provider of data on the key questions on this issue. We also interviewed directors general of three NARIs and senior leaders in three universities, and assessed the capacity of individual national programs and scientists through their research, presentations, reports and quality of interactions with the team. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 67 68 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 8. The CAADP Agenda First, “how effectively has ASARECA provided technical support to the CAADP Process?” The MDTF MTR should provide the information on whether the CAADP Implementation Plan was agreed to and is operating as planned. Second, “how clearly are the outputs and expected outcomes of ASARECA￾supported activities aligned with the objectives and implementation plans for CAADP?” Answering this broader effectiveness question requires a timeline of inputs provided by ASARECA programs to plot: a) the development of Pillar 4 elements as the designated SRO under the FARA working within the context of the FAAP, and b) the CAADP Compacts, the Country Investment Plans, and the implementation of the Investment Plans as part of the Regional Economic Community. Our team will interview CAADP Country Contact Points and the Pillar 4 country team leads in Kenya, Uganda, Tanzania, and Rwanda. Method Used: The MTDF MTR was not completed within the time frame of this evaluation and did not provide the required information. The CAADP Pillar 4 strategy document provided the framework for the team’s assessment. Interviewing the head of the PCD Unit as ASARECA’s focal point for CAADP was central to this issue. ASARECA provided their qualitative information on the types of services that they delivered or are prepared to deliver at each stage of the Compact implementation. The team also questioned the ASARECA focal point on their knowledge of and interaction with CAADP activities in each of the countries visited, as well as representatives of five donor missions met across the countries visited. 9. Future Programs and Feed the Future “Does ASARECA have the capacity and the flexibility to organize responses to specific demands for technologies and knowledge to overcome constraints in those (FTF) value chains, improving their efficiency and competitiveness?” There is no immediately apparent conflict of objectives because FTF is USAID’s support toward country-owned, value chain concentrated, geographically targeted, and ultra￾poor-oriented agricultural and food security programs that are well aligned with the accelerated achievement of MDG targets that are enshrined in the CAADP framework and Country Compacts. We will use the publicly available FTF country implementation plans to identify the staple value chains that are specifically named and compare and contrast these to the pipeline of ASARECA programs to look at their alignment with FTF on a percentage of program resource allocation basis. The team will request from USAID/EA the planning time horizon and its benchmarks for program flexibility to help us answer this question. For this purpose, by benchmarks we mean the specific type and level of efficiency and competitiveness constraint between or within a value chain that would trigger the demand for a program shift. We would then be able to examine whether and how the research pipeline commitments and the programming approval process constrains or permits change. Examples of program shifts could include halting one commodity program to replace it with another, abandoning crop breeding to focus on crop management research, and shifting all programs to incorporate a larger mandatory focus on a crosscutting issue such as climate change, drought tolerance, and female labor saving. Method Used: The Evaluation team reviewed FTF documents provided by USAID/East Africa and/or by the USAID officers interviewed in three of the four countries visited. These plans were compared with ASARECA pipeline programs by the evaluation team. ANNEX C: LIST OF PEOPLE CONTACTED Country, Organization, Name Designation UGANDA ASARECA, Entebbe (www.asareca.org) Dr Seyfu Ketema Executive Director (ED) Dr Eldad Tukalinwa Deputy Executive Director (DED) Programmes Pamela Tumwikirize Programme Assistant, DED Office Dr Fina Opio Programme Manager, Staple Crops Dr Ivan Rwomushana Programme Assistant, Staple Crops Diana A. Oyena Programme Assistant, Staple Crops Dr Mwamburi Mcharo Programme Manager, Non Staple (High Value) Crops Katafire Maureen Programme Assistant, Non Staple Crops Dr Hezron Mogaka Programme Manager, Natural Resources Management and Forestry (NRM) Dr Jean Ndikumana Programme Manager, Livestock and Fisheries (LFT) Sarah Mubiru Programme Assistant, LFT Dr Charles Mugoya Program Manager, Agro-Biodiversity and Biotechnology (Agro-Bio) Clet Wandui Masiga Programme Assistant, Agro-Bio Dr Michael Wiathaka Programme Manager, Policy Analysis and Advocacy Program (PAAP) Miriam K. Programme Assistant, PAAP Forough Olinga Gender Mainstreaming Expert Dr Lydia Kimenye Programme Manager, Up-Scaling and Knowledge Management (USKM) Kazoba M. Programme Assistant, USKM Dr Joseph Methu Head, Partnerships and Capacity Development (PCD) Doris Akishule Programme Assistant, PCD Nyagahima Jacky Head, Information and Communications Unit Enock Warinda Head, M&E Unit Sserwadda Patrick Programme Assistant, M&E Anne Rictveld Associate Expert Wellington Jogo Associate Expert William Tinzara Associate Scientist Ocimati Walter Associate Scientist Mukuriah Nelson Head, HR & Admin Siifa B. Lwasa Admin. Coordinator Miracle Arikiza Admin. Assistant Techalew Negasi Head of Finance Unit Felix Onama Senior Accountant Herbert Mbuga Accounts Assistant John Kyebagade I.A-Audit ASARECA Mid-Term Review of Multi-Donor Trust Fund (TripleLine Consultants) Dr Barbara Adolph Team Leader David Smith Member Dr Wyn Richards Member Winston Manzi Procurement Specialist EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 69 Country, Organization, Name Designation National Agricultural Research Organization, Entebbe (www.naro.go.ug) Dr Denis T. Kyetere Director General, and ASARECA Board Member Dr Sylvestor Dickson Baguma Director, Quality Assurance National Crops Resources Research Institute (NaCRRI), Namulonge, Kampala (www.naro.go.ug) Dr James Ogwang Director Dr Michael A. Ugen Team Leader, National Bean Programme Dr Titus Alicai Research Officer (Plant Pathologist), Cassava Dr Julius P. Serumaga Research Officer, Maize and Rice Programme, and ASARECA project leader “Quality Protein Maize” Dr Bua Anton Team Leader, National Cassava Program Dr Jimmy Lamo Rice Breeder USAID Uganda Martin Fowler Agricultural Advisor John Brighenti Agricultural Officer Gaudensia Kenyangi Agricultural Development Specialist European Union, Delegation to Uganda (www.deluga.ec.europa.eu) Jean-Louis Veaux Attache/Programme Officer – Rural Development Dr Michael Duerr Consultant (Formerly European Union Advisor at ASARECA, 2003-2009) Bioversity International, Uganda Office (bioversityinternational.org) Dr Eldad Karamura (and colleagues) Country Director and Coordinator of BARNESA International Center for Tropical Agriculture (CIAT), Coordination for Africa (www.ciat.cgiar.org/Africa) Dr Robin Buruchara Africa Coordinator and Coordinator of the Pan-Africa Bean Research Alliance Uganda Seed Trade Association, P.O. Box 29726, Kampala Dr Ruth N. Ssebuliba Executive Secretary KENYA USAID John Power Deputy Director, USAID/EA Dr Peter Ewell Agriculture Advisor, USAID/EA Jeff Austin Regional SPS Advisor Dr Shirley Erves Kore Senior Policy Advisor and Strategic Performance Management Hudson Masambu Program Specialist-USAID/EA Washington Otieno Oluoch Agricultural Biotechnology Specialist, USAID-Kenya Kenya Agricultural Research Institute (KARI), Nairobi (www.kari.org) Dr Peter F. Wandera National Focal Point for ASARECA Dr Lawrence Mose A/Director, Planning, Monitoring and Evaluation Dr Patrick Gicheru Director, National Agricultural Research Laboratories (NARL) Dr Zakayo M. Kinyua Coordinator, Crop Health Programme (NARL) and Principal Investigator ASARECA project “Enhancing Utilization of Quality Seed Potato by Smallscale Farmers” Hortensia Mwangi Investigator on ASARECA project “Upscaling of Soil and Water Technologies and Drought Tolerant Cultivars for Maize” (NARL) International Service for the (www.isaaa.org) Acquisition of Agri-Biotech Applications (ISAAA), AfriCenter, Nairobi Dr Margaret Karembu Director, AfriCenter University of Nairobi, Faculty of Agriculture, Kabete, Nairobi (www.uonbi.ac.ke) Dr James W. Muthomi Senior Lecturer, and Co-Principal Investigator on ASARECA project on Banana Wilt Johnson Kwach PhD student Jomo Kenyatta University of Agriculture and Technology (www.jkuat.ac.ke) 70 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA Country, Organization, Name Designation Dr Jesse Machuka Professor; Principal Investigator on ASARECA project Eastern Africa Farmers Federation, Nairobi (www.eaffu.org) Mainza Mugoya Program Officer, Policy and Advocacy International Livestock Research Institute (ILRI), Nairobi (www.ilri.org) Dr Carlos Sere Director General, and ASARECA Board Member Bruce Scott Director, Training, Communications and Partnerships International Potato Centre (CIP), Nairobi (www.cipotato.org) Dorvin E. Stockdale Regional Leader, Sub-Saharan Africa Dr Elmar Schulte-Geldermann Integrated Crop Management Expert Dr Berga Lemaga Country Director, Uganda (met in Addis) Dr Margaret McEwan DONATA Research Leader RWANDA Institut des Sciences Agronomiques du Rwanda (ISAR) (www.isar.rw) Dr Daphrose Gahakwa Director General Tona Isibo Head of Planning Dr Claver Ngaboyisonga Director Scientific Research Victor Ruganzu National Focal Point for ASARECA Jean Bosco Shingiro Head, Post Harvest Unit Gashaka Gervais Head, Cassava Research Programme Svetlana Head, Banana Research Programme Gertrude Knight Entomologist Theodor Asiimwe Head of Biotechnology Unit, Head of Agroforestry National University of Rwanda (NUR), Butare (www.nur.ac.rw) Dr Jean Nduwamungu Professor Adrie Mukashema PhD Student (Geo-Informatics) ETHIOPIA Ethiopian Institute for Agricultural Research (EIAR), Addis Ababa (www.eiar.gov.et) Dr Solomon Asefa Director General, and ASARECA Board Member Dr Eshetu Ahmed Coordinator, Planning, Monitoring & Evaluation; and National Focal Point for ASARECA Dr Tolessa Debele Director, Soil and Water Research Yeshi Chiche Agricultural economist; Coordinator for Gender Research Dr Rezene Fessehaie Coordinator, UNEP/GEF Project on Removing Barriers to Invasive Plants Management in Africa Dr Belayneh Admassu Coordinator, Agricultural Biotechnology Research Center Research Program, Holetta Agricultural Ethiopian Seed Growers and Processors Association (ESGPA), Addis Ababa Dr Hailu Beyene Manager, ESPGA Melaku Admasu Chair ESPGA and Pioneer Ethiopia USAID – Ethiopia Daniel Moore Agricultural Advisor [by telephone] The World Bank, Ethiopia and Sudan Offices (Alegesse@worldbank.org) Assaye Legesse Senior Agricultural Economist Rural Development (AFTAR) Melkassa Agricultural Research Center (MARC) (www.eiar.gov.et) Dr. Setegn Gebeyehu Director Habtamu Admassu ASARECA Project Leader “Making the Best of Climate Change” Erensu Degu Member of scientific staff Mulatwa Wondimu Member of scientific staff Etagegnehu Mariam Member of scientific staff EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 71 Country, Organization, Name Designation Laike Kebede Member of scientific staff Legese Teshome Member of scientific staff Daniel Bekele Member of scientific staff Dejeru Abera Member of scientific staff Freyew Kelemu Member of scientific staff Solomon Jamal Member of scientific staff Dr Adam Bekele ASARECA Project Leader, “Upscaling of Soil and Water Technologies and Drought Tolerant Cultivars for Maize” Asheba Tesesa Member of scientific staff Ahmed Ibrahim Member of scientific staff Dr Dagane Wegary Member of scientific staff Getachu Ayana Member of scientific staff Mohamed Saleh Member of scientific staff Gesaheza Toloha Member of scientific staff Israel Bekele Member of scientific staff Husain Harim Member of scientific staff Ermias Alemu Member of scientific staff Tsegae Wabet Information and Communications Girma Mamo Member of scientific staff Tamirat Fikada Member of scientific staff Aleleyn Aseja Member of scientific staff Libibisa Village, Shoa Magisa Yufa Farmer/Participant in trials or demonstrations Kabede Lenjisa Farmer/Participant in trials or demonstrations Alamu Tufa Farmer/Participant in trials or demonstrations Giba Denis Farmer/Participant in trials or demonstrations Alamu Bejiga Farmer/Participant in trials or demonstrations Jambeletu Sida Farmer/Participant in trials or demonstrations Girma Bejiga Farmer/Participant in trials or demonstrations Kori Bonjise Farmer/Participant in trials or demonstrations Talibo Terisa Farmer/Participant in trials or demonstrations Etenesh Kunbi Farmer/Participant in trials or demonstrations Dabe Belde Farmer/Participant in trials or demonstrations Abara Nagade Farmer/Participant in trials or demonstrations Marmarsaa Village, Shoa Abara Frua Farmer/Participant in trials or demonstrations Lalisa Cawaka Farmer/Participant in trials or demonstrations Malba Biru Farmer/Participant in trials or demonstrations Danse Bose Farmer/Participant in trials or demonstrations Yami Buna Farmer/Participant in trials or demonstrations Koreut Jima Farmer/Participant in trials or demonstrations Ayala Katam Farmer/Participant in trials or demonstrations Cala Jima Farmer/Participant in trials or demonstrations Gada Talita Farmer/Participant in trials or demonstrations Gabare Benya Farmer/Participant in trials or demonstrations Gamada Wakayo Farmer/Participant in trials or demonstrations Muhammed Hamu Farmer/Participant in trials or demonstrations Tohaye Biruu Farmer/Participant in trials or demonstrations Jamam Abdulah Farmer/Participant in trials or demonstrations 72 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 73 Country, Organization, Name Designation Abarash Tashaye Farmer/Participant in trials or demonstrations Saida Fadir Farmer/Participant in trials or demonstrations Aman Ahmed Farmer/Participant in trials or demonstrations Etenesh Kunbi Farmer/Participant in trials or demonstrations Dabe Belde Farmer/Participant in trials or demonstrations Abara Nagade Farmer/Participant in trials or demonstrations USAID Washington (for debriefing on June 6) Dr Julie Howard Deputy Coordinator for Development, Feed the Future Dr Jeff Hill Bureau of Food Security (BFS)/Country Strategy and Implementation Amber Kenny BFS/Country Strategy and Implementation Beth Dunford BFS Jennifer Maurer BFS Larry Beach BFS Woody Navin BFS/Country Strategy and Implementation John McMurdy Research and Technology BFS Susan Thompson Africa Bureau Kaarli Sundsmo USAID/EA ANNEX D: REFERENCES AND KEY LITERATURE CONSULTED ASARECA. (2005). Strategic Plan 2006-1015. ASARECA. (2007). Seed Certification Standards for Ten Selected Crops of Major Importance in Kenya, Uganda, Tanzania and Rwanda. Compiled by O. Nyachai. 19p. ASARECA Operational Plan 2008-2014. 4 November 2008. ASARECA. Agro-biodiversity and Biotechnology Programme (AGROBIO) Strategic Plan 2008-2016. ASARECA. Knowledge Management and Up-scaling Program Strategic Plan 2009-2014. ASARECA Gender Mainstreaming Strategic Plan 2009-2014, Final Draft. ASARECA. (2009). Joint Mission of Development Partners, 1 – 23 October 9, 2009. ASARECA. Annual Performance Report January-December 2010: December 6, 2010. ASARECA. Audit Report for 2010. ASARECA (ed.) (2010a). Contributing to the achievement of NEPAD’s agricultural agenda (CAADP) Pillar IV and the Millennium Development Goals: Snapshots of ASARECA success stories through regional collective action in Agricultural research for development. ASARECA. (2010b). Contributing to the achievement of NEPAD's agricultural agenda (CAADP) Pillar IV and the Millennium Development Goals - Snapshots of ASARECA Success Stories, Entebbe, Uganda. ASARECA. (2010c). Joint Mission of Development Partners, 3-22 May 2010. ASARECA. (2011a). Joint Mission of Development Partners -Final Report, January 2011. ASARECA. (2011b). Assessing the Impact of USAID funded Projects under Agro-Biodiversity and Biotechnology Program. ASARECA. (2011c). Opio, F., Oyema, D., Ivan, R. The ASARECA's Staple Crops - Progress and Achievements -Power Point Presentation. ASARECA. (2011d). Improving the participation of weaker NARS in ASARECA activities. J. Methu, Ndikumana, J. and Waithaka, M. March 2011. ASARECA. (2011e), The Agri-Forum Newsletter, Nov 2010 – April 2011. ASARECA. (2011f). Integrated Management of Cassava Brown Streak Disease and Cassava Mosaic Disease to Enhance Productivity and Utilization of Cassava in East and Central Africa. Update to ASARECA/USAID Review Mission on Activities in Uganda. ASTI/IFPRI. (2010). Agricultural Science and Technology Indicators - Recent Developments in Agricultural Research, Country Notes: Kenya June 2010;Uganda July 2010, Tanzania, October 2010. EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 75 ASTI/AWARD Fact Sheet - Women's Participation in Agricultural Research and Higher Education, Uganda, Kenya, December 2008. Blackie, M., Blackie, R., Lele, U. and Beintema, N. (2010). Capacity Development and Investment in Agricultural R&D in Africa. Lead Background Paper, Ministerial Conference on Higher Education in Agriculture in Africa, Munyonyo, Kampala, Uganda, 15-19 November 2010. Bua, A., Acola, G., Adupa, R. L., Otim-Nape, G. W., Baguma, Y. K., Sserunkuma, D., Manyong, V. and Coulibaly, O. (2007). The economic impact of investments in cassava research in Uganda. In: Impact of science on African agriculture and food security. Anandajayasekeram, P., Rukuni, M., Babu, S., Liebenberg, F. and Keswani, C. L. (editors). Pp. 147-151. CGIAR Science Council. (2006). Impact Assessment of Policy-Oriented Research in the CGIAR: A Scoping Study Report. CGIAR Science Council Secretariat: Rome, Italy. CIMMYT (2010). Ethiopia study on biofortified maize reveals tasty results. CIMMYT E-Newsletter. www.cimmyt.org/ COMESA. (2010). Status of Biotechnology Policies and Biosafety Legislation in the COMESA Region – Revised. M. Karembu, D. Wafula, M. Waithaka and G. Belay. 4p. COMESA. (2010). The RABESA Project 2004-2010: Achievements and Future Prospects. D. Wafula, M. Karembu, M. Waithaka and G. Belay. 4p. DONATA. (2011). Enhanced uptake and adoption of orange-fleshed sweet potato (OFSP) technologies in Ethiopia, Kenya, Rwanda, Tanzania, and Uganda. July 2008-December 2012. Briefing to USAID Review Mission 27 May 2011. EAC Gazette. (2010). Declaration of East African Standards. Legal Notice no. 22. Vol. AT. – 1. No. 007. Arusha. 16th July 2010. Eastern Africa Farmers Federation Brochure -The Secretariat, Nairobi Kenya. FARA. (2006). Framework for African Agricultural Productivity (FAAP) - Forum for Agricultural Research in Africa Secretariat, June 2006, Accra, Ghana. Feed The Future East Africa FY 2010 Implementation Plan - U.S. Government Working Document Feed The Future Rwanda FY 2010 Implementation Plan - U.S. Government Working Document FARA. (2011). CAADP Pillar IV Strategy and Operational Plan (2011-2013). Gildemacher, P. et al. (2007). Select the Best - Positive Selection to Improve Farm Saved Seed Potatoes - Trainers Manual - International Potato Center, Lima Peru, 2007. HarvestPlus (2010). www.harvestplus.org Investing in a Knowledge and Information Base for African Agriculture - A Workshop for CAADP Pillar IV Institutions and Development Partners - Summary Workshop Report 16 February 2011, Zurich, Switzerland. 76 EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA EVALUATION OF USAID/EAST AFRICA SUPPORT TO ASARECA 77 Kaguongo, W., P. Gildemacher, P. Demo, W. Wagoire, P. Kinyae, J. Adrade, G. Forbes, G. Thiele and K. Fuglie. (2007 unpublished draft). Farmer practices: Adoption of CIP-derived potato varieties and economic impact in Kenya and Uganda. International Potato Center. Kalyebara, R. and R. Buruchara. (2008). The impact of new bean technologies on rural livelihoods in seven African countries. CIAT in Africa: Highlights No. 44. CIAT and ECABREN. Kenya Agricultural Research Institute. Strategic Plan 2009-2014, June 2009, Nairobi, Kenya. Kiambi, D. (2011). Assessing the Impact of USAID Funded Projects under the Agro-biodiversity and Biotechnology Program of ASARECA. 50p. Memorandum of Understanding between the Association for Strengthening Agricultural Research in Eastern and Central Africa (ASARECA) and the Development Partners (DPs) Supporting ASARECA to Implement Its Strategic and Operational Plans. 2009. 12p. NRI. (2007). End of Programme Review of ASARECA's Networks, Projects and Programmes - Final Report, September 2007. Natural Resources Institute, Kent, United Kingdom. Omamo, W. et al. (2006). Strengthening Strategic Priorities for Agricultural Development in Eastern and Central Africa. IFPRI Research Report #150. IFPRI and ASARECA. Washington, DC. Spielman, D.J. and Pandya-Lorch, R. (2009). “Fifty years of progress.” In Millions Fed: Proven Successes in Agricultural Development. David J. Spielman and Rajul Pandya-Lorch, eds. IFPRI.) Status of Biotechnology Policies and Biosafety Legislation in COMESA Region. COMESA, September 2010. Waithaka, M., Nzuma, J., Kyotalimye, M and Nyachae, O. (2011). Impacts of an improved seed policy environment in Eastern and Central Africa. ASARECA. World Bank. (2007) World Development Report 2008: Agriculture for Development. http://www.asareca.org/agrobio/ http://www.asareca.org/staples/ http://www.asareca.org/tuusi/index.php?option=com_tuusiviewbestpractice&Itemid=42&act=search