STRATEGIES AND ANALYSES FOR GROWTH AND ACCESS (SAGA) MID-TERM EVALUATION REPORT JANUARY 2006 This publication was produced for review by the United States Agency for International Development. It was prepared by Dirck Stryker, Associates for International Resources and Development and Jacqueline Vavra, Management Systems International. STRATEGIES AND ANALYSES FOR GROWTH AND ACCESS (SAGA) MID-TERM EVALUATION REPORT Contracted under AEP-I-00-00-00024-00, Task Order No. 15 Management Systems International Corporate Offices 600 Water Street, SW Washington, DC 20024 DISCLAIMER International Deve lopment or the United States Government. SAGA MID-TERM EVALUATION REPORT i CONTENTS LIST OF ACROYNMS USED...................................................................III 1.0 EXECUTIVE SUMMARY......................................................................1 1.1 Overview......................................................................................1 1.2 Key Findings, Conclusions, and Recommendations.........................4 2.0 PROGRAM BACKGROUND & DESIGN............................................14 2.1 Historical Context ....................................................................... 14 2.2 Strategic Framework................................................................... 15 2.3 Management Structure................................................................. 15 2.4 Contractual Mechanisms for Implementation................................ 15 3.0 EVALUATION PURPOSE, METHODOLOGY AND TEAM COMPOSITION.............................................................................16 3.1 Purpose ...................................................................................... 16 3.2 Methodology .............................................................................. 16 3.3 Team Composition...................................................................... 18 3.4 Problems Encountered During the Evaluation ............................... 19 3.5 Acknowledgements..................................................................... 19 4.0 IMPLEMENTATION...........................................................................20 4.1 African Economic Research Consortium (AERC) ......................... 20 4.2 Cornell University Consortium .................................................... 26 4.3 IDRC/SISERA............................................................................ 30 4.4 International Lawyers and Economists Against Poverty (ILEAP).................................................................................. 37 5.0 PROGRAM MANAGEMENT AND PERFORMANCE MONITORING..............................................................................42 5.1 Highlights of Program Management............................................. 42 5.2 Program Management Problems Encountered............................... 42 5.3 Performance Monitoring.............................................................. 44 6.0 PRINCIPAL FINDINGS, CONCLUSIONS AND RECOMMENDATIONS................................................................46 6.1 Building Capacity through Training & Research – AERC.............. 46 6.2 Collaborative Research – Cornell ................................................. 48 6.3 Strengthening African Economic Research Institutes..................... 48 6.4 Trade Negotiations – ILEAP........................................................ 49 6.5 The Role of Economic Research in Influencing Policy................... 51 6.6 Gap Between Researchers & Mid-Level Policy Analysts............. 52 6.7 Networking................................................................................. 53 6.8 Raising SAGA’s Profile .............................................................. 53 6.9 Reduction in USAID Funding of SAGA ....................................... 54 REFERENCES...........................................................................................57 ANNEX A: EVALUATION SCOPE OF WORK........................................66 SAGA MID-TERM EVALUATION REPORT ii ANNEX B: LIST OF PERSONS INTERVIEWED.....................................71 ANNEX C: INTERVIEW PROTOCOLS...................................................78 ANNEX D: WEB-BASED SURVEYS.........................................................85 SAGA MID-TERM EVALUATION REPORT iii LIST OF ACROYNMS USED ACBF African Capacity Building Foundation AERC African Economic Research Consortium AIRD Associates for International Resources and Development ATRIP Africa Trade and Investment Program CEPA Centre for Policy Analysis CODESRIA Council for the Development of Social Science Research in Africa CMAP Collaborative Masters Program CPP Collaborative PhD Program CREA Centre de Recherche en Economie Appliquée DFID Department for International Development DPRU Development Policy Research Unit EAGER Equity and Growth through Economic Research EPA Economic Partnership Agreements EPRC Economic Policy Research Centre FTA Free Trade Agreements IDRC International Development Research Centre ILEAP International Lawyers and Economists Against Poverty INSTAT Institut National de la Statistique IPAR Institute of Policy Analysis and Research ISSER Institute of Statistical, Social and Economic Research JFE Joint Faculty Elective KIPPRA Kenya Institute for Public Policy Research and Analysis MSI Management Systems International PIDLMI Poverty, Income Distribution and Labor Markets PTCI Programme de Troisième Cycle Interuniversitaire RFA Request for Application SAGA MID-TERM EVALUATION REPORT iv SAGA Strategies and Analyses for Growth and Access SISERA Secretariat for Institutional Support for Economic Growth in Africa SSA Sub-Saharan Africa USAID United States Agency for International Development WB The World Bank WTO World Trade Organization SAGA MID-TERM EVALUATION REPORT 1 1.0 EXECUTIVE SUMMARY 1.1 OVERVIEW 1.1.1 PROGRAM BACKGROUND AND DESIGN The United States Agency for International Development has been a leader in supporting the buildup of research institutions and educational programs that increase the supply of highly skilled economists able to devise locally suitable solutions to the myriad of problems that African countries face. The Equity and Growth through Economic Research (EAGER) project, the predecessor to the Strategies and Analyses for Growth and Access (SAGA) project, was designed to support collaborative research between senior African researchers and U.S. researchers, as well as strong dissemination and outreach to influence policy change. With the ending of the EAGER project in FY2001, SAGA continued to emphasize the goals of EAGER, but with increased emphasis on capacity building and local African ownership. More of the resources under SAGA were to go to African institutions than was the case under EAGER. In addition, in contrast to the EAGER project, which emphasized growth and trade with equity, relatively more attention was to be paid in the SAGA project to poverty and access by the poor to the means for reducing their poverty. The result is a five year (2001-2006), $16 million project with the overall goal of increasing the economic capacity of Africans to produce high quality, policy-oriented research on key issues affecting economic growth and access in Sub-Saharan Africa. In support of this goal, SAGA has four main objectives: 1) strengthen selected African economic research institutes; 2) expand the pool of highly trained African economists; 3) conduct policy-oriented research on economic growth and access issues; and 4) facilitate linkages between U.S. and African researchers. During the conception of the SAGA project, the Africa Bureau’s Office of Sustainable Development, Strategic Analysis Division (AFR/SD/SA) operated under “Strategic Objective 14: Adoption of Strategies, Programs, and Activities for Accelerated, Sustainable, and Equitable Economic Growth.” This strategic objective had two components: · Develop strategies, policies, and activities to increase trade and investment, mobilize domestic resources, and liberalize key markets; and · Strengthen African capacity to design, advocate, and manage strategies, policies, and activities for accelerated, sustainable, and equitable growth. With this objective in mind, SAGA was designed as an umbrella program that supports several African economic capacity building initiatives. These are complementary and mutually supportive in nature. One such initiative, led by the African Economic Research Consortium (AERC), based in Nairobi, Kenya, supports training programs for African economists at the Masters and PhD levels, as well as their continued growth within a professional research environment. In order to enhance the quality of economic policy research, SAGA also supports a US-African collaborative economic research program funded through a cooperative agreement with Cornell University. Since economic research has a greater impact on policy if it is undertaken by teams of researchers working out of well-established research centers, SAGA also supports the Secretariat for Institutional Support for Economic Research in Africa (SISERA), which provides funding and managerial assistance to African research centers. These were the three original pillars of SAGA. A fourth initiative was added as result of the increased need for SAGA MID-TERM EVALUATION REPORT 2 specialized research and training related to multilateral, regional, and bilateral trade negotiations. This was achieved through support for International Lawyers and Economists Against Poverty (ILEAP). The USAID office providing technical oversight to the SAGA implementing partners was to be the Africa Bureau’s Office of Sustainable Development, Strategic Analysis Division (AFR/SD/SA). The SAGA Activity Manager initially was Rita Aggarwal and the Cognizant Technical Officer (CTO) was Yoon J. Lee. CTO and Activity Manager responsibility for the Cornell portion of the cooperative agreement was transferred to Don Sillers and Borany Penh, respectively, on the PASSN team at the end of FY05. These Washington DC-based USAID officers had primary responsibility for monitoring the progress of the implementing partners and overseeing the technical aspects of the agreements. USAID awarded grant and cooperative agreements to the selected organizations for implementation of the project, with Cornell’s award being based on a competitive RFA. 1.1.2 EVALUATION PURPOSE, METHODOLOGY, AND TEAM COMPOSITION The purpose of the mid-term evaluation of the SAGA program is (a) to review the project design documents and work plans for all three implementers and one sub-contracting implementing organization, providing a summary of each implementer’s objectives in order to establish a benchmark of what the program was designed to achieve and how it was to contribute to the overall objective; (b) to determine whether the program(s) of each implementer is meeting the stated objectives, documenting the major successes/shortfalls of each program since the inception of SAGA and indicating what results are likely to be achieved by the completion of the SAGA program; (c) to provide recommendations for improving performance and the documenting/dissemination of this performance in the remaining year of the SAGA activity; and (d) to develop an options paper to assist EGAT/EG in considering possible alternatives in the area of economic capacity building after SAGA expires in 20061 . Each implementer has a number of different activities and functions under its overall program, i.e. research, training, technical assistance, and institutional strengthening. Each of these sub-components was examined with respect to how it builds researchers’ capacity to conduct research, how successful the program is at promoting networking, how successful the program is at influencing policy, and how successful it is at building institutional capacity and generally contributing to the overall program objective. Currently, one of the SAGA implementers, SISERA, is “winding down”, leaving only two implementers and one sub-contracting organization to complete the SAGA program. The objective of the evaluation related to providing recommendations for increasing its impact over the rest of the project applies, therefore, only to the activities of AERC, Cornell University, and ILEAP. The basic methodology used for the evaluation consisted of first, an examination of key program documents from the project implementers: Cornell University, AERC, SISERA, and the AERC sub￾contractor, ILEAP. Of central importance were the evaluation of SISERA, the mid-term and final evaluations of Phase V of AERC, and the review of ILEAP. Subsequently, key informant interviews were conducted with USAID staff, project implementers, SISERA Partner Institutions, researchers, policy makers, and alumni of the AERC Master’s program to assess how well the SAGA program has worked in practice, the problems encountered, the success that has been achieved, and the lessons learned. A web￾based survey questionnaire, available in both English and French, was also sent to directors of Partner Institutions, researchers funded through AERC’s research program, a sample of alumni of the Collaborative MA Program (CMAP), and ILEAP workshop participants to gather information from people in countries the evaluation team was not able to visit. 1 Cornell’s cooperative agreement has been extended to 2007. SAGA MID-TERM EVALUATION REPORT 3 While the evaluation offers some recommendations for future programming following the end of the SAGA program, including what aspects of the program are still needed for African capacity building, what components need modification, and what USAID’s unique role is in comparison to other donors, the options paper, which is to be prepared separately, will provide more detailed information regarding the extent to which the goals and objectives of SAGA are applicable to other developing regions, the best practices that can be distilled and used for replication, and how such activities will fit into the new strategic objectives of State/USAID joint strategic planning. The evaluation team comprised Dirck Stryker, President and Chief Economist of Associates for International Resources and Development (AIRD), and Jackie Vavra, Social Science Analyst for Management Systems International. 1.1.3 MAJOR PROBLEMS HINDERING IMPLEMENTATION While there was minimal formal structure within the SAGA project for consultation and oversight, this does not appear to have caused any significant problems regarding day-to-day management of the project. On the other hand, this structure also meant that it was difficult to make mid-course corrections in any coordinated way. For example, as it became obvious that SISERA was seriously understaffed and unable to perform all the tasks assigned to it, a more tightly knit management structure might have been able to compensate for this by having other organizations fill in. The major program management problem was related to the transfer of USAID’s management of the SAGA project from the Africa Bureau to the Economic Growth office of the Economic Growth, Agriculture, and Trade (EGAT) Bureau approximately two years after the project began. Funding for the project remained with the Africa Bureau. In contrast to the Africa Bureau, EGAT/EG was less interested in building capacity in Africa than in producing policy-relevant conclusions from the research that would have global applicability. Increased emphasis was placed on responding to the EGAT Bureau’s programmatic interests, which involved a focus on trade and growth more than a range of poverty issues. While this was consistent with the overall orientation of AERC, ILEAP, and SISERA, it differed from Cornell’s cooperative agreement and the AERC collaborative research project that USAID was funding, which were strongly focused on poverty. However, funding from EGAT’s Poverty Analysis and Social Safety Nets (PASSN) team beginning in FY04 and their management of the Cornell portion of the cooperative agreement beginning in FY05 has refocused at least the Cornell portion on poverty reduction research and on building local capacity to conduct this research. This separation of lines of responsibility also meant that the Africa Bureau had less ownership over the management process and coordination with various Agency partners. Severe cuts in EG funds in FY 04 and FY05 meant that many Missions and activities, including SAGA, experienced significant reductions. Despite this and due to the Activity Manager’s efforts to increase awareness of SAGA within various USAID/W offices, funding for SAGA in FY04 increased by one million dollars from what was originally anticipated, with additional funding requested and provided from the EGAT/Poverty office, EGAT/EG, and the Africa Bureau’s Initiative to End Hunger in Africa (IEHA). Nevertheless, there were substantial delays in knowing how much funding would be available for the implementers. In Fiscal Year 2004, Cornell did not know how much funding it would receive until the last week of the fiscal year. In FY05, due to significant confusion over how funding for EG activities was to be handled within the Africa Bureau, SAGA was drastically cut without notification to the Activity Manager or CTO until fairly late in the funding cycle In FY 2005, Cornell received limited funding from the Africa Bureau - $50,000 from AFR/DP and $20,000 from USAID/Madagascar. Most of its funding, $400,000, came from EGAT/PASSN. This is in comparison with Cornell’s average annual obligation of $1.2 million budgeted in its cooperative agreement. As of February 2006, Cornell had received obligations totaling $3,983,000, SAGA MID-TERM EVALUATION REPORT 4 in comparison with the $5,900,000 budgeted in its cooperative agreement, and was scheduled to receive only $400,000 more for FY 2006 from EGAT/PASSN. Although AERC as of FY 2005 had received obligations totaling $3,720,000, which were designed to cover the original grant agreement for $3,600,000 plus overhead on its management of ILEAP’s sub￾grants, the size of its annual obligations was substantially decreased and there appeared to be little likelihood of renewed funding in the near future. The ILEAP sub-grant, which is funded through EGAT/EG, had its funding maintained at the budgeted level and is likely to receive additional funding in FY 2006. SISERA, which is winding down its activities, was not expected to and did not receive any FY2005 obligation. The decision to wind down SISERA was made by the International Development Research Center (IDRC) and was not related to USAID funding. 1.2 KEY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS This section provides an overview of key findings, conclusions, and recommendations from the evaluation. Details in support of the findings are contained principally in Section 4 on project implementation. The recommendations of the evaluation refer primarily to the remaining duration of the project. The options paper will provide recommendations that go beyond the SAGA project and will indicate the extent to which their impact will be felt in the short, medium, and longer term. 1.2.1 BUILDING CAPACITY THROUGH TRAINING & RESERCH – AERC Findings AERC has administered a Collaborative Masters Program (CMAP) since 1988. A group of African universities throughout the sub-continent participate in the program, simultaneously offering students a core curriculum in the first year of the program. During the second year, students come physically together for the Joint Faculty Elective (JFE), which is guided by a common pool of local and international expertise. The CMAP is generally acknowledged to be the highest quality advanced program in economics in Africa. The Joint Faculty Elective (JFE) is given particularly high marks by all who have been associated with it. Beneficiaries overwhelmingly replied that the program was instrumental in preparing them for careers in the field. However, some of the partic ipating universities do not have the staff to sufficiently implement the CMAP. This has important implications for the ability of national universities to take over financial responsibility for the CMAP program. AERC support for PhD studies in the past has led to an increasing number of graduates holding key positions in government ministries and central banks. Many of these graduates are involved in the CMAP as lecturers of core and elective courses, student supervisors, external examiners, and members of liaison committees. However, the quality of the PhD programs in Africa varies, and programs overseas are not necessarily well-attuned to African issues. As a result, AERC has recently initiated a Collaborative PhD Program (CPP). This program has proven to be very popular. There is a general consensus that the CPP is a rigorous and relevant program with dedicated people. The CPP has the potential to help fill the critical gap of PhD faculty members in economics departments and other institutions in SSA. However, the program needs to be streamlined. There is considerable variation from university to university. For the moment, the CPP program is still quite narrowly circumscribed geographically. The program also needs more money since it is very difficult for students to complete the dissertation in one year. More advanced training occurs through the thematic research grants, semi-annual workshops, and collaborative research projects. The peer review process at AERC workshops is seen to be an avenue to meet colleagues from other institutions, gain new insights, and receive valuable feedback on research that SAGA MID-TERM EVALUATION REPORT 5 is instrumental in helping younger researchers develop their skills. Peer resource persons from other institutions, in both Africa and elsewhere, provide valuable feedback on research. This is instrumental in helping young researchers improve their research capabilities. The research program also builds capacity to conduct research by teaming up more senior researchers with younger ones. This is especially true of the collaborative research projects. There is a general perception that the quality of research has substantially improved since the beginning of the program. Nevertheless, there is a need in the thematic research, after completing one or two research projects, to move beyond mastering new methodologies and techniques to developing the skills needed to identify and analyze specific issues that are important in Africa. In addition, there is a need for more funding to publish research. Although AERC has made substantial progress in extending its activities to the smaller, poorer countries of Africa, there is still more to do. Thematic research tends to be too focused on a handful of countries. This is even truer of the collaborative research projects, which tend to be run by “old boy” networks in a handful of countries. Future cutbacks in funding of the CMAP and CPP by USAID will have a very adverse effect on these programs and especially on the ability to extend them more widely throughout Africa. There is a general perception that this will reflect badly on USAID. Conclusions The general view of AERC is that it is a premier organization, that its graduates have gone on to assume key roles in government and elsewhere, and that this has had a fundamentally important influence on the quality of analysis available to and used by policy makers. If these trends continue and are strengthened, the quality of policy decisions all over Africa will be vastly improved. The major need is to extend AERC’s activities into the smaller, poorer countries of Africa. This will expand their pool of professionally qualified economists and strengthen their capacity for sound decision￾making. The problem with doing this is that it will draw resources away from the countries that have benefited from these activities in the past. This is being rationalized by an attempt on the part of AERC to have national universities take over much of the cost of the CMAP. Whether they are willing and able to do this, given their own precarious financial situation, is an open question. This implies that the donors may have to increase their contributions to these programs if the objective of extending them to cover the continent of Africa is to be achieved. USAID has been supporting AERC for over a decade, and the question can be asked as to whether it should not now be self-sustaining. The answer is that it really is not the type of institution that should be self-sustaining as long as the countries of Africa are as poor as they are and the university systems lack the resources that that they do. AERC has stepped in to fill a void that exists in the national systems. This is essential to provide the African expertise that can assist in extracting African countries from their poverty. Once this is done and they are able to support their own university systems, there will be no further need for AERC, at least not on the relative scale that exists today. Recommendations AERC’s activities should be extended more to smaller, poorer countries. There is very strong case for donors providing additional support to these activities, especially those that increase the base of qualified economists in Sub-Saharan Africa, such as the CMAP and CPP programs. USAID should renew its commitment in this respect The CPP should be harmonized to a greater extent across countries. The possibility of video conferencing should be explored. More funding should be allocated to CPP thesis research. SAGA MID-TERM EVALUATION REPORT 6 More attention needs to be paid in thematic research to topics that are important for policy in Africa. This can be done by providing training in the skills needed to identify and analyze issues of importance. Collaborative research projects also need to be oriented in this direction. Research needs to be more integrally linked to relevant policy interests. In addition to having more impact, it could potentially reach a wider audience if more funding could be made available for some form of publication of research results, even as working papers. In this instance, the possibility of establishing an economics journal or book series should be explored. Procedures for managing the collaborative research projects should be established and administered in a transparent manner. 1.2.2 COLLABORATIVE RESEARCH – CORNELL Findings The overall consensus is that the quality of research undertaken by Cornell in collaboration with African researchers and Partner Institutions (national policy institutes) has been very high and has contributed to substantial capacity building within the research community. Exchange of scholars, workshops, study tours, and other activities have been highly appreciated by African researchers. Cornell has also played an important role in helping to build capacity within the Partner Institutions to prepare proposals, obtain funding for research projects, manage those projects, and assure the maintenance of high quality standards, though this capacity building has been concentrated on a few of Africa’s premier research centers, namely in Uganda, Ghana, Madagascar, S. Africa, Kenya, and Senegal. There is a major need to extend this type of collaboration to other research centers that are less advanced. Clark Atlanta University administers a small grants program that provides opportunities for researchers, including PhD students and faculty from US universities, to develop partnerships with African researchers and research institutions by financing their collaborative research in Africa. See Section 4.2.2 for more details. The small grants program funds some useful research and enables researchers to gain experience in Africa, but the benefits are minimal. The researchers typically do not work with anyone while they are in-country, and they are there for a very short time. Conclusions The impact of Cornell’s research and capacity building efforts has been highly positive. The overall quality and relevance for policy of economic research in Africa has improved significantly in the last 15 or so years, and Cornell has played an important role in this process. Nevertheless, Cornell’s efforts have been highly targeted at a very few institutions and countries within Sub-Saharan Africa. This needs to be extended. Recommendations Consideration should be given to having more expatriate researchers visit African research centers more often, staying for longer periods of time, and working with a broader range of people, including policy makers and stakeholders. Consideration should also be given to having African researchers make short visits to a number of universities in the US so that they can broaden their horizons. A few changes need to be made to the Small Grants Program. Collaboration should start at the design stage and should involve the Partner Institutions more in the elaboration of the proposals sent to them as part of the selection process. The budget for this activity should be increased to allow for collaborative research between local and visiting researchers. SAGA MID-TERM EVALUATION REPORT 7 1.2.3 STRENGTHENING AFRICAN ECONOMIC RESEARCH INSTITUTES Findings SAGA has had mixed results in achieving its goal of strengthening selected African economic research institutes. SISERA’s core grants to its Partner Institutions have contributed substantially to this goal and purchased valuable breathing space for the centers that are losing African Capacity Building Foundation (ACBF) support. However, this has only involved eight research centers. Competitive research grants have been much smaller than the core funding, which has been less useful for some centers. Most important has been the failure of SISERA to reach out more effectively to the Emerging Centers to build their capacity. The reasons for this are twofold. First, the seed grants have been too small and, second, SISERA does not have the staff resources to stay in effective contact with these centers and supply them with technical and managerial assistance in addition to its frequent visits to the Partner Institutions to oversee their work. This is critically important if the objective of strengthening African economic research institutes is to extend to the smaller, poorer countries of Africa, which are most in need. Conclusions Whether it is SISERA or another institution, such as ACBF, there is a clear need for an African institution dedicated to building the capacity of African research centers to undertake research that is used by policy makers in their decisions. It has been shown that these research centers are the most effective means of reaching policy makers on a sustained basis, but many of them have major financial, technical, and managerial needs. SISERA has made some progress in meeting these needs, but its legal situation has been too precarious and its staff too insufficient to effectively carry out its mission. It is important to note that USAID took the initiative to offer some funding for the SISERA institutions to determine what type of post-SISERA structure could be created. Developing economic research capacity in Africa will require a substantial expansion of funding and technical assistance. The rewards are very great, however. The involvement of research centers in policy making in the most advanced African countries, such Ghana, Kenya, and South Africa, and the measurable improvements in policy making that have resulted, clearly point the way to what can be achieved elsewhere. Recommendations In the short run, over the rest of the life of the project, action taken with respect to strengthening African economic research institutes depends on what happens with SISERA. If SISERA continues, under a new institutional umbrella, such as that of Council for the Development of Social Science Research in Africa (CODESRIA), USAID should resume the financial support that it was offering earlier. At the same time, it should push for the appointment of a permanent Executive Director and an increase in staff size, so that SISERA is capable of carrying out its mandate. Over the longer run, regardless of what happens to SISERA, there is a need for increased donor support of an institution devoted to strengthening the African economic research centers. This is vital for creating a professional environment for decision-making based on sound economics and for establishing a capacity to influence those decisions with sound policy–relevant research. 1.2.4 TRADE NEGOTIATIONS – ILEAP Findings Respondents almost universally felt that the support they have received and the work that ILEAP is conducting is both relevant and policy oriented. Workshop participants overwhelmingly responded that their training has helped enhance their trade negotiation skills. ILEAP is seen as willing and able to SAGA MID-TERM EVALUATION REPORT 8 stimulate open discussion of sensitive policy issues and to support work that acknowledges differences in national or sub-regional interests. Nevertheless, the number of people who participate in ILEAP workshops represents a small group of stakeholders, though the issues covered are quite broad. As a result, there may not be enough people present to discuss adequately all of the areas covered. Furthermore, because ILEAP’s program is largely demand-driven, and because that demand tends to be better and more forcefully expressed by the more advanced African countries, there is a tendency not to focus on the problems of the poorer countries. ILEAP has to date supported its extensive activities with a very meager organization and administrative staff. There is serious risk of overextension. Any future expansion of activities, or even continuing the current level of activities, will require deeper levels of organization and more staff. ILEAP also needs to have a more permanent presence in the major areas it serves. ILEAP’s legal orientation is underdeveloped in relation to its concentration of economics. Conclusions Research and workshops conducted by ILEAP are highly relevant and able to bring together a wide range of stakeholders around trade issues. If USAID funding were to stop, there would be a significant gap in African’s knowledge of these vitally important areas such as World Trade Organization (WTO), Economic Partnership Agreements (EPAs) and Free Trade Agreements (FTAs), and regional trade integration. Indeed, there is a need to go further in supporting a wider range of perspectives, including those of the smaller, poorer countries of Sub-Saharan Africa. ILEAP appears to be moving in this direction with its recent focus on potential conflicts between trade and development. Given the scale of its activities and the small size of the staff overseeing them, it is inevitable that the quality of the papers commissioned by ILEAP will vary. However, the use made of these papers and the dissemination of their findings through workshops and other means is generally balanced and judicious. ILEAP has played a key role in focusing on those issues that are important for African participation in multilateral trade negotiations. It has examined those issues in an unbiased way from several different perspectives, leaving it to the countries to decide how they want to proceed. Much of the work that ILEAP does is controversial in that it supports the African side in international trade negotiations. This is very much in keeping with the commitment made by the developed countries at Doha to build capacity in the developing countries to participate in the global economy. USAID funding of ILEAP is focused on trade in services, which is an area in which the developing nations are generally expected to make concessions in exchange for concessions by the developed countries on agriculture, non-market market access, and other areas. ILEAP has commissioned research on trade in services and has held at least one major workshop devoted to this area. However, its Board has gone well beyond the concessions to be made by African countries. It recently approved a Strategic Plan over the next three years that focuses on other areas in which it is expected that it will be the developed countries that will make most of the concessions. This is entirely consistent with its mandate and with the sprit of the Doha Round. Recommendations Wider representation is needed at the workshops. There is also a need to incorporate the perspectives of the smaller, poorer countries of Sub-Saharan Africa into the ILEAP program. ILEAP needs a substantial expansion of its staff in order to carry out its mission. Consideration should be given to ways of further institutionalizing the acquisition of knowledge and skills regarding trade negotiations. More attention to the legal dimensions of trade negotiations is also required. ILEAP should consider establishing a presence in the recipient countries. SAGA MID-TERM EVALUATION REPORT 9 1.2.5 THE ROLE OF ECONOMIC RESEARCH IN INFLUENCING POLICY Findings The SISERA strategy of allowing the Partner Institutions to build linkages with policy makers has had considerable success. Some research centers have been very effective in influencing policy. Often this is because of longstanding personal ties of the director and/or a few senior researchers, or their ability to identify important emerging themes and to focus the work of the center on these themes. Research conducted by established, credible researchers and institutions is also more likely to be used by policy makers. On the other hand, in many African countries, such research centers either do not exist or have not yet attained the status and experience necessary to command the respect of policy makers. Policy makers emphasize that when they are involved with researchers in setting up the research agenda at the beginning of a project, this ensures the research is on target and that they will use it to implement their programs. Demand driven research in core areas is more likely to influence policy. A substantial amount of the research conducted by Cornell in collaboration with the Partner Institutions, was not demand driven, was conducted within smaller projects, and had fewer coherent clusters in core research areas. As a result, it had a less significant impact and a lower profile than it could have had. Researchers have not generally explored programmatic implications of research findings. This leaves a gap between research results and their implications for policy. Exploring these programmatic dimensions in terms of alternative scenarios would assist policy makers to incorporate the research results into their decision-making. High-level workshops that bring public officials, researchers, and development partners together to discuss relevant economic research have been seen as very successful in Ghana, Kenya, Senegal, South Africa, and Uganda to discuss research and influence policy makers. The ability of research to influence policy makers in other countries has been less successful. One reason may be that the capacity of policy makers to make use of the research, especially in smaller, poorer countries, may be very limited. This calls for careful consideration of the areas in which policy research will be useful in defining research programs. In the larger, better off countries, where policy makers are more likely to be better trained in economics, more sophisticated research can be conducted in support of policy decisions, but in smaller, poorer countries, the level of the analysis, and especially its presentation to policy makers, should be less demanding. Conclusions The SAGA project supports the basic need that exists in SSA to develop local capacity for economic analysis in support of policy decisions. Historically this analysis has been undertaken at five different levels: · Government departments. These are notoriously weak because of low salaries and pressing day-to day demands. However, increasingly, senior leaders are becoming aware of the importance of having good research input into policy. · Academic researchers. These are somewhat stronger professionally, but salaries are low and the demands of the universities for teaching are very high. · Consulting firms. Quality varies considerably, but these firms are generally called on to undertake studies with considerable time pressure and not much depth. · Expatriate researchers. Some are very good, have considerable field experience, and have a very useful international perspective. However, they are expensive and often lack the local knowledge required for good policy research. In addition, they do not contribute to sustainability. · Research centers. Some research centers, especially those associated with universities, date back to the 1960s or early 1970s. Others were created starting in the 1990s, often with support from the ACBF. SAGA MID-TERM EVALUATION REPORT 10 Many of these have become quite useful resources for policy input. The ones that are extensions of government or publicly owned, such as KIPPRA in Kenya, tend to be quite closely linked to policy makers. The challenge is how to combine these different types of institutions effectively in support of policy. AERC’s program concentrates on strengthening of university programs and research undertaken by individual researchers regardless of where they are located. SAGA has chosen to support this effort but also to concentrate on the research centers. Recommendations SAGA should play a more active role in promoting the use of research by policy makers. For example, the implementing institutions might support a workshop devoted to best practice techniques for ensuring that research is used by policy makers. Case studies should be prepared and presented. Programmatic implications of research findings need to be explored in ways that reduce the gap between research findings and policy decisions. Research agendas should be developed in collaboration with policy makers. If policy makers are not involved or consulted in the initial decision-making process regarding research, it is much more difficult for the research to influence policy. Consideration should be given to how policy makers can be further involved in the research process. High-level workshops, such as AERC’s Senior Policy Seminars, that bring together public officials, development partners, and the press, should be continued. After discussion and debate, the results should be published and made widely available. Policy papers should be produced that provide policy makers with the information they need in a non￾technical yet substantive way. They should also be produced in a timely manner, not long after the work has been completed. Dissemination of research results to donors can be an important way to influence policy by ensuring that the results are incorporated into the design and implementation of programs or projects that are funded with foreign assistance. However, it is important to avoid the model in which donors identify policy makers’ research needs, fund the research, and ensure that the results of the research are used through conditionality. 1.2.6 GAP BETWEEN RESEARCHERS AND MID-LEVEL POLICY ANALYSTS Findings Middle-level policy analysts, who are supposed to be the conduit and promoters of policy-related research, often lack the capacity to engage in these discussions. In addition, at times research is overly academic and technical and is not written in a way that is user friendly or policy-oriented. Conclusions There is a need to bridge the gap between researchers and middle -level policy analysts. Research should be targeted towards influencing policy and should not be written in technical language that is overly academic. More interaction between researchers and policy analysts should be promoted. Consideration should be given to building the capacity of middle-level policy analysts for understanding economic research. Recommendations Consider ways to bridge the gap between researchers and policy makers. One way might be to fund researchers to spend time in public service studying how government departments operate. This not only will enable them to understand government better but also will help them see how to target research to SAGA MID-TERM EVALUATION REPORT 11 public institutions and produce materials that are not overly academic and technical. It will also help researchers gain access to policy makers. Forge professional links between research centers and government policy units, enabling middle -level policy analysts to consult on a regular basis with outside researchers and to farm out research projects when needed. At the same time, researchers will become more aware of the real constraints on policy makers and how these can be incorporated into their research. Develop methodologies for producing materials that present research results in a user friendly way and allow policy makers to use simulation techniques to examine alternative options. Conduct capacity building workshops for researchers that teach them how to produce these materials and present them to policy makers and analysts. 1.2.7 NETWORKING Findings One goal of SAGA is to support networking among researchers and research centers. The most successful networking is a product of AERC’s activities. The CMAP, CPP, thematic research workshops, collaborative research projects, and Senior Policy Seminar all serve to provide networking opportunities for participants. SISERA has been less successful in its networking activities, partly because it has not had the resources to maintain close contact with its Emerging Centers. Internet linkages have not worked as well as had been hoped. However, SISERA’s Directors’ meetings and researcher workshops have provided some opportunities for networking. ILEAP workshops have also contributed to networking by bringing together those with interest in or responsibility for international and regional trade negotiations. Conclusions Networking is important. It provides for the exchange of ideas and experience. Some of the most important advances in policy have been achieved because of the le ssons learned from other countries. The internet is providing new opportunities for exchange of information, but personal contact is still the most important medium for networking. Recommendations Continue to support networking opportunities between researchers and research centers through workshops, seminars, and collaborative research projects. Consider programming options that would strengthen regional networks either through SISERA, if it continues under a new institutional umbrella, or through ACBF. 1.2.8 RAISING SAGA’S PROFILE Findings One of the most commonly noted themes that researchers, policy makers, and others mentioned was that they had never heard of the SAGA project and did not know it was funded through USAID. Among the implementing institutions, SISERA has a particularly low profile. Although some of Cornell’s researchers have kept USAID missions informed, this has not uniformly been the case. Conclusions Mission staff is generally interested in the work that is being conducted under the SAGA project. While considerable synergy can come from centrally funded projects such as SAGA, the real issue is less about funding and more about communication and control over implementation. Involving Mission staff in project design and implementation allows them to buy into the project and creates a constituency in SAGA MID-TERM EVALUATION REPORT 12 Washington. In addition, much more needs to be done to publicize the SAGA project and what it is trying to do. Recommendations Mission staff should be involved in the design and implementation of SAGA research. Implementers should maintain consistent contact with Mission staff members, updating them on the work they are doing and informing them about activities that are taking place. Mission staff members should receive copies of documents and reports that are produced for the project. Consideration should be given to organizing an annual conference that targets a number of countries and focuses on a specific topic area. One might be the current state of international trade negotiations involving WTO, EPAs with the European Union, and new customs unions such as ECOWAS, and what this means for economic growth and poverty reduction in Sub-Saharan Africa. The goal would be to highlight the work that SAGA is doing and its relevance for the key issues Africa is facing. 1.2.9 REDUCTION IN USAID FUNDING OF SAGA Findings Due to budget constraints, USAID has been unable to follow through on the terms of the cooperative agreement with Cornell. Many of Cornell’s research and training activities have had to be curtailed and no new activities have been started. USAID also has been supporting AERC for more than a decade with the expectation that this funding would continue. So, when further funding was disrupted, this was particularly disheartening. USAID has been contributing about 13 percent of AERC’s total budget, and the curtailment of this contribution is likely to severely impact the CMAP, CPP and collaborative research programs. Since many other donors are participating in the financing of AERC and are convinced of the importance of its activities, this put USAID in a very bad light. Conclusions USAID’s curtailment of funding for SAGA has severely impacted the attainment of the two components of Strategic Objective 14:2 · Develop strategies, policies, and activities to increase trade and investment, mobilize domestic resources, and liberalize key markets; and · Strengthen African capacity to design, advocate, and manage strategies, policies, and activities for accelerated, sustainable, and equitable growth. This Strategic Objective was the cornerstone for the SAGA project, which was developed recognizing the strong comparative advantage of the US in economic research and teaching related to international development and the building of research institutions that support policy reform. AERC is widely recognized as the premier African institution supporting policy-relevant economic research and training. Although other donors also provide support, the US has played a critical role in shaping the directions in which AERC has advanced with respect to lines of research supported, its links with policy making, and soundness of the graduate-level training that is being undertaken. USAID’s ability to influence the directions in which this institution will move in the future, however, will be seriously compromised by lack of financial support. This will also have repercussions at the national level on the quality of 2 The issue of whether this has also jeopardized the Agency’s new Strategic Framework for Africa will be examined in the options paper. SAGA MID-TERM EVALUATION REPORT 13 researchers and research institutions, which are vital to improving the analytical basis for policy decisions. SAGA’s support for collaborative research between American and African researchers recognizes the fact that the most important research on international development has for some time been undertaken in the US. This has served as the intelle ctual basis for market-oriented, trade-based policy reform, with due consideration of the impact of this reform on poverty. Collaborative research not only produces important findings that feed into the policy process but also provides a critical mechanism for transferring knowledge regarding analytical techniques and research methodologies to African researchers. In this way it contributes enormously to the sustainability and African ownership of the policy reform process. Research cannot be undertaken in a vacuum. Experience has shown that research centers, which bring senior and junior researchers together in collaboration, not only ensure that the research has the depth and substance required for sound policy decisions but also provide the most effective vehicle for transmitting research findings to policy makers. Although a number of such centers have attained premier status in the larger, more advanced African countries, there is an urgent need to support the development of similar centers in the smaller, poorer nations. Whether this is to be done through SISERA or another similar organization, the need for such support exists if policy decisions are to be based on sound empirical analysis. Recommendations While funding by USAID for the types of activities supported by SAGA is likely to be quite limited, it is nevertheless important that some reasonable level of funding be maintained. Only in this way will USAID be able to bring the comparative advantage of the US in economic policy research to bear on the directions in which such research is pursued in Africa and how it is linked with policy decisions. The highest priority for the duration of SAGA is to maintain at least the existing annual level of support to Cornell ($400,000) and to increase the annual level of support to AERC from $200,000 to at least $600,000. Given Cornell’s diverse research areas, including education and HIV/AIDS, it would be appropriate for USAID to consider funding sources outside of economic growth funds for this portion of the cooperative agreement. This would enable the collaborative research to continue, albeit at a reduced level, and would contribute something to the CMAP and CPP programs, which are vital to further upgrading of the economics profession. The current annual level of support to ILEAP of about $500,000 should also be continued, given the urgency posed by ongoing negotiations regarding regional integration (e.g., implementation of the common external tariff in ECOWAS) and economic partnership agreements, principally with the EU. Although some reference has been made in this report to the need to strengthen the linkages between research and policy making, this should not be taken to suggest that SAGA has failed in this respect. Both SAGA and its predecessor project, EAGER, have contributed substantially to improving these linkages. The situation is much better than it was ten years ago in the countries in which these projects have operated. But Africa is vast and there is an urgent need to extend this success to other countries where sound policies are needed based on solid analysis. It is also important to recognize that this evaluation report only goes part of the way towards indicating directions for future funding. The Options Paper to be produced subsequently will examine these directions in much greater detail. What is recommended here is essentially only bridging until these options can be fully spelled out and a longer term strategy decided upon. SAGA MID-TERM EVALUATION REPORT 14 2.0 PROGRAM BACKGROUND & DESIGN 2.1 HISTORICAL CONTEXT Despite some positive signs of economic growth for many African countries during the latter half of the 1990s, there is widespread agreement that overall growth in Sub-Saharan Africa (SSA) still remains weak and fragile, and that even in relation to other developing regions, Africa lags behind on a number of major fronts. Furthermore, severe economic crises in the 1970s and 1980s caused many of the premier universities in the region to become financially weakened and to lose some of their best faculty. Thus, the pool of locally trained personnel available to work on economic reform issues was seriously eroded and African countries remained heavily reliant on foreign expertise. During the 1990s, The United States Agency for International Development was a forerunner in supporting the buildup of research institutions and educational programs that increased the supply of highly skilled economists able to devise locally suitable solutions to the myriad of problems that African countries faced. Through the Equity and Growth through Economic Research (EAGER) activity, USAID provided strong support for the African Economic Research Consortium (AERC), which provided a professional environment for young African economists, and funded policy-relevant research undertaken by teams of African and expatriate researchers. The EAGER program goals were to: · Conduct policy analyses relevant to the needs of the host countries, as well as to those of USAID and other donor agencies, and disseminate the results of these analyses; · Provide capacity building for African economic training, research, policy, and management institutions; and · Supply analytical guidance for the technical assistance implemented through the Africa Trade and Investment Program (ATRIP). EAGER was designed on the principle of supporting collaborative research between senior African researchers and U.S. researchers, together with strong dissemination and outreach to policy makers. The program focused on supporting timely, policy-oriented research, and was successful in influencing policy changes in approximately a dozen African countries. With the ending of the EAGER project in FY2001, the Strategies and Analyses for Growth and Access (SAGA) project continued to emphasize the goals of EAGER, but with increased emphasis on capacity building and local African ownership. An EAGER external evaluation conducted in 1999 and other reviews cited many of the accomplishments noted above, but emphasized that there was a need to place greater emphasis on African led research, build capacity of local economic research institutes, and increase support for higher education programs on the continent. Accordingly, more of the resources under SAGA were to go to African institutions than was the case under EAGER.3 In addition, in contrast to the EAGER project, which emphasized growth and trade with equity, relatively more attention was to be paid in the SAGA project to poverty and access by the poor to the means for reducing their poverty. The result was a five year (2001-2006), $16 million project with the overall goal of increasing the economic capacity of Africans to produce high quality, policy-oriented research on key issues affecting 3 Under EAGER, approximately one third of the resources went to the Africans, whereas in the SAGA program approximately two thirds of the resources are directed to African institutes. SAGA MID-TERM EVALUATION REPORT 15 economic growth and access in Sub-Saharan Africa. In support of this goal, SAGA had four main objectives: 1) strengthen selected African economic research institutes; 2) expand the pool of highly trained African economists; 3) conduct policy-oriented research on economic growth and access issues; and 4) facilitate linkages between U.S. and African researchers. SAGA is an umbrella program that supports several African economic capacity building activities. These are complementary and mutually supportive in nature. One such activity, led by the African Economic Research Consortium (AERC), based in Nairobi, Kenya, supports training programs for African economists at the Masters and PhD levels, as well as their continued growth within a professional research environment. In order to enhance the quality of economic policy research, SAGA also supports a U.S.-African collaborative economic research program funded through a cooperative agreement with Cornell University. Since economic research has a greater impact on policy if it is undertaken by teams of researchers working out of well-established research centers, SAGA supports the Secretariat for Institutional Support for Economic Research in Africa (SISERA), which provides funding and managerial assistance to African research centers. These were the three original pillars of SAGA. A fourth was added as result of the increased need for specialized research and training related to multilateral, regional, and bilateral trade negotiations. This was achieved through support for International Lawyers and Economists Against Poverty (ILEAP) through a sub-contract agreement with AERC. 2.2 STRATEGIC FRAMEWORK During the conception of the SAGA project, the Africa Bureau’s Office of Sustainable Development, Strategic Analysis Division (AFR/SD/SA) operated under “Strategic Objective 14: Adoption of Strategies, Programs, and Activities for Accelerated, Sustainable, and Equitable Economic Growth.” This strategic objective had two components: · Develop strategies, policies, and activities to increase trade and investment, mobilize domestic resources, and liberalize key markets; and · Strengthen African capacity to design, advocate, and manage strategies, policies, and activities for accelerated, sustainable, and equitable growth. 2.3 MANAGEMENT STRUCTURE The USAID office providing technical oversight to the SAGA implementing partners was to be the Africa Bureau’s Office of Sustainable Development, Strategic Analysis Division (AFR/SD/SA). The SAGA Activity Manager was Rita Aggarwal and the Cognizant Technical Officer (CTO) was Yoon J. Lee. CTO and Activity Manager responsibility for the Cornell portion of the cooperative agreement was transferred to Don Sillers and Borany Penh, respectively, on the PASSN team at the end of FY05. These Washington DC-based USAID officers had primary responsibility for monitoring the progress of the implementing partners and overseeing the technical aspects of the agreements. 2.4 CONTRACTUAL MECHANISMS FOR IMPLEMENTATION USAID/AFR/SD/SA issued the primary mechanism to solicit proposals from U.S. institutions to implement the five-year SAGA activity. This mechanism was a Request for Application (RFA) and was issued on May 24, 2001. U.S. organizations had to abide by the submission standards outlined in the RFA and submit separate technical and cost applications by June 25, 2001. USAID awarded grant and cooperative agreements to the selected organizations. It is worthy to note that, in general, grant and cooperative agreements require less intensive management on the part of USAID than contracts. SAGA MID-TERM EVALUATION REPORT 16 3.0 EVALUATION PURPOSE, METHODOLOGY AND TEAM COMPOSITION 3.1 PURPOSE The purpose of the mid-term evaluation of the SAGA program, an economic capacity building program focused on Sub-Saharan Africa, is (a) to review the project design documents and work plans for all three implementers and one sub-contracting implementing organization, providing a summary of each implementer’s objectives in order to establish a benchmark of what the program was designed to achieve and how it was to contribute to the overall objective; (b) to determine whether the program(s) of each implementer is meeting the stated objectives, documenting the major successes/shortfalls of each program since the inception of SAGA, and indicating what results are likely to be achieved by the completion of the SAGA program; (c) to provide recommendations for improving performance and the documenting/dissemination of this performance in the remaining year of the SAGA activity; and (d) to develop an options paper to assist EGAT/EG in considering possible alternatives in the area of economic capacity building after SAGA expires in 20064 . The Scope of Work for the evaluation is found in Annex A. Each implementer has a number of different activities and functions under its overall program, i.e. research, training, technical assistance, and institutional strengthening. Each of these sub-components was examined with respect to how it builds researchers’ capacity to conduct research, how successful the program is at promoting networking, how successful the program is at influencing policy, and how successful it is at building institutional capacity and generally contributing to the overall program objective. Currently, one of the SAGA implementers, SISERA, is “winding down”, leaving only two implementers and one sub-contracting organization to complete the SAGA program. The objective of the evaluation related to providing recommendations for increasing its impact over the rest of the project applies, therefore, only to the activities of AERC, Cornell University, and ILEAP. While the evaluation offers some recommendations for future programming following the end of the SAGA program, including what aspects of the program are still needed for African capacity building, what components need modification, and what USAID’s unique role is in comparison to other donors, the options paper, which is to be prepared separately, will provide more detailed information regarding the extent to which the goals and objectives of SAGA are applicable to other developing regions, the best practices that can be distilled and used for replication, and how such activities will fit into the new strategic objectives of State/USAID joint strategic planning. 3.2 METHODOLOGY Institutional strengthening and capacity building efforts such as those undertaken by SAGA are typically very difficult to capture by quantitative indicators. Accordingly, the evaluation team employed a number of evaluation techniques to collect data. 4 Cornell’s cooperative agreement has been extended to 2007. SAGA MID-TERM EVALUATION REPORT 17 The basic methodology used for the evaluation consisted of: · Examination of key program documents from the project implementers: Cornell University, AERC, SISERA, and ILEAP. These documents include annual work plans and budgets, semi-annual and annual progress reports, program evaluations, contracts, monitoring and evaluation plans, and research papers, as well as other relevant documents on the AERC, International Development Research Centre (IDRC), and Cornell University/SAGA websites. Of central importance were the evaluation of SISERA (Yeo, Hathie, and Mendizabal, 2004), the mid-term and final evaluations of Phase V of AERC (Bevan and Yansane, 2003; Wuyts, 2004; Hassan and Rempel, 2005), and the review of ILEAP (Hoffman, 2005). Key informant interviews were conducted with USAID staff, project implementers, Partner Institutions, researchers, policy makers, and alumni of the AERC Master’s program to assess how well the SAGA program has worked in practice, the problems encountered, the success that has been achieved, and the lessons that have been learned. All interviews were conducted one-on-one. Interview protocols listing questions specific for each identified group were developed and used during interviews with key informants. Since the two-member evaluation team (see Section 3.3: Team Composition) traveled to separate countries concurrently to conduct the evaluation, developing and following the interview protocols was especially important to maintain consistency when gathering information. Annex B is a List of Persons Interviewed. Annex C includes the interview protocols that the evaluation team members used when conducting interviews with informants. During the evaluation, interviews were conducted with the following: · Program implementers and researchers at each of the implementing institutions, i.e. Cornell University in Ithaca, NY; SISERA in Dakar, Senegal; and AERC in Nairobi, Kenya were visited and interviewed by Dirck Stryker, a member of the evaluation team. · Directors of Partner Institutions in five countries: Ghana, Kenya, Senegal, South Africa, and Uganda were interviewed. Dirck Stryker visited Kenya and Senegal. Jackie Vavra, the second member of the evaluation team, went to Ghana, Uganda and South Africa. These five countries were chosen so as to have a reasonably diversified sample across geographical regions and sub-sectors. Additionally, each of the implementing partners carries out work in these five countries, so the evaluation team could interview a wide range of program beneficiaries across all programs. For example, interviews were conducted in these countries with policy makers as well as with researchers funded through AERC and ILEAP’s research capacity building programs and alumni of the Collaborative MA Program (CMAP). · USAID Mission staff members were interviewed to assess their level of involvement and engagement in the SAGA program as well their thoughts on possible future programming. A web-based survey was sent to directors of Partner Institutions, researchers funded through AERC’s research program, a sample of alumni of the CMAP program, and ILEAP workshop participants to gather information from people in countries the evaluation team was not able to visit. Distribution lists were taken from SISERA and Cornell for Partner Institutions. AERC provided names of CMAP alumni. Their researcher database on the AERC website was used to gather contact information for AERC researchers. Participant lists from ILEAP’s workshops in Kenya and Ghana were used to contact ILEAP program beneficiaries. Informants had the option to complete the web-based survey in either English or French depending on their language of choice. The English versions of the web-based surveys are attached in Annex D. The biggest obstacle in conducting the web-based survey was that the contact information gathered from the implementing partners, either through websites or lists, was not always up to date. As a result, many of the emails bounced back and were not received by the targeted beneficiaries. SAGA MID-TERM EVALUATION REPORT 18 Response rate to the web-based survey varied from group to group. Proportionately, ILEAP workshop participants had the highest response rate, 39%. This could be due to the fact that both workshops were conducted within the past year, so the contact information may have been more up to date and the workshop fresher in their minds. In contrast, the researcher and CMAP alumni lists were more dated. As such there was a high bounce back rate. Researchers’ response rate was 10%; CMAP alumni response rate was 26%. Time elapsed since conducting the research or participating in the MA program may have influenced response rates for these groups as well. The lowest response rate was with the research centers. Only one center out of twenty-two responded. There could be many factors responsible for this, some of which are discussed in this report. Emails sent Emails bounced back Surveys complete % Completed of those contacted Researchers 134 56 8 10% CMAP Alumni 76 22 22 26% ILEAP Participants 36 5 12 39% Partner Institutions 27 5 1 .05% 3.3 TEAM COMPOSITION The Evaluation Team included Dirck Stryker, President and Chief Economist of Associates for International Resources and Development (AIRD), and Jackie Vavra, Social Science Analyst for Management Systems International. Dr. Stryker, who also acted as Team Leader, has undertaken trade policy analysis and worked on trade capacity building in the developing world for over thirty years. From 1995 to 2001, he was Chief of Party for the Trade Regimes and Growth component of the Equity and Growth through Economic Research (EAGER) project within the Africa Bureau, the predecessor of the SAGA project. He has been team leader for evaluations of projects centered on trade and investment policy, poverty reduction, and economic growth. He has worked in Africa for almost thirty years on agricultural price policy, comparative costs and incentives, monetary and exchange rate policy, and trade tax policy. Jackie Vavra provides technical assistance for Management Systems International in research and evaluation, strategic planning, and training and facilitation. She has over ten years experience in the areas of performance measurement, evaluation, training and facilitation. She has conducted a variety of evaluations and has provided training on designing and implementing performance monitoring systems. Ms. Vavra is experienced with numerous evaluation data collection methodologies including survey design, key informant interviews, focus groups, and participatory information gathering techniques. She has designed and conducted evaluations of programs and processes for governmental and non￾governmental organizations in both domestic and international settings. SAGA MID-TERM EVALUATION REPORT 19 3.4 PROBLEMS ENCOUNTERED DURING THE EVALUATION An evaluation of this nature, which involves four implementing institutions, a host of research centers, and numerous policy makers, researchers, and alumni – spread out all over the continent of Africa, as well as in Washington and Toronto – inevitably requires compromises to be made regarding the depth of analysis that is possible within the limited time and financial resources available. The team therefore relied heavily on external evaluations already undertaken of the programs of the implementing agencies, as well as extensive discussions with the major participants. There was a fair amount of cross-checking with beneficiaries of the project, but perhaps not as much as would have been the case if the evaluation had been more circumscribed. Nevertheless, it is the strong opinion of the team that its major findings and conclusions are solidly based and that the recommendations resulting from these are deserving of serious consideration. 3.5 ACKNOWLEDGEMENTS The project team is very grateful for the cooperation and assistance rendered by the implementing agencies – AERC, ILEAP, Cornell University, and SISERA – which allocated substantial staff time to providing documentation and answering questions of the team. In addition, the Partner Institutions that were visited – CREA, DPRU, EPRC, IPAR, ISSER – also provided strong support and encouragement, as did other research centers, policy makers, researchers, CMAP alumni, and other observers and participants. Finally, USAID staff in Washington and the field missions were very gracious with the time they allocated to answering questions and assisting team members. SAGA MID-TERM EVALUATION REPORT 20 4.0 IMPLEMENTATION 4.1 AFRICAN ECONOMIC RESEARCH CONSORTIUM (AERC) 4.1.1 AGREEMENT PURPOSE, PERIOD OF PERFORMANCE, AND AWARD AMOUNT USAID continued supporting AERC’s research and training activities through a five-year grant covering FY2001-2006, from September 30, 2001 through September 29, 2006. AERC’s overall goal is to increase the economic capacity of Africans to produce high quality, policy oriented research on key issues affecting economic growth and access in Sub-Saharan Africa. The grant supports Phase V of AERC’s program activities and will: · Provide continued support for core activities such as thematic research workshops which support research by junior African researchers; · Provide continued support for the Collaborative Research Project on Poverty, Income Distribution, and Labor Markets that involves senior African researchers; · Provide continued support for the Collaborative Masters Program (CMAP) that provides post graduate training at the equivalent level of a Masters Degree in Economics; · Pilot the Collaborative PhD (CPP) program that will increase the pool of African economists; · Provide one year funding for FY2001 to the AERC Collaborative Research Project on Explaining African Economic Growth Performance. The grant, number HFM-G-00-01-00133-00, commits a total amount of $5,490,000. Initially, the award amount was for $3,600,000, but was increased to provide additional assistance on World Trade Organization (WTO) capacity building. The total grant monies include $100,000 funding for FY2001 to support the AERC Economic Growth Project. As of FY05, even with the budget constraints, USAID has been able to obligate $5,100,000, nearly meeting the original program budget. The original grant budget for FY2001-2006 is below. Description Amount Administration & Support Services $357,518 Program Management $153,152 Research Program $2,832,224 External Liaison & Communications $198,326 Collaborative PhD Program $1,111,654 Collaborative MA Program $837,126 TOTAL PROGRAM $5,490,000 4.1.2 PROGRAM DESCRIPTION AERC has four principal objectives under Phase V of its program. They are: · Maintain the focus on strengthening policy-oriented economic research and promoting the development of research skills for addressing the evolving policy concerns. This capacity building activity will continue to involve peer reviews to assure the quality of the thematic research. SAGA MID-TERM EVALUATION REPORT 21 · Continue to strengthen graduate training in economics in a cost-effective manner. AERC will extend support to institutions and to individuals associated with institutions, by ensuring the high academic quality of such training, and by improving the policy content of the training. It will continue to support the Collaborative Masters Program and introduce a Collaborative PhD program. · Enhance the contribution of AERC in analyzing evolving policy concerns for Africa. This effort will include dissemination of research findings as a basis for active regional and international policy debate and dialogue. · Engender sustainability in activities pursued by AERC. Key among these are continuing efforts to enhance the professional profile and credibility of the network and strengthening the Secretariat’s information clearing roles. To achieve its objectives, AERC is taking a multi-faceted approach that includes supporting research, providing technical assistance, and providing institutional and individual support. Research · AERC administers a research program that offers junior African researchers support to undertake specific research studies. The thematic Research Program is expected to maintain a total of 32 research grants per year with the average grant size at approximately $14,000 in the first year, increasing to $14,500 in the second and third years, and to $15,000 in the last two years of the project. Topic selection will be in accordance with the four broad thematic areas: Poverty, Income Distribution, and Labor Market Issues; Trade, Regional Integration, and Sectoral Policies; Macroeconomic Policy, Stabilization, and Growth; and Finance, Resource Mobilization, and Investment. · AERC will continue to administer the Collaborative Research Project on Poverty, Income Distribution, and Labor Markets (PIDLMI). The PIDLMI provides senior African researchers with the opportunity to participate in the sub-continent wide poverty network that AERC created through this project. It is expected that a number of framework papers will be produced related to each of the new sub-themes under Phase V. In addition to the framework papers, approximately 15 research proposals from teams of African researchers will be funded through this activity. Three types of proposals were to be funded: those from countries not represented in the first phase, proposals for new activities from countries that were represented, and proposals from new researchers in countries already represented in Phase I. · The PIDLMI has twinning arrangements with several North American and European institutions including Cornell, Laval, Copenhagen, and Gothenburg Universities. These institutions provide technical workshops to train researchers in poverty methodologies. These arrangements are expected to continue in Phase V. · During FY 2001 only, AERC was to receive $100,000 for its collaborative research project on explaining African economic growth performance. The research was to examine a broad range of elements at both the macro and micro levels that determine African countries growth prospects. Approximately twenty-nine case studies were to be carried out under this project. Technical assistance A working group for each of the four thematic areas will provide technical guidance and oversight during bi-annual meetings. Researchers will make presentations to their respective groups and solicit feedback and commentary on their interim reports. Institutional and individual support · AERC has administered a Collaborative Masters Program since 1988. A group of African universities throughout the sub-continent participate in the program, simultaneously offering students a core curriculum that combines local and international expertise to guide students through the two-year program. Student enrolment is expected to increase from 100 to 120 graduates annually, with the proportion of female graduate students rising to 25%. In Phase V, AERC will begin phasing out its SAGA MID-TERM EVALUATION REPORT 22 financial support for CMAP. Since CMAP is now an established program, it should move toward self￾financing, with AERC exploring other means for making CMAP more sustainable. · AERC will launch a new Collaborative PhD program. The four-year program will be offered at a small group of African universities and will combine course work and a thesis, which many PhD programs in Africa do not require. The project will develop the program and the curriculum, producing graduates to meet the demand for individuals with economics doctorates. · Through its involvement with the Masters and PhD programs, AERC will also strengthen academic institutions. The academic boards for CMAP and CPP, with members drawn from participating universities, are largely responsible for the academic substance of the two programs. AERC facilitates curriculum development and joint enforcement of standards. By working with participating universities, AERC will build the capacity of participating economics departments, support library facilities, and manage the Joint Facility for Electives (JFE). 4.1.3 HIGHLIGHTS OIF PROGRAM IMPLEMENTATION Both mid-term and final evaluations of Phase V of the AERC program determined that the program was on track and that AERC is a uniquely successful organization building policy-relevant capacity in the economics profession of Sub-Saharan Africa. During Phase V, AERC adopted four strategic approaches to achieve its four aforementioned principal objectives. (i) Emphasizing more explicit responsiveness to the evolving policy context, including the gradual reorientation of Consortium activities from issues pertaining to macroeconomic questions to broader issues of poverty reduction and long-term growth. (ii) Recognizing the policy value of collaborative research while taking measures to contain the management and resource pressures arising from that activity. (iii) Enhancing the development value of AERC by re-emphasizing the Consortium’s policy role, extending outreach to francophone and other under-represented countries, strengthening and diversifying dissemination approaches, and improving gender balance in activities. (iv) Promoting greater use of information communications technology, including the internet, for effectiveness and integration into the international research community. The details of AERC’s achievements in research, training, communications and outreach, and management and administration are spelled out in its evaluations (Bevan and Yansane, 2003; Wuyts, 2004; Hassan and Rempel, 2005) and Strategic Plan for April 2005 – March 2010 (AERC, March 2005). The thematic research fell slightly behind its target of 32 grants per year, largely because one of its semi￾annual workshops had to be cancelled for political reasons. There was a shift in orientation away from macroeconomics, with much greater emphasis on poverty and trade. There was also considerable improvement in the efficiency with which research proposals and reports were handled, with a substantial reduction in average turn around time. AERC completed three collaborative research projects during Phase V: Poverty, Income Distribution, and Labor Markets (Poverty I); Managing the Transition to Less Aid Dependence in SSA; and Africa and the World Trading System. Three new collaborative research projects were also started during Phase V: African Imperatives in the New World Trade Order, Poverty, Income Distribution, and Labor Markets (Poverty II), and Explaining African Economic Growth Performance. AERC achieved considerable success in reaching out to a new constituency, almost doubling the share of francophones awarded research grants. Increased participation by women proved more difficult, however, and this goal remains a major priority. SAGA MID-TERM EVALUATION REPORT 23 More than 1000 Masters students have graduated from CMAP. The number of students was maintained close to the 120 figure envisioned for the first two years of the planning cycle. It increased to about 140 students by 2003. The JFE, which brings CMAP students from all over Africa together for joint course work, is deemed by students, employers, and economics department heads to be highly effective for the training and networking opportunities it provides. Although it was envisioned that AERC would begin phasing out its financial support for CMAP, making the CMAP more sustainable, the capacity of African universities to pick up this burden remains in doubt. AERC launched the new Collaborative PhD Program at eight universities in 2002. The CPP provided 19 scholarships in the first year of operation and 21 in the second year. The program combines course work, including a second track of the JFE, with PhD thesis support. On average, 12 thesis grants were allocated annually during the first two years. Women’s enrollment averaged 19% by the close of Phase V. 4.1.4 PROBLEMS ENCOUNTERED IN IMPLEMENTATION Research Realization of Phase V targets was hampered in Year 4 when the semi-annual workshop had to be cancelled because of heightened security concerns in Nairobi. Funding from USAID was severely curtailed in the fifth year of implementation of Phase V. This resulted in a substantial reduction in activities, including the collaborative research project on Poverty, Income Distribution, and Labor Markets (Poverty II) and expansion of the CPP program. Training One of the challenges of the CMAP and CPP programs is retaining former alumni as faculty members once they have completed the program. Alumni from the program are very marketable. Many of them leave the university setting for UNDP, IMF, UN, World Bank or other private institutions, or government positions within their countries. Universities do not have much money for research, salaries are low, and the teaching workload is very high. Many of the younger faculty members need mentoring, but there are too few people to do so. On the basis of interviews, the following problems with the CPP were identified: · The syllabi for the core courses are too ambitious. The breadth and depth is vast, and the reading list is very long. Lecturers speed through the topics and are not able to cover all of them in depth. As a result, there is a lot of pressure on both the students and the professors. The syllabus is in the process of being reformed. It is a slow process as this is a collaborative program, involving four host institutions in four countries, namely, the University of Yaounde II, Cameroon; University of Cape Town, South Africa; University of Dar es Salaam, Tanzania and the University of Ibadan, Nigeria. Faculty from all four universities must work together to get everyone on board. · Drawing up the comprehensive exam questions is very difficult. There are four professors from four host universities for each core course (Microeconomics, Macroeconomics, Quantitative Methods, plus an elective.) With an ambitious syllabus for each course, professors emphasize different topic areas within each course. When devising questions for the students, they have to ‘strike deals’ to put together exam questions that are fair for the students and still cover the material adequately. · Students need a break after courses to prepare for the comprehensive exams. However, this cuts into the PhD writing time, which has not been factored into the four-year time frame to complete the program. After four years, the funding stops, so this is potentially a big issue. · Exam results are often late. External examiners take 3-5 months to finish reading the comprehensive exams. Since readers are not paid, administrators do not have control over getting the exam results. SAGA MID-TERM EVALUATION REPORT 24 4.1.5 MAJOR SUCCESSES AND SHORTFALLS IN IMPLEMENTATION 4.1.5.1 Successes Research The peer review process that occurs at AERC workshops is seen to be an avenue to meet colleagues from other institutions, gain new insights, and receive valuable feedback on research that is instrumental in helping younger researchers to develop their skills. It is frequently said that people who have been involved in this process 2-3 times feel that they have been successful and have graduated into becoming a real professional. One beneficiary said that the program trained him to be very competitive, “If you have gone through the system, you know how to survive, write proposals, conduct research, write a paper, and network with people.” Others said it has improved their ability to win other highly competitive research fellowships. Resource persons from other institutions, mostly professional economists from Africa and elsewhere, provide valuable feedback on research that researchers are conducting. This is very useful and is instrumental in helping young researchers improve their research capabilities. The research program builds capacity to conduct research by teaming up more senior researchers with younger ones. This is especially true of the collaborative research projects There is a general perception that the quality of research has substantially improved since the beginning of the program. The collaborative research projects have addressed the demand from stakeholders and donors to strengthen AERC’s policy orientation. Policy makers are more involved in both the design and use of this research, largely because the senior researchers have established reputations in the policy-making community and are themselves often called upon for policy advice. The Senior Policy Seminar, which is undertaken annually in a different country, has proven to be an effective forum for outreach to policy makers, making them aware of AERC research and encouraging their staffs to seek out AERC research results. This has led to a need to create a network of policy makers and research economists to organize research projects and to provide points of communication between the research and policy making communities. Training The CMAP is generally acknowledged to be the highest quality advanced program in economics in Africa. The JFE is given particularly high marks by all who have been associated with it. Beneficiaries overwhelmingly replied that the program was instrumental in preparing them for careers in the field. The standards of the CMAP program are considered to be quite high, and once beneficiaries complete the program, in the words of one graduate, “You are not afraid of competition.” In Uganda, impacts from AERC programs, specifically the CMAP, made a substantial difference in the economics department at Makerere University, as nearly all the faculty in the department were trained by AERC. They still send students to the program, and there are currently 4 students in CPP, the PhD program. As a result of its association with AERC, the level of achievement of the department has increased, a common curriculum is used, and a larger number of faculty members have PhDs. AERC support for PhD studies has led to an increasing number of graduates holding key positions in government ministries and central banks. Many graduates are involved in the CMAP as lecturers of core and elective courses, student supervisors, external examiners, and members of liaison committees. SAGA MID-TERM EVALUATION REPORT 25 Initiating the CPP has proved to be very popular. More than 80 applications are received annually, of which AERC is only able to support 25 percent. 4.1.5.2 Shortfalls Research There is still a need to involve policy makers more centrally in the identification of research needs and design of research projects. AERC recognizes this and has counted on its strengthened relationship with national research centers to put it in closer touch with policy makers. Success in this respect has varied from center to center, but it has too often been ad hoc in nature. The Senior Policy Seminar has been one mechanism for overcoming this deficiency. Although some progress has been made in shortening the pre-workshop proposal review phase, it is still too long. Funds to publish research are lacking. Some very good research funded by AERC is never published due to lack of funds. There is a need in the thematic research, after the first one or two research projects, to move beyond mastering new methodologies and techniques to developing the skills needed to identify and analyze specific issues that are important to Africa. Management of the Collaborative Research projects needs to be improved. Access to these projects has been based too much on “old boy” networks. There is a need to have explicit criteria for participation in these projects and to apply these criteria in a transparent way. There is insufficient research being undertaken on regional issues such as customs integration, trade facilitation, regional regulatory structures, regional norms and standards, etc. There is a need to increase the number of African resource persons. The recent designation of senior researchers as AERC Research Fellows should help in identifying such persons. Progress has been measured too much in terms of research outputs rather than impact on policy making. More attention needs to be devoted to devising and measuring indicators of this impact. Training Although the quality of instruction at both the CMAP institution attended and the JFE is considered above average, generally, and very effective at preparing people to conduct a PhD, a few alumni feel that some lecturers are simply invited to teach because they have been affiliated with the program for a long time even if their teaching is not very effective. Some of the participating universities do not have the staff to sufficiently implement the CMAP programs. In Ghana, for example, 6o percent of the faculty in the economics department at the University of Ghana graduated from CMAP between 1996 and 2002. While this contributes to building the capacity of the staff, some of these faculty members feel that they need mentoring and additional capacity building to be able to teach at the level they consider appropriate for the department, and especially the CMAP program. There are unrealistic funding expectations in the CPP, i.e. it is very difficult to complete the dissertation in one year. Students need more time, especially given the current delays in getting back exam results. SAGA MID-TERM EVALUATION REPORT 26 4.1.5 IMPACT RESULTS It is always difficult evaluating the impact that capacity building efforts such as those of AERC have on policy making, but there does appear to be a substantial consensus within the community of professional economists in Africa that AERC is a premier organization, that its graduates have gone on to assume key roles in government, and that this has had a fundamentally important influence on the quality of analysis available to policy makers. If these trends continue and are strengthened, the quality of policy decisions all over Africa will be vastly improved. 4.2 CORNELL UNIVERSITY CONSORTIUM 4.2.1 AGREEMENT PURPOSE, PERIOD OF PERFORMANCE, AND AWARD AMOUNT Cornell University received a cooperative agreement, award number HFM-A-00-01-00132-00, to implement the SAGA program, with Clark-Atlanta University being a sub-recipient of the agreement. The period of the agreement was from September 28, 2001 to September 27, 2006, but it has been extended for one year. The total estimated amount of this award was $5,999,537. This is a cost-share agreement whereby Cornell University is supposed to contribute an additional $2,232,152. The breakdown of the original budget is as follows: Description USAID Contribution Cost Share Contribution Collaborative Research $2,418,067 $984,274 Technical Assistance $653,472 $99,479 Small Grant $1,499,755 $508,985 Modified Direct Costs $2,696,538 $1,083,753 On Campus F&A $1,300,045 $639,414 Off Campus F&A $128,199 $639,414 TOTAL PROGRAM $5,999,537 $2,232,152 The goals of the cooperative agreement are to conduct high quality, policy relevant research and to increase African capacity to produce such research on key issues affecting economic growth and improved living standards in Africa. This is in response to the four main objectives of the SAGA project, which are to: 1. Strengthen selected African economic research institutes; 2. Expand the pool of highly trained African economists; 3. Conduct policy-oriented research on economic growth and access issues; and 4. Facilitate linkages between U.S. and African researchers. The RFA called for the implementer to have three responsibilities: (1) undertake collaborative economic research focusing on one or two areas; (2) administer a small-grants program to place US-based PhD students and faculty members in selected economic research institutes to conduct field research; and (3) provide technical assistance to strengthen the research capabilities of selected African economic research institutes. SAGA MID-TERM EVALUATION REPORT 27 Cornell chose to take a ‘bottom-up’ approach to its analysis of poverty and poverty reduction. This approach starts with the capabilities of individuals, households, and communities -- specifically their productivities, vulnerabilities, institutions, and environment – to consider how development plays out at the ground level and to understand what factors keep Africa’s poor from prospering. The research program was to be developed collaboratively with the Partner Institutions, USAID missions, policy makers and other stakeholders. In its annual progress reports and work plans, Cornell has also focused on ensuring that research informs the policy process in African countries and in USAID. 4.2.2 PROGRAM DESCRIPTION Cornell’s activities are divided into three major components: Research, Technical Assistance, and Small Grants Program. Research The research component of SAGA has four broad themes: 1) schooling, education, and human capital; 2) health and nutrition; 3) risk, vulnerability, and poverty dynamics; and 4) empowerment and institutions. The aim is to better understand the economic, social, institutional, and natural constraints that keep Africa’s poor from prospering in the context of growth-oriented reforms. While some of its research activities are multi-country in nature, most are focused on working with SISERA Partner Institutions in a set of specific core countries: Ghana (Institute of Statistical, Social, and Economic Research (ISSER)), Kenya (Institute of Policy Analysis and Research (IPAR)), Madagascar (Institut National de la Statistique (INSTAT)), Senegal (Centre de Recherche en Economie Appliquée (CREA)), South Africa (Development Policy Research Unit (DPRU)), and Uganda (Economic Policy Research Center (EPRC)). These were selected in consultation with USAID missions, potential collaborating research centers, and local policy makers. Technical Assistance Cornell provides technical assistance to SISERA partner institutes on a demand-driven basis on research methods, proposal preparation, and generating science-based information for policy-making. Technical assistance includes three types of activities: formal training workshops, support to SISERA’s research competition, and support to individual SISERA institutes in proposal preparation, planning research projects, and executing those projects. An important subsidiary goal is to help raise the national and international profiles of these Partner Institutions. Small Grants Program The project provides opportunities for researchers, including PhD students and faculty from US universities, to develop partnerships with African researchers and research institutions by financing their collaborative research in Africa. This program is administered by Clark Atlanta University. For the selection process, external referees, one of whom is African, first review the grant proposals. Subsequently, a five-person selection committee makes a first cut and distributes the proposals that pass to the relevant SISERA institute to ask for comments, including the feasibility of the proposed research, travel and collaborations. Finally, the selection committee reviews these responses to select grantees. 4.2.3 HIGHLIGHTS OF PROGRAM IMPLEMENTATION Cornell University was able to accomplish a great deal under the SAGA project despite the major funding shortfalls described below. In all, 208 research papers were prepared, many of which reported findings that challenge the conventional wisdom and have important implications for policy. These deal with areas as diverse as factors explaining children’s performance in school, the relationship between education and HIV/AIDS, explanations of the risk of households falling into chronic poverty in Kenya, and the startling high prevalence of unemployment in South Africa, even among those who are educated. SAGA MID-TERM EVALUATION REPORT 28 Cornell assisted the Partner Institutions to build capacity to conduct high quality research, to raise funds for research, and to increase their national and international profiles in a number of ways. One was the twenty policy-oriented conferences and workshops that it helped to organize, which provided an opportunity for research results to be presented and for policy makers to interact with researchers in discussions about significant policy issues. Another was the joint research that was undertaken by teams from Cornell and the African Partner Institutions. For many of these projects, Cornell researchers spent time at the African institution and/or African researchers came to Cornell. The Small Grants Program has assisted the exchange of researchers between Africa and the US. As of November 2005, 26 grants have been awarded, of which 10 have gone to women. The SISERA institutions participating in this program extend well beyond the six core research centers to comprise a total of 14 research centers in 13 countries. Cornell has encouraged the dissemination of research results and their use in policy making through its workshops and conferences, its highly popular Web site, and its policy seminars for USAID/Washington and the field missions. It has also worked directly with policy makers, in some instances helping to ensure that research results are available to them. A major goal has been to foster a culture of evidence￾based policy making. 4.2.4 PROBLEMS ENCOUNTERED IN IMPLEMENTATION The most significant problem encountered in implementation was a serious delay in the disbursement of funds from USAID to Cornell and the almost complete cutoff of funds in Project Years 4 and 5. This drastically curtailed most ongoing activities and meant that no new activities that had been planned for a second phase could be undertaken (see Section 5.2). Even in the early years of the project, late disbursement of funds made planning very difficult. In Project Year 3, for example, the disbursement was not made until almost the end of the fiscal year, and Cornell did not know until the very end how much they were going to receive. As it was, the actual disbursement was about $200,000 short of what had been budgeted. Cornell was able to accomplish as much at it did because it used some of its own funds for purposes that had not been planned, and it was able to reallocate some of the resources that USAID had provided in earlier years but that Cornell had not been able to use because of a slow start to the project following the destruction of the World Trade Center on September 11, 2001. 4.2.5 MAJOR SUCCESSES AND SHORTFALLS IN IMPLEMENTATION 4.2.5.1 Successes Cornell’s efforts have been quite successful in building capacity in Partner Institutions as illustrated by the following experiences: · In the workshops that Cornell sponsored with ISSER in Ghana, EPRC in Uganda, and DPRU in South Africa, policy makers, researchers and others were brought together in an effort to have maximum policy impact. In South Africa, for example, DPRU beneficiaries felt that policy was influenced as a result of the workshops. This is in part due to the fact that DPRU’s research output is of far higher quality than it was in the past, partly because of Cornell’s influence. As a result, the Ministry of Education was aware of the research and was able to use the results in its policy decisions. · In Ghana, ISSER has had a very strong, successful, and fruitful collaborative relationship with Cornell University, particularly with Ravi Kanbur, their main contact. They had a previous link with Cornell before SAGA, but SAGA has made it easier to be a stronger partner and has strengthened their relationship. Kanbur’s regular visits have built research capacity by introducing new theories and concepts – in particular regarding poverty and social capital. ISSER staff have also learned proposal writing skills that helped them secure funding from other sources. Cornell sponsored workshops have SAGA MID-TERM EVALUATION REPORT 29 also been instrumental in building capacity. Under SAGA, during the first year of the project, ISSER received $25,000 grant from Cornell and organized a book project on poverty in Ghana. In the second year, Cornell supported a national conference, ‘Ghana at the Half Century.’ This year Cornell supported an international conference “Shared Growth in Africa”. · Steve Younger from Cornell University provided technical assistance to EPRC in Uganda. Beneficiaries at EPRC said that the main benefits they have received from the SAGA program have been a result of Steve Younger and their interaction and support they have received from him. When he visits the office, he spends time with individual researchers, answering questions they have about their research and research methodology. He has also written joint papers with 3-4 researchers and has participated in collaborative workshops. After the lapse in funding, Younger has continued to come to Uganda with funding from other projects, and because of the relationship established, he visits EPRC and meets with researchers in the name of SAGA. · DPRU receives research support from Cornell for an education project. The money is from SISERA, but Cornell provided assistance in the proposal process. Steve Younger was also involved in evaluating the proposals and provided ongoing technical assistance to DPRU in this regard. The most valuable capacity building activity for the DPRU director and two senior researchers in the unit was a recent two-week trip they took to Cornell University. They focused on research methodology skills transfer. “It was very useful to sit with David, Steve and Peter and discuss issues. The conversation on methodology was phenomenal. It was a huge skills transfer that you cannot get at a workshop.” As key Cornell personnel, Ravi Kanbur provides technical assistance to DPRU. The working relationship with him was described as exceptional. He has been instrumental in bringing in other experts for workshops. For example, through his connections, Kanbur has been able to solicit papers and bring in researchers from all over the world to attend an Africa-wide research conference that was held last year. This provides additional TA and networking opportunities. Kanbur and the director of DPRU are editing a book together on Poverty and Well-being in Post Apartheid South Africa. This would have not been possible without SAGA, even though there has been little funding of the research itself, only $20,000. Instead Kanbur often has to fund his trips with his own research funds. Without these trips, this collaborative effort would be impossible. In general, researchers said that workshops and conferences were good places to network, particularly through the Cornell network, which provides access to resource people internationally and on the continent as well. 4.2.5.2 Shortfalls Some EPRC researchers feel that research should not be confined only to micro issues and that they would like funding to conduct research in areas outside micro/growth related studies such as trade policy, regional integration, and global partnership issues. They also feel that research that links macro and micro issues, such as linking poverty reduction to aid and sustainability would be more meaningful. Yet the cooperative agreement between Cornell and USAID, which is sharply focused on poverty reduction, to a large extent drives the research agenda. While the interaction with Cornell has been instrumental in building capacity of researchers at EPRC, the funding has not been adequate to help them achieve what they would like. Additional funding would allow resource persons, e.g., Steve Younger, to stay longer and work with more people; would fund more research; and would facilitate dissemination of their work. Researchers at EPRC want more training on how to make their research less academic/technical and more reader friendly. SAGA MID-TERM EVALUATION REPORT 30 While Cornell has been very successful at building capacity at several of Africa’s premier research institutions, there is a major need to extend this type of collaboration to other research centers that are less advanced. At present, capacity building of this type is very narrowly concentrated. The small grants program administered by Clark-Atlanta University funds some useful research and enables US-based researchers to gain experience in Africa, but the benefits are minimal. For example, EPRC assists researchers who receive small grants to come to Uganda to conduct their dissertation research. They set them up with an office, email, phone, etc. Typically, the researchers do not work with anyone while they are in Uganda, and they are in the country for a very short time. They do interact, mingle, and exchange some ideas, and a few of them give seminars. ERPC receives no funding for hosting them. In Uganda, the Mission is familiar with Cornell’s work with EPRC. Steve Younger is the main contact for SAGA and is good at linking the Mission with the program and keeping them updated. He has been proactive in seeing what would be useful to the Mission and has facilitated meetings with Mission staff to see what research they want. Often the Mission does not have time to follow up, however. In South Africa, the Mission has limited interaction with Cornell. Ravi Kanbur visits the Mission at least once a year as does another colleague from Cornell. During these visits, Mission personnel are briefed about activities, though they receive more information directly from DPRU. Those interviewed said they would like additional contact and involvement because the topics are so interesting. “With additional information, we could incorporate the research into our program.” When asked how well Cornell has responded to needs of the Mission and worked with Mission staff to coordinate activities, staff members said Cornell responded moderately well, but that they would be hard pressed to identify what was been funded and how useful it was. They also have not received final copies of anything that was identified as having been produced by the CU/DPRU collaboration, but they felt the basic topics ought to be of enormous interest to the Mission. In Kenya, Chris Barrett did a very good job of keeping the mission informed regarding a second tier of studies that was receiving some support from Cornell. On the other hand, the Program Economist was unaware of work that was being carried out over several years, with Cornell’s support, analyzing a household survey as part of the collaborative research project on Poverty, Income Distribution, and Labor Markets. 4.2.6 IMPACT RESULTS The impact of Cornell’s research and capacity building efforts has been highly positive. Any number of instances can be cited of how research in which Cornell participated jointly with African researchers has been disseminated to policy makers and used to influence policy decisions. This finding, however, is limited to the results of direct interaction of researchers and policy makers. It is much more difficult to estimate the impact good research that is widely disseminated has on policy decisions taken in countries that are remote from those in which the research was undertaken. Nevertheless, it is apparent that the overall quality and relevance for policy of economic research in Africa has improved significantly in the last 15 or so years, and that Cornell has played an important role in this process. 4.3 IDRC/SISERA 4.3.1 AGREEMENT PURPOSE, PERIOD OF PERFORMANCE, AND AWARD AMOUNT USAID entered into a grant agreement with the International Development Research Centre (IDRC) in order to support the Secretariat for Institutional Support for Economic Research in Africa (SISERA). The grant number between USAID and IDRC is HFM-G-00-01-000148-00 and the period of performance is SAGA MID-TERM EVALUATION REPORT 31 September 28, 2001 through September 27, 2006. SISERA’s activities are currently in the process of coming to an end as a result of a decision by IDRC to terminate support, which was made in June 2004. There is some possibility, however, that SISERA may continue after March 2006 under the umbrella of Council for the Development of Social Science Research in Africa (CODESRIA). The purpose of this agreement is to maintain support to IDRC so that the Centre may support SISERA’s research and training activities. SISERA will advance the objective and goals of SAGA through the provision of capacity building technical support to a number of economic research institutes operating throughout Sub-Saharan Africa; strengthen the African research network; and facilitate improved dissemination of research findings. The grant provides a total amount of $4,076,000, which consists of ‘core support’ (namely institutional support grants), capacity utilization, and technical assistance. Below is the total grant budget that includes yearly breakdowns. SAGA MID-TERM EVALUATION REPORT 32 4.3.2 PROGRAM DESCRIPTION IDRC provides financial and administrative oversight while SISERA is responsible for implementing specific activities. SISERA’s main overall objective is to increase the capacity of Sub-Saharan African economic research institutions so that economic policy research informs and contributes to meaningful debate among policymakers, private sector, civil society, and the research community. Specific sub￾objectives are: · Improve working conditions and incentive systems in research centers; · Improve managerial capacity and governance structures; · Facilitate networking among research centers and research users; · Facilitate funding of commissioned research work; · Support training activities for researchers through their research centers; and · Improve the dissemination of research results. SISERA undertakes the following activities to achieve its objectives: institutional support to member economic institutes; training and technical assistance; policy-oriented economic research program; and dissemination to targeted audiences. The purpose of SISERA’s institutional support to these institutes is to strengthen their institutional capacity to design, implement, and disseminate economic research for policymakers and other appropriate end users. This support is through SISERA’s approval of at least two core grants and two seed grants per year. The core grants are to be used by the selected institutes to cover such expenses such as acquisitions of research support facilities, support for individual research projects, staff training, seminars, participation in academic meetings, and publication costs. The seed grants are to be used by “emerging institutes” of the SISERA network and will have approximately the same purpose as the core grants, but will be smaller in value. SISERA provides technical assistance to the member institutes as they have expressed need for assistance in the following areas: proposal reviews; training courses and workshops on research topics and methodologies; communication and outreach strategies; and strategies for translating research recommendations into government policy. SISERA works with the institutes to identify specific needs and then obtain the relevant services. SISERA is also expected to continue to manage funding support to the Programme de Troisième Cycle Inter-universitaire (PTCI), an MA program in economics for Francophone Africa at a level agreed upon each year by USAID. SISERA manages a policy-oriented economic research program through a solicitation process designed in consultation with members of the SISERA network and approved by USAID. As part of its work plan, SISERA prepares a document detailing how the research program will be organized. This is approved by USAID and serves as the basis for managing the proposal approval process. A centerpiece of SISERA’s mission is the production of policy-oriented economic research that will be available to end-users. SISERA is expected to ensure a constant flow of information between research centers and research users through dissemination of research outputs in an appropriate format to targeted audiences; by involving policymakers and private sector stakeholders from the planning stage in the research output conference; and by maintaining a website where current information on SISERA and partner institution activities can be found along with research methodology and results. 4.3.3 HIGHLIGHTS OF PROGRAM IMPLEMENTATION During the initial years following the creation of SISERA, the major emphasis was on the establishment of a governance structure, the design and implementation of support instruments, mobilization of SAGA MID-TERM EVALUATION REPORT 33 financial resources, and the finalization of collaborative agreements with diverse stakeholders. Specific achievements in each of the four program areas are summarized below (SISERA, Strategic Plan 2004- 2008, 2004). Grants SISERA has relationships with nineteen Partner Institutions, a significant number considering the resources and staff they have. Some of the institutions are farther along than others. Support for these institutions is intended to create the enabling conditions under they will become internationally renowned centers of excellence. Others are weaker institutes that receive seed money to help them begin to develop their research capacity. These are called Emerging Centers Since it creation, SISERA has made eight core grants totalling 2,120,851 CAD to Partner Institutions. The beneficiaries of these grants are listed in the table below. Partner Institute – Grant Beneficiary Amount in CAD Faculté des Sciences Economiques et de Gestion (FASG), Université de Yaoundé II 206,411 Economic and Social Research Foundation (ESRF), Dar es Salaam 218, 000 Institute of Statistical, Social and Economic Research (ISSER), Accra 278,000 Centre de Recherché Economiques Appliqués (CREA), Dakar 298,300 Development Policy Research Unit (DPRU), Capetown 246,900 Nigerian Institute of Social and Economic Research (NISER), Nigeria 282,400 Economic Policy Research Centre (EPRC), Kampala 299, 500 Institute of Policy Analysis and Research (IPAR), Nairobi 295,340 As of December 31, 2003, six seed grants had been made for a combined sum of 279, 821 CAD to Emerging Research Centers. A number of other grants were made including three connectivity grants, five dissemination grants, and 15 collaborative research grants. SISERA has also disbursed over 2 million CAD for training activities, including grants for the “Programme de Troisième Cycle Inter￾Universitaire (PTCI). Four research institutions (CEREG, CEDRES, CIRES and CREA) have also received institutional grants under the PTCI doctoral program to strengthen their capacity to host doctoral students. Technical support SISERA program officers have undertaken over 50 technical supervisory visits to its member institutions since its creation in 1997. The visits provide an opportunity for the Secretariat staff to assist the institutions to prepare proposals for institutional support, to improve their administrative and accounting procedures, to design work programs, to formulate governance structures, and to resolve internal conflicts. One of the main concerns reflected in SISERA’s technical support program is the need to assist centres in their efforts to improve their managerial capacity. In that regard, the Secretariat tries to be in close contact with heads of these institutions. Through questionnaires and checklists, SISERA also tries to SAGA MID-TERM EVALUATION REPORT 34 monitor the progress achieved by Partner Institutions and Emerging Centres and to alert them to areas of possible improvement in their operations. Regional Research Networks SISERA has facilitated the creation of two sub-regional networks, one for Southern and Eastern Africa known as the Southern and Eastern Africa Policy Research Network (SEAPREN) and the other for West and Central Africa West, the West and Central Africa Policy Research Network (WECAPREN). The main objective of these networks is to undertake research on policy issues of regional relevance. SEAPREN has received a grant of 378, 830 CAD and WECAPREN a grant of 325, 000 CAD for their activities. Resource expansion Since its inception, SISERA has been successful in raising about 17 million CAD. Its major donors have been USAID, IDRC, the European Commission, the Dutch government, the World Bank, the French Government, and the Canadian International Development Agency. Collaboration with donor agencies and regional African organizations SISERA has reached collaborative agreements with the African Development Bank and the United Nations Economic Commission for Africa. These wide-reaching agreements expand the programming scope of SISERA and provide it with significant leverage in its support policy. Isolated collaborative initiatives have also been undertaken with the African Capacity Building Foundation (ACBF), the African Economic Research Consortium, and the World Bank Institute. These initiatives were expected to translate into more permanent collaborative arrangements in the future. 4.3.4 PROBLEMS ENCOUNTERED IN IMPLEMENTATION From the start, SISERA sought to establish a governance structure that would be effective, supportive, and, for cost-saving reasons, relatively light. It is managed by a Steering Committee comprising six organizations -- three from Africa and three representing donor agencies. As of December 31, 2003, the personnel of the Secretariat included an Executive Director, two program officers, a coordinator of program operations, a program assistant, a financial assistant, and a secretary. The position of Executive Director has been vacant since March 2002, shortly after the SAGA project started, and is currently being temporarily filled by one of the program officers. This has contributed to a lack of dynamism in the institution, especially given the heavy load of ongoing activities. Current staff is focused on internal organization, process, and activities rather than on external activities such profile raising and attendance at international conferences and workshops. IDRC normally provides three years of operational support for one of the Secretariats that it creates. SISERA has long ago exceeded that limit. The options are (1) closing the Secretariat, (2) bringing it under the umbrella of another organization, or (3) creating a self-standing NGO. One of the reasons why the previous Executive Director left in early 2002 was uncertainty regarding SISERA’s future. For a time, an effort was made to bring SISERA under the umbrella of another international organization – the African Institute for Economic Development and Planning (IDEP), based in Dakar -- but that fell through. As a result, the decision was made by IDRC to cease its support of SISERA effective March 31, 2006. As of September 2005, there was some indication that SISERA might continue for two years beyond that under CODESRIA, and then become an NGO, but this depends very much on finding appropriate financing. SISERA has suffered in the past when access to funding has decreased because of gaps created by donor funding cycles. With SISERA’s imminent closure, IDRC is not allowing additional new funding, which is having a severe impact on the research centers. SAGA MID-TERM EVALUATION REPORT 35 4.3.5 MAJOR SUCCESSES AND SHORTFALLS IN IMPLEMENTATION 4.3.5.1 Successes Core funding is considered to be well targeted and successful. It has been very useful in buying time for Partner Institutions to get on their feet financially, especially because ACBF money is beginning to run out after three successive phases of support. Core funding has enabled the Partner Institutions not only to establish themselves as centers of research excellence but also to forge closer links with national policy makers. EPRC, for example, received over $200,000 in core funding from SISERA. This gave the center two years of institutional support, which funded a research fellow position, research expenses, and training. SISERA has also achieved some success with Emerging Centers. A good example is the African Institute for Applied Economics (AIAE) in Nigeria, which has come to play a major role in Nigerian policy￾making after only four years of existence. Some of these centers have found the opportunity to meet their peers through SISERA to be particularly valuable in reducing their isolation. Collaboration with the World Bank Institute was particularly successful in enabling SISERA to organize training for researchers in the area of poverty reduction. However, the World Bank discontinued this program. SISERA also offered methodology training workshops to young researchers that were considered valuable and provided support to individual researchers. However, the last one was held was in Uganda in 2003. Linking researchers and end users is considered as an important task for SISERA. In practice, SISERA has not played a direct role in creating these linkages at the national level, leaving this to the Partner Institutions. This seems to have worked fairly well, though experience varies from center to center. At the same time, SISERA has focused successfully on fostering linkages with global and regional institutions such as the World Bank Institute, the African Development Bank, and Cornell University. Regional networks play a useful role, especially for smaller centers. They enable researchers to tap into other sources, learn about other research being conducted, and network with others in general. Initially SISERA helped institutions get to know similar institutions; however, it has not been particularly effective at maintaining regional networks and fostering these relationships. Shortfalls ISSER felt that the annual core funding they received from SISERA was not adequate to support the organization in the way they needed and that it placed constraints on what they could do. There are several things they would do with more funding. They would try to attract other competent Ghanaians to come to ISSER to conduct research, including bringing home Ghanaians who are studying abroad to work with them. They would also like to use additional funds to bring in Americans to provide short-term technical assistance for the organization. SISERA was to provide managerial support to economic research centers through visits by the Secretariat staff. This would take advantage of the staff’s expertise in this area and the comparative experience that the network was developing. The heavy workload of the staff, however, has meant that relatively few of these visits were undertaken. SISERA has a very limited institutional profile and brand name recognition in the development policy research community, in general, and among donors and potential supporters, in particular. This is partly because the Acting Executive Director has been too overburdened with day-to-day administration to be able to devote much time to enhancing SISERA’s image. It is also partly because SISERA lacked a clear strategic direction, a problem that was not overcome until early 2004, when its Strategic Plan 2004-2005 SAGA MID-TERM EVALUATION REPORT 36 was drafted. This plan was never implemented because shortly thereafter IDRC decided to eliminate its support for SISERA. SISERA has observed that some of the Emerging Centers not only lack strong institutional structures but also have a deficiency of well-trained researchers. This relates to the problem that AERC‘s CPP program is for the moment too concentrated on a few, more advanced African countries. The workload on existing staff is such that it has been difficult for SISERA to maintain contact with the Emerging Centers, particularly those operating in difficult circumstances. The $40,000 cap on collaborative research grants discouraged a number of the better research centers from applying. For example, EPRC said they never applied because the grants were too small. On the other hand, DPRU has received a grant for a project on education. A recent trip by three researchers to Cornell was part of this program. DPRU used $20,000 that they got annually (for the past 2 years) from SISERA for conferences. Overall, beneficiaries responded that SAGA has not been instrumental in helping them network with other researchers or with other research institutions. Directors of partner institutes felt that a network of researchers had been developed under EAGER, but was not sustained under the SAGA project. The logic behind the launching of two sub-regional networks is unclear, since they do not seem to be vehicles for direct capacity building, a function that was retained by SISERA. Under SAGA, some online discussions were held, but the systems were not very effective, so researchers were not able to sustain the exchange of materials on line. In addition to the systemic challenges, the cost of maintaining email access for smaller institutions or individuals is prohibitive for some. Also, people are overstretched, particularly professors/lecturers in universities. Research conducted under SAGA is not always seen to be policy relevant. The research proposals generated by Partner Institutions under SISERA have not been demand driven and have not centered on coherent clusters; there are too many focus areas. As such, Partner Institutions have come up with research proposals that are not linked to research being conducted by other institutions. SISERA’s Web site provides basic information on the Secretariat but little concerning the research centers in the network. 4.3.6 IMPACT RESULTS It is difficult to estimate the impact of SISERA on the research centers and on policy making because there has been no systematic effort to measure these effects. However, from the comments of the research center directors, it appears that SISERA has played a useful role in supporting these institutions with core funding at a time when they have needed this support because of the withdrawal of ACBF. On the other hand, competitive research funding appears to have been less useful. Furthermore, it is not evident how important SISERA’s support has been to the Emerging Centers. The one example of success of AIAE in Nigeria is not a good one because it is difficult to distinguish the role SISERA played compared with that of the highly respected Director of this institute. SAGA MID-TERM EVALUATION REPORT 37 4.4 INTERNATIONAL LAWYERS AND ECONOMISTS AGAINST POVERTY (ILEAP) 4.4.1 AGREEMENT PURPOSE, PERIOD OF PERFORMANCE, AND AWARD AMOUNT International Lawyers and Economists Against Poverty received three sub-grants from AERC to provide support for enhanced African participation in International Trade Negotiations and Commercial Policy Formulation and Implementation. The purpose of the project is to provide analytical backstopping and capacity building to support and enhance African participation in international trade negotiations. The first period of performance was January 1- September 30, 2004. The second period of performance was January 1 – December 30, 2005 and the current one is from January 2006 to September 2006. ILEAP’s initial sub-grant was for $500,000. The second sub-grant that ILEAP received from AERC was for $470,000, and the FY 05 sub-grant was for $500,000. Each sub-grant is dispersed in three payments. AERC disburses 50% of the funds once the two organizations have a signed Memorandum of Understanding; the second disbursement of 30% is given after ILEAP submits a satisfactory interim and financial report. The remaining 20% is disbursed after the final report and financial report are submitted with original receipts and documentation. SAGA MID-TERM EVALUATION REPORT 38 Jan – September, 2004 Sub-grant Description Amount Stakeholders/Capacity Building Workshops $179,950 Analytical Support to Negotiations $200,050 Administration $120,000 TOTAL PROGRAM $500,000 Jan – December, 2005 Sub-grant Description Amount Priority Assessment Meeting $45,000 National Stakeholders’ Consensus-building meetings $40,000 Sub-regional Services Meetings $141,500 Trade in Services Training and Collaboration with the EAC Hub $20,000 On-going Support in the Development of Services Trade Positions $35,000 Update of USAID officials $1,500 Administration $187,000 TOTAL PROGRAM $470,000 The current workplan from January 2006 to September 2006 is still being finalized but will focus on trade in services and greater coordination with the Integrated Framework program. 4.4.2 PROGRAM DESCRIPTION ILEAP’s objective is to improve the ability of African countries and regions to participate more effectively in international trade negotiations and discussions by strengthening negotiating capacities, trade-related research, and analytical procedures. It will achieve this objective by: · Being responsive to national policy decision-makers’ and negotiators’ specific requests and needs; · Determining the local commercial relevance of the support and trade-related analysis provided; · Building the sustainability and expansion of African analysts’ capacity to supply timely policy support as well as adequate research and analysis relevant to negotiating needs and commercial interests; and · Providing substance and technical quality of the support, research, and analysis. ILEAP will also deliver a series of substantive demand-driven policy supports and/or research and analytical reports on individual countries’ national or regional interests and needs in relation to international trade and commercial policy issues. The activity will support international trade negotiations by African countries at the multilateral or bilateral levels. Each contribution shall include a thorough analysis of interests and options, stakeholders’ consultation, and support to the negotiations in a format agreed upon between trade policy officials of the countries and ILEAP. 4.4.3 HIGHLIGHTS OF PROGRAM IMPLEMENTATION ILEAP’s current research program focuses on issues that were identified in consultation in March 2004: agriculture, non-agricultural market access, GATS (financial services, transportation), “special products”, rules of origin, and trade facilitation. Papers on these topics were commissioned by ILEAP from mostly African scholars, mentored by senior professionals. SAGA MID-TERM EVALUATION REPORT 39 A workshop focusing on “Post-July 2004 African Strategies for Bilateral and Multilateral Trade Negotiations” was organized in Nairobi in November 2004. This workshop further refined the research program and presented the results to date of ILEAP, as well as other partner institution, research to strengthen the capacity of Eastern African trade policy makers and negotiators, as well as other stakeholders, to pursue their commercial interests without losing focus on their overall development objective of poverty alleviation. A similar workshop was held in Point Noire (Republic of Congo) in December 2004. With USAID funding, ILEAP embarked on a program of research and dialogue focused on trade in services. Activities included · Preparation of a database of reference materials for Africa-based researchers and advisors; · Preparation of issue-specific background papers commissioned by ILEAP on trade in services, with peer review of these papers; · Training and sharing of experiences regarding GATS and other areas linked with trade in services; · Organization of activities targeted at the private sector; · Preparation of country-specific background papers to assist in needs assessment exercises; · Assistance in the preparation of regional services agreements and the negotiation of positions in GATS or EPAs; and · Preparation of analytical papers to help develop negotiating positions. A capacity building workshop was held in Accra in March 2005 to assess the research priorities in services of the participating countries and to assist them in their preparation of requests and offers to WTO. National Stakeholder Consensus-Building Meetings were also held to discuss each country’s requests and offers in the area of services. In addition, a regional workshop was held in Yaounde, Cameroon to ensure adequate understanding of the current state of play in the services negotiations and to ensure coherence between trade negotia tions and other domestic policies. Finally, ILEAP collaborated with the East African Community Hub in the development and dissemination of a training module to assist trade negotiators to acquire the understanding and skills needed to participate in regional and WTO negotiations on trade in services. In preparation for negotiations concerning EU-ACP Economic Partnership Agreements, ILEAP sponsored a workshop in Antigua in April 2005. The Board of Directors approved a new three-year work program covering the period from May 2005 to April 2008 in March 2005. As part of this work program, ILEAP will assist African and Caribbean countries in identifying and prioritizing their interests and in negotiating effectively at the Hong Kong Ministerial Conference in December 2005. This will include focusing on such areas as market access, special and differential treatment, and aid for trade. ILEAP will also concentrate on trade negotiations with the EU and US as part of Economic Partnership Agreements or Free Trade Agreements, with the goal of ensuring that these agreements do not compromise development objectives such as industrialization, poverty reduction, and revenue raising in Africa and the Caribbean. ILEAP will help to ensure that coherence is maintained between commitments made at the bilateral, regional, and multilateral levels (ILEAP, 2005). As part of this program a workshop on “Trade Negotiations and Development for East Africa: EPAs and the WTO” was held in Tanzania in October 2005. 4.4.4 PROBLEMS ENCOUNTERED IN IMPLEMENTATION One particularly vexing problem has been the absence of multi-year funding for ILEAP’s activities. This has resulted in a significant portion of the Executive Director’s time being allocated to fund-raising and has made it difficult to engage in longer term planning. At the same time, its sources of funding are reasonably diverse, comprising a mix of multilateral agencies (World Bank Institute, UNDP), bilateral donors (DFID, IDRC, and USAID), and foundations (Ford, Comart), which helps to preserve some stability. However, it complicates the process of reporting and sometimes raises issues associated with restrictions imposed on the uses of funds by different donors. SAGA MID-TERM EVALUATION REPORT 40 4.4.5 MAJOR SUCCESSES AND SHORTFALLS IN IMPLEMENTATION 4.4.5.1 Successes Respondents almost universally felt that the support they have received and the work that ILEAP is conducting is both relevant and policy oriented. Economists throughout the region attended the regional ILEAP workshops in Nairobi and Accra. One of the most impressive aspects of ILEAP’s workshops was the quality of people who were involved in looking at trade issues. The research that is being conducted is deemed to by highly topical and relevant. It fills an important void at this time because many countries are getting ready for the WTO Ministerial Meetings in Hong Kong. ILEAP’s Director is seen as a highly motivated individual who seeks to keep people well-informed of ILEAP’s activities, including key USAID Mission staff. The WARP Mission in Ghana was particularly interested in what ILEAP does since the Mission supports ECOWAS in their work on tariffs. ILEAP builds capacity in a number of ways. First, it promotes interaction between African and non￾African researchers related to WTO negotia tion skills. Workshop participants overwhelming responded that the training they received has helped them enhance their trade negotiation skills. The ILEAP workshops also sensitized researchers to the international trade negotiation process, clearing up a number of misconceptions. All survey participants said that the training/assistance they received has expanded their understanding of the strategies and issues necessary to engage in trade negotiations. This was especially true for researchers in francophone countries. ILEAP is seen as willing and able to stimulate open discussion of sensitive policy issues and to support work that acknowledges differences in national or sub-regional interests. It does this by pointing out the strengths and weaknesses of different negotiating approaches, and facilitates access to information that can help to inform a party’s negotiating strategy. ILEAP appears to have had an important influence on policy. The WTO work funded by ILEAP was seen to have influenced policy in Ghana. A chief negotiator for Ghana was brought in as one of the writers of a paper, and this research was incorporated into Ghana’s trade policy. ILEAP workshops were perceived to have more policy influence since all stakeholders, including policy makers, participated in the workshops. ILEAP has established a good working relationship with the Ministry of Tourism, Trade and Industry in Uganda, which the Ministry would like to continue. They feel that ILEAP understands what their priorities are and their work has had an impact on trade policy and negotiating positions. If funding for ILEAP’s activities were not continued, the Ministry feels that it would disrupt the process that has been established. If ILEAP had a presence in the region (it is currently based in Canada), researchers and policy makers would have easier access to it. The ILEAP workshops highlight the interconnectedness within regions, i.e. if one country accepts a proposition; this affects all other countries around it. 4.4.5.2 Shortfalls Given the scale of its activities and the small size of the staff overseeing them, it is inevitable that the quality of the papers commissioned by ILEAP will vary. However, the use made of these papers and the dissemination of their findings through workshops and other means is generally balanced and judicious. ILEAP has played a key role in focusing on those issues that are important for African participation in multilateral trade negotiations. It has examined those issues in an unbiased way from several different perspectives, leaving it to the countries to decide how they want to proceed. SAGA MID-TERM EVALUATION REPORT 41 The numbers of people who participate in ILEAP workshops represent a small group of stakeholders whose expertise is rather narrow, though the issues covered are quite broad. Academic researchers, policy makers, donors, consultants, etc. attend the workshops, so there is broad range of people with technical experience, but this may not represent very well the interests of the stakeholders who are not present. As with many new organizations, ILEAP is still working to get its internal administrative processes in place and to develop long-term, sustainable financing. ILEAP has to date supported its extensive activities with a very meager organization and administrative staff. There is serious risk of overextension. As an example, there have been a number of activities that have been planned by ILEAP but have not been carried out. For instance, under its funding from USAID, a collaborative training module was supposed to be developed with the East Africa Competitiveness Hub. There were several efforts to collaborate with the Hub, which will continue, but this has not resulted in the training module output. Any future expansion of activities, or even continuing the current level of activities, will require deeper levels of organization and more staff Although there are arguments for and against being based in Canada rather than in Africa, where most of its activities take place, regardless of where it is based, ILEAP needs to have a more permanent presence in the major areas it serves. ILEAP’s legal orientation is underdeveloped in relation to its concentration on economics. Because ILEAP’s program is largely demand-driven, and because that demand tends to be better and more forcefully expressed by the more advanced African countries, there is a tendency not to focus on the problems of the poorer countries. 4.4.6 IMPACT RESULTS It appears highly likely that ILEAP will have a significant impact on the conduct of African countries regarding bilateral, regional, and multilateral trade negotiations, given the paucity of other sources of information and analysis on these issues. However, this impact is likely to be much more important for the more advanced countries. The poorer countries have very little capacity to be involved in these negotiations regardless of the training that they receive from ILEAP. Furthermore, as noted earlier, ILEAP’s analysis is more likely to be focused on the problems of the more advanced African countries than on those of the poorest. This raises the issue of whether there should not be more focus within ILEAP on the areas of greatest immediate relevance for the poorest countries. In particular, these countries should probably be involved to a greater extent in regional trade negotiations, and possibly those involving Economic Partnership Agreements (EPAs) or Free Trade Agreements (FTAs), than in negotiations at the level of the WTO. Yet very little of ILEAP’s focus is on regional integration. SAGA MID-TERM EVALUATION REPORT 42 5.0 PROGRAM MANAGEMENT AND PERFORMANCE MONITORING 5.1 HIGHLIGHTS OF PROGRAM MANAGEMENT No formal structure was established for coordinating the various components of the project, i.e., the activities of the different implementers – AERC, ILEAP, Cornell University, and SISERA -- though the project design envisioned a very close relationship between Cornell and SISERA. Shortly after the project commenced, Cornell visited SISERA in Dakar to coordinate activities and to determine with which of the Partner Institutions Cornell would work. The head of SISERA and the activity manager visited Cornell as well to initiate a close relationship. The choice ultimately was CREA (Senegal), DPRU (South Africa), EPRC (Uganda), INSTAT (Madagascar), and ISSER (Ghana). SISERA was invited to and attended annual meetings and workshops held in Ithaca to present the results of the research. Cornell also attended meetings of the SISERA research center directors and annual meetings of the researchers. In addition, Steve Younger of Cornell assisted in the evaluation of SISERA proposals, and Cornell researchers interacted to some extent with SISERA staff at various conferences and workshops. Links with AERC have been a bit more formal, though not directly related to the SAGA proje ct. David Sahns, Director of the SAGA Project at Cornell, is on the oversight committee of the collaborative research project on Poverty, Income Distribution, and Labor Markets (PIDLMI), which gives him an opportunity to interact with AERC staff on SAGA matters on an informal basis. The Executive Director of AERC is a member of the Steering Committee of SISERA, which helps to strengthen relationships between these two organizations. ILEAP is the Sub-Recipient of a sub-grant from AERC governed by a Memorandum of Understanding. Working relationships between the two organizations appear to be primarily personal, the Executive Director of ILEAP having previously been a staff member of AERC. Management meetings with USAID/Washington take place in Washington on a fairly regular basis. These generally involve only one implementer at a time. The activity manager for SAGA attends AERC and SISERA board meetings, and in the latter case, the AERC director is present as well, allowing for the opportunity of capturing program synergies between the two organizations. SISERA staff attends the Cornell annual project review meetings as do USAID representatives. Other management business is conducted by telephone or e-mail, or on the occasion of a conference, workshop, or other meeting which USAID/Washington and one or more implementing agencies attend. 5.2 PROGRAM MANAGEMENT PROBLEMS ENCOUNTERED Despite having only a minimal structure within the SAGA project for consultation and oversight, this does not appear to have caused any significant problems for day-to-day management of the project. Although communication was not perfect, respondents did not bring up any major problems. On the other hand, this structure also meant that it was difficult to make mid-course corrections in any coordinated way. For example, as it became obvious that SISERA was seriously understaffed and unable to perform all the tasks assigned to it, a more tightly knit management structure might have been able to compensate for this by having other organizations fill in. Cornell, for example, might have been given the SAGA MID-TERM EVALUATION REPORT 43 task of providing some management assistance to the research centers. However, lack of funding for Cornell made this a limited option. The major program management problem was related to the transfer of USAID’s management of the SAGA project from the Africa Bureau to the Economic Growth office of the Economic Growth, Agriculture, and Trade (EGAT) Bureau approximately two years after the project began. Funding for the project remained with the Africa Bureau. In contrast to the Africa Bureau, EGAT/EG was less interested in building capacity in Africa than in producing policy-relevant conclusions from the research that would have global applicability. Increased emphasis was placed on responding to the EGAT Bureau’s programmatic interests, which involved a focus on trade and growth more than a range of poverty issues. While this was consistent with the overall orientation of AERC, ILEAP, and SISERA, it differed from Cornell’s cooperative agreement and the AERC collaborative research project that USAID was funding, which were strongly focused on poverty. However, funding from EGAT’s Poverty Analysis and Social Safety Nets (PASSN) team beginning in FY04 and their management of the Cornell portion of the cooperative agreement beginning in FY05 has refocused at least the Cornell portion on poverty reduction research and on building local capacity to conduct this research. This separation of lines of responsibility also meant that the Africa Bureau had less ownership over the management process and coordination with various Agency partners. Severe cuts in EG funds in FY 04 and FY05 meant that many Missions and activities, including SAGA, experienced significant reductions. Despite this and due to the Activity Manager’s efforts to increase awareness of SAGA within various USAID/W offices, funding for SAGA in FY04 increased by one million dollars from what was originally anticipated, with additional funding requested and provided from the EGAT/Poverty office, EGAT/EG, and the Africa Bureau’s Initiative to End Hunger in Africa (IEHA). Nevertheless, there were substantial delays in knowing how much funding would be available for the implementers. In Fiscal Year 2004, Cornell did not know how much funding it would receive until the last week of the fiscal year. In FY05, due to significant confusion over how funding for EG activities was to be handled within the Africa Bureau, SAGA was drastically cut without notification to the Activity Manager or CTO until fairly late in the funding cycle In FY 2005, Cornell received limited funding from the Africa Bureau - $50,000 from AFR/DP and $20,000 from USAID/Madagascar. Most of its funding, $400,000, came from EGAT/PASSN. This is in comparison with Cornell’s average annual obligation of $1.2 million budgeted in its cooperative agreement. As of February 2006, Cornell had received obligations totaling $3,983,000, in comparison with the $5,900,000 budgeted in its cooperative agreement, and was scheduled to receive only $400,000 more for FY 2006 from EGAT/PASSN. Although AERC as of FY 2005 had received obligations totaling $3,720,000, which were designed to cover the original grant agreement for $3,600,000 plus overhead on its management of ILEAP’s sub￾grants, the size of its annual obligations was substantially decreased and there appeared to be little likelihood of renewed funding in the near future. The ILEAP sub-grant, which is funded through EGAT/EG, had its funding maintained at the budgeted level and is likely to receive additional funding in FY 2006. SISERA, which is winding down its activities, was not expected to and did not receive any FY2005 obligation. The decision to wind down SISERA was made by the International Development Research Center (IDRC) and was not related to USAID funding. Clearly, this failure of USAID to follow through on the terms of its cooperative agreement with Cornell has wrecked havoc with their activities. Many of the Cornell research and training activities have had to be curtailed and no new activities have been started. This also has had an impact on their ability to work collaboratively with researchers funded by AERC. Furthermore, the inability of USAID to commit further to the funding of AERC puts it in a very bad light in the eyes of other donors. USAID has been contributing about 13 percent of AERC’s total budget, and the curtailment of this contribution is likely to impact severely the CMAP, CPP and collaborative research programs. This reduces USAID’s ability to SAGA MID-TERM EVALUATION REPORT 44 influence the directions in which this institution will move in the future. It also has repercussions at the national level because of the impact that this has on researchers and research institutions, which are vital to improving the analytical basis for policy decisions. 5.3 PERFORMANCE MONITORING The USAID/AFR/SD/SA SAGA Activity Manager was to periodically review the implementing partners’ performance reports in order to monitor progress being made towards SAGA’s programmatic objectives and goals. Both the Activity Manager and the CTO were responsible for oversight while the CTO’s role also emphasized financial management. The implementing partners were required to submit quarterly, annual, and other performance reports to keep USAID apprised of programmatic activities and trends. In addition, the mid-term evaluation of the project was intended to present USAID with a set of findings and conclusions regarding the first three years of implementation and recommendations for improving the project’s implementation during the remaining two years before its completion. Cooperative agreements and grants generally call for less oversight and tracking than contracts. As a result, there were very few monitoring requirements written into the implementing partners’ agreements. On the other hand, USAID has made a strong effort to monitor implementation of the agreements, albeit usually without quantified targets. The following is what is required of each organization: · In their semi-annual reports, AERC is required to identify accomplishments in their research and training activities in accordance with the indicators they have established to monitor activities, to assess events with the potential to affect the program, and to make any recommendations for adjustments in the project work plan. AERC does have a list of performance indicators and outputs for its research and training as well as its publication, finance and human resource activities. · In Sec. 1.6 of its cooperative agreement, there is mention of the need for USAID to approve Cornell’s monitoring and evaluation plan. With encouragement from USAID/Washington, Cornell developed a series of instruments and conducted a survey of user and clients to assess internally their progress. They distinguished among four groups: USAID Missions, African research collaborators, stakeholders and conference and workshop participants, and recipients of grant awards. For the most part, responses have been positive. Cornell’s reports on performance are also folded into its program reports, which are submitted annually. Here, Cornell’s monitoring and evaluation focuses on a set of yearly quantitative indicators, such as number of publications, workshops, hits on website, small grants issued, etc. These are indicators of output rather than impact. · SISERA was supposed to generate a monitoring and evaluation plan within its initial FY2002 work plan for the activities it would undertake. This plan was to include a mix of quantitative and qualitative indicators that would allow progress to be tracked and reported on a regular basis. Ultimately, this was to be used to assess the degree to which SISERA member institutions have been strengthened. The annual and quarterly reports incorporate progress described in the written narrative, but this is not linked to any specific monitoring plan that has been identified. Although SISERA assisted the Partner Institutions to track their institutional strengthening, there were few attempts to measure the impact of SISERA’s actions on the research centers and no systematic attempt to measure the impact of these centers on the policy process. · ILEAP had no monitoring requirements built into the contract, but on USAID’s request, did compile an indicator table. It identified three indicators (number of papers commissioned, number of countries involved in meetings/workshops, and number of private sector representatives at meetings) to track progress of activities in 2003 and 2004. However, no targets for these indicators were established. SAGA MID-TERM EVALUATION REPORT 45 Although each collaborating institution has set out a number of qualitative and quantitative indicators that it is supposed to track as outputs, AERC is the only implementing partner that has identified specific targets. Thus it is difficult to know at a glance whether or not the SAGA project is on track. In addition, although there is mention in the progress reports of Cornell and SISERA of specific implementation plans for the future, there are few clear links or quantifiable indicators established to permit the monitoring of whether or not they are achieving their implementation goals. This is not to say that their implementation of the project has been unsuccessful but only that this success is difficult to track. Despite the absence, or at least near absence, of a formal M&E structure, USAID has made a strong effort to regularly monitor some of SAGA’s activities. For example, USAID encouraged, and ILEAP and Cornell positively responded to, having evaluation questionnaires submitted at their workshops in an effort to measure the quality of the discussions, something that quantifiable indicators cannot easily capture. For the most part, these evaluations were quite favorable, with at least 70 to 80 % of the responses in the positive and very positive categories. In one instance, the ILEAP workshop in Nigeria, USAID and ILEAP were dissatisfied with aspects of the planning and ILEAP subsequently made corrections in how it organized future workshops. Selected ILEAP papers were distributed amongst USAID staff internally to check on quality, again something hard to measure quantitatively. As stated elsewhere in this report, the papers varied considerably in quality, partly because so many were commissioned in a very short period of time. However, the use made of the papers in ILEAP’s workshops and other dissemination activities was balanced and judicious, as evidenced by the very high quality of the proceedings emanating from these events USAID has not followed SISERA and AERC as closely, but by attending regular board meetings and providing input at these forums (particularly in the case of SISERA), USAID has been able to monitor progress of the program and suggest corrections based on lessons learned. SAGA MID-TERM EVALUATION REPORT 46 6.0 PRINCIPAL FINDINGS, CONCLUSIONS AND RECOMMENDATIONS This section provides an overview of key findings, conclusions, and recommendations from the evaluation. Details in support of the findings are contained principally in Section 4 on implementation. The recommendations of the evaluation refer primarily to the remaining duration of the project. The options paper will provide recommendations that go beyond the SAGA project and will indicate the extent to which their impact will be felt in the short, medium, and longer term. 6.1 BUILDING CAPACITY THROUGH TRAINING & RESEARCH – AERC 6.1.1 FINDINGS AERC has administered a Collaborative Masters Program (CMAP) since 1988. A group of African universities throughout the sub-continent participate in the program, simultaneously offering students a core curriculum in the first year of the program. During the second year, students come physically together for the Joint Faculty Elective (JFE), which is guided by a common pool of local and international expertise. The CMAP is generally acknowledged to be the highest quality advanced program in economics in Africa. The JFE is given particularly high marks by all who have been associated with it. Beneficiaries overwhelmingly replied that the program was instrumental in preparing them for careers in the field. However, some of the participating universities do not have the staff to sufficiently implement the CMAP. This has important implications for the ability of national universities to take over financial responsibility for the CMAP program. AERC support for PhD studies in the past has led to an increasing number of graduates holding key positions in government ministries and central banks. Many of these graduates are involved in the CMAP as lecturers of core and elective courses, student supervisors, external examiners, and members of liaison committees. However, the quality of the PhD programs in Africa varies, and programs overseas are not necessarily well-attuned to African issues. As a result, AERC has recently initiated a Collaborative PhD Program. This program has proven to be very popular. There is a general consensus that the CPP is a rigorous and relevant program with dedicated people. The CPP has the potential to help fill the critical gap of PhD faculty members in economics departments and other institutions in SSA. However, the program needs to be streamlined. There is considerable variation from university to university. The program also needs more money since it is very difficult for students to complete the dissertation in one year. More advanced training occurs through the thematic research grants, semi-annual workshops, and collaborative research projects. The peer review process at AERC workshops is seen to be an avenue to meet colleagues from other institutions, gain new insights, and receive valuable feedback on research that is instrumental in helping younger researchers develop their skills. Peer resource persons from other institutions, in both Africa and elsewhere, provide valuable feedback on research. This is instrumental in helping young researchers improve their research capabilities. The research program also builds capacity to conduct research by teaming up more senior researchers with younger ones. This is especially true of SAGA MID-TERM EVALUATION REPORT 47 the collaborative research projects. There is a general perception that the quality of research has substantially improved since the beginning of the program. Nevertheless, there is a need in the thematic research, after the first one or two research projects, to move beyond mastering new methodologies and techniques to developing the skills needed to identify and analyze specific issues that are important in Africa. In addition, there is a need for more funding to publish research. Although AERC has made substantial progress in extending its activities to the smaller, poorer countries of Africa, there is still more to do. Thematic research tends to be too focused on a handful of countries. This is even truer of the collaborative research projects, which tend to be run by “old boy” networks in a handful of countries. For the moment, the CPP program is still quite narrowly circumscribed geographically. Cutbacks in funding of the CMAP and CPP by USAID have had a very adverse effect on these programs and especially on the ability to extend them more widely throughout Africa. There is a general perception that this reflects badly on USAID. 6.1.2 CONCLUSIONS The general view of AERC is that it is a premier organization, that its graduates have gone on to assume key roles in government and elsewhere, and that this has had a fundamentally important influence on the quality of analysis available to and used by policy makers. If these trends continue and are strengthened, the quality of policy decisions all over Africa will be vastly improved. The major need is to extend AERC’s activities into the smaller, poorer countries of Africa. This will expand their pool of professionally qualified economists and strengthen their capacity for sound decision￾making. The problem with doing this is that it will draw resources away from the countries that have benefited from these activities in the past. This is being rationalized by an attempt on the part of AERC to have national universities take over much of the cost of the CMAP. Whether they are willing and able to do this, given their own precarious financial situation, is an open question. This implies that the donors may have to increase their contributions to these programs if the objective of extending them to cover the continent of Africa is to be achieved. USAID has been supporting AERC for over a decade, and the question can be asked as to whether it should not now be self-sustaining. The answer is that it really is not the type of institution that should be self-sustaining as long as the countries of Africa are as poor as they are and the university systems lack the resources that that they do. AERC has stepped in to fill a void that exists in the national systems. This is essential to provide the African expertise that can assist in extracting African countries from their poverty. Once this is done and they are able to support their own university systems, there will be no further need for AERC, at least not on the relative scale that exists today. 6.1.3 RECOMMENDATIONS AERC’s activities should be extended more to smaller, poorer countries. There is very strong case for donors providing additional support to these activities, especially those that increase the base of qualified economists in Sub-Saharan Africa, such as the CMAP and CPP programs. USAID should renew its commitment in this respect and make up for the lapses in funding that have been experienced The CPP should be harmonized to a greater extent across countries. The possibility of video conferencing should be explored. More funding should be allocated to CPP thesis research. SAGA MID-TERM EVALUATION REPORT 48 More attention needs to be paid in thematic research to topics that are important for policy in Africa. This can be done by providing training in the skills needed to identify and analyze issues of importance. Collaborative research projects also need to be oriented in this direction. Research needs to be more integrally linked to relevant policy interests. In addition to having more impact, it could potentially reach a wider audience if more funding could be made available for some form of publication of research results, even as working papers. In this instance, the possibility of establishing an economics journal or book series should be explored. Procedures for managing the collaborative research projects should be established and administered in a transparent manner. 6.2 COLLABORATIVE RESEARCH – CORNELL 6.2.1 FINDINGS The overall consensus is that the quality of research undertaken by Cornell in collaboration with African researchers and Partner Institutions has been very high and has contributed to substantial capacity building within the research community. Exchange of scholars, workshops, study tours, and other activities have been highly appreciated by African researchers. Cornell has also played an important role in helping to build capacity within the Partner Institutions to prepare proposals, obtain funding for research projects, manage those projects, and assure the maintenance of high quality standards, though this capacity building has been concentrated on a few of Africa’s premier research centers. There is a major need to extend this type of collaboration to other research centers that are less advanced. Clark Atlanta University administers a small grants program that provides opportunities for researchers, including PhD students and faculty from US universities, to develop partnerships with African researchers and research institutions by financing their collaborative research in Africa. The small grants program funds some useful research and enables researchers to gain experience in Africa, but the benefits are minimal. Typically, the researchers do not work with anyone while they are in-country, and they are there for a very short time. 6.2.2 CONCLUSIONS The impact of Cornell’s research and capacity building efforts has been highly positive. The overall quality and relevance for policy of economic research in Africa has improved significantly in the last 15 or so years, and Cornell has played an important role in this process. Nevertheless, Cornell’s efforts have been highly targeted at a very few institutions and countries within Sub-Saharan Africa. This needs to be extended. 6.2.3 RECOMMENDATIONS Consideration should be given to having more expatriate researchers visit African research centers more often, stay for longer periods of time, and work with a broader range of people, including policy makers and stakeholders. Consideration should also be given to having African researchers make short visits to a number of universities in the US so that they can broaden their horizons. A few changes need to be made to the Small Grants Program. Collaboration should start at the design stage and should involve the Partner Institutions more in the elaboration of the proposals sent to them as part of the selection process. The budget for this activity should be increased to allow for collaborative research between local and visiting researchers. SAGA MID-TERM EVALUATION REPORT 49 6.3 STRENGTHENING AFRICAN ECONOMIC RESEARCH INSTITUTES 6.3.1 FINDINGS SAGA has had mixed results in achieving its goal of strengthening selected African economic research institutes. SISERA’s core grants to its Partner Institutions have contributed substantially to this goal and purchased valuable breathing space for the centers that are losing ACBF support. But this has only involved eight research centers. Competitive research grants have been much smaller than the core funding, which has been less useful for some centers. Most important has been the failure of SISERA to reach out more effectively to the Emerging Centers to build their capacity. The reasons for this are twofold. First, the seed grants have been too small and, second, SISERA does not have the staff resources to stay in effective contact with these centers and supply them with technical and managerial assistance. This is critically important if the objective of strengthening African economic research institutes is to extend to the smaller, poorer countries of Africa, which are most in need. 6.3.2 CONCLUSIONS Whether it is SISERA or another institution, such as ACBF, there is a clear need for an African institution dedicated to building the capacity of African research centers to undertake research that is used by policy makers in their decisions. It has been shown that these research centers are the most effective means of reaching policy makers on a sustained basis, but many of them have major financial, technical, and managerial needs. SISERA has made some progress in meeting these needs, but its legal situation has been too precarious and its staff too insufficient to effectively carry out its mission. It is important to note that USAID took the initiative to offer some funding for the SISERA institutions to determine what type of post-SISERA structure could be created. Developing economic research capacity in Africa will require a substantial expansion of funding and technical assistance. The rewards are very great, however. The involvement of research centers in policy making in the most advanced African countries, such Ghana, Kenya, and South Africa, and the measurable improvements in policy making that have resulted, clearly point the way to what can be achieved elsewhere. 6.3.3 RECOMMENDATIONS In the short run, over the rest of the life of the project, action taken with respect to strengthening African economic research institutes depends on what happens with SISERA. If SISERA continues, under a new institutional umbrella, such as that of CODESRIA, USAID should resume the financial support that it was offering earlier. At the same time, it should push for the appointment of a permanent Executive Director and an increase in staff size, so that SISERA is capable of carrying out its mandate. Over the longer run, regardless of what happens to SISERA, there is a need for increased donor support of an institution devoted to strengthening the African economic research centers. This is vital for creating a professional environment for decision-making based on sound economics and for establishing a capacity to influence those decisions with sound policy–relevant research. 6.4 TRADE NEGOTIATIONS – ILEAP 6.4.1 FINDINGS Respondents almost universally felt that the support they have received and the work that ILEAP is conducting is both relevant and policy oriented. Workshop participants overwhelmingly responded that their training has helped enhance their trade negotiation skills. ILEAP is seen as willing and able to SAGA MID-TERM EVALUATION REPORT 50 stimulate open discussion of sensitive policy issues and to support work that acknowledges differences in national or sub-regional interests. Nevertheless, the number of people who participate in ILEAP workshops represents a small group of stakeholders, though the issues covered are quite broad. As a result, there may not be enough people present to discuss adequately all of the areas covered. Furthermore, because ILEAP’s program is largely demand-driven, and because the demand tends to be better and more forcefully expressed by the more advanced African countries, there is a tendency not to focus on the problems of the poorer countries. ILEAP has to date supported its extensive activities with a very meager organization and administrative staff. There is serious risk of overextension. Any future expansion of activities, or even continuing the current level of activities, will require deeper levels of organization and more staff. ILEAP also needs to have a more permanent presence in the major areas it serves. ILEAP’s legal orientation is underdeveloped in relation to its concentration of economics. 6.4.2 CONCLUSION Research and workshops conducted by ILEAP are highly relevant and able to bring together a wide range of stakeholders around trade issues. If USAID funding were to stop, there would be a significant gap in African’s knowledge of these vitally important areas such as WTO, EPAs and FTAs, and regional trade integration. Indeed, there is a need to go further in supporting a wider range of perspectives, including those of the smaller, poorer countries of Sub-Saharan Africa. ILEAP appears to be moving in this direction with its recent focus on potential conflicts between trade and development. Given the scale of its activities and the small size of the staff overseeing them, it is inevitable that the quality of the papers commissioned by ILEAP will vary. However, the use made of these papers and the dissemination of their findings through workshops and other means is generally balanced and judicious. ILEAP has played a key role in focusing on those issues that are important for African participation in multilateral trade negotiations. It has examined those issues in an unbiased way from several different perspectives, leaving it to the countries to decide how they want to proceed. Much of the work that ILEAP does is controversial in that it supports the African side in international trade negotiations. This is very much in keeping with the commitment made by the developed countries at Doha to build capacity in the developing countries to participate in the global economy. USAID funding of ILEAP is focused on trade in services, which is an area in which the developing nations are generally expected to make concessions in exchange for concessions by the developed countries on agriculture, non-market market access, and other areas. ILEAP has commissioned research on trade in services and has held at least one major workshop devoted to this area. However, its Board has gone well beyond the concessions to be made by African countries. It recently approved a Strategic Plan over the next three years that focuses on other areas in which it is expected that it will be the developed countries that will make most of the concessions. This is entirely consistent with its mandate and with the sprit of the Doha Round. 6.4.3 RECOMMENDATIONS Wider representation is needed at the workshops. There is also a need to incorporate the perspectives of the smaller, poorer countries of Sub-Saharan Africa into the ILEAP program. ILEAP needs a substantial expansion of its staff in order to carry out its mission. Consideration should be given to ways of further institutionalizing the acquisition of knowledge and skills regarding trade negotiations. More attention to the legal dimensions of trade negotiations is also required. ILEAP should consider establishing a presence in the recipient countries. SAGA MID-TERM EVALUATION REPORT 51 6.5 THE ROLE OF ECONOMIC RESEARCH IN INFLUENCING POLICY 6.5.1 FINDINGS The SISERA strategy of allowing the Partner Institutions to build linkages with policy makers has had considerable success. Some research centers have been very effective in influencing policy. Often this is because of longstanding personal ties of the director and/or a few senior researchers, or their ability to identify important emerging themes and to focus the work of the center on these themes. Research conducted by established, credible researchers and institutions is also more likely to be used by policy makers. On the other hand, in many African countries, such research centers either do not exist or have not yet attained the status and experience necessary to command the respect of policy makers. Policy makers emphasize that when they are involved with researchers in setting up the research agenda at the beginning of a project, this ensures the research is on target and that they will use it to implement their programs. Demand driven research in core areas is more likely to influence policy. A substantial amount of the research conducted by Cornell in collaboration with the Partner Institutions, was not demand driven, was conducted within smaller projects, and had fewer coherent clusters in core research areas. As a result, it had a less significant impact and a lower profile than it could have had. Researchers have not generally explored programmatic implications of research findings. This leaves a gap between research results and their implications for policy. Exploring these programmatic dimensions in terms of alternative scenarios would assist policy makers to incorporate the research results into their decision-making. High-level workshops that bring public officials, researchers, and development partners together to discuss relevant economic research have been seen as very successful in Ghana, Kenya, Senegal, South Africa, and Uganda to discuss research and influence policy makers. The ability of research to influence policy makers in other countries has been less successful. One reason may be that the capacity of policy makers to make use of the research, especially in smaller, poorer countries, may be very limited. This calls for careful consideration of the areas in which policy research will be useful in defining research programs. In the larger, better off countries, where policy makers are more likely to be better trained in economics, more sophisticated research can be conducted in support of policy decisions, but in smaller, poorer countries, the level of the analysis, and especially its presentation to policy makers, should be less demanding. 6.5.2 CONCLUSIONS The SAGA project supports the basic need that exists in SSA to develop local capacity for economic analysis in support of policy decisions. Historically this analysis has been undertaken at five different levels: · Government departments. These are notoriously weak because of low salaries and pressing day-to day demands. However, increasingly, senior leaders are becoming aware of the importance of having good research input into policy. · Academic researchers. These are somewhat stronger professionally, but salaries are low and the demands of the universities for teaching are very high. · Consulting firms. Quality varies considerably, but these firms are generally called on to undertake studies with considerable time pressure and not much depth. · Expatriate researchers. Some are very good, have considerable field experience, and have a very useful international perspective. However, they are expensive and often lack the local knowledge required for good policy research. In addition, they do not contribute to sustainability. SAGA MID-TERM EVALUATION REPORT 52 · Research centers. Some research centers, especially those associated with universities, date back to the 1960s or early 1970s. Others were created starting in the 1990s, often with support from the ACBF. Many of these have become quite useful resources for policy input. The ones that are extensions of government or publicly owned, such as KIPPRA in Kenya, tend to be quite closely linked to policy makers. The challenge is how to combine these different types of institutions effectively in support of policy. AERC’s program concentrates on strengthening of university programs and research undertaken by individual researchers regardless of where they are located. SAGA has chosen to support this effort but also to concentrate on the research centers. 6.5.3 RECOMMENDATIONS SAGA should play a more active role in promoting the use of research by policy makers. For example, the implementing institutions might support a workshop devoted to best practice techniques for ensuring that research is used by policy makers. Case studies should be prepared and presented. Programmatic implications of research findings need to be explored in ways that reduce the gap between research findings and policy decisions. Research agendas should be developed in collaboration with policy makers. If policy makers are not involved or consulted in the initial decision-making process regarding research, it is much more difficult for the research to influence policy. Consideration should be given to how policy makers can be further involved in the research process. High-level workshops, such as AERC’s Senior Policy Seminars, that bring together public officials, development partners, and the press, should be continued. After discussion and debate, the results should be published and made widely available. Policy papers should be produced that provide policy makers with the information they need in a non￾technical yet substantive way. They should also be produced in a timely manner, not long after the work has been completed. Dissemination of research results to donors can be an important way to influence policy by ensuring that the results are incorporated into the design and implementation of programs or projects that are funded with foreign assistance. However, it is important to avoid the model in which donors identify policy makers’ research needs, fund the research, and ensure that the results of the research are used through conditionality. 6.6 GAP BETWEEN RESEARCHERS & MID-LEVEL POLICY ANALYSTS 6.6.1 FINDINGS Middle-level policy analysts, who are supposed to be the conduit and promoters of policy-related research, often lack the capacity to engage in these discussions. In addition, at times research is overly academic and technical and is not written in a way that is user friendly or policy-oriented. 6.6.2 CONCLUSIONS There is a need to bridge the gap between researchers and middle -level policy analysts. Research should be targeted towards influencing policy and should not be written in technical language that is overly academic. More interaction between researchers and policy analysts should be promoted. Consideration should be given to building capacity for understanding economic research among middle-level analysts. SAGA MID-TERM EVALUATION REPORT 53 6.6.3 RECOMMENDATIONS Consider ways to bridge the gap between researchers and policy makers. One way might be to fund researchers to spend time in public service studying how government departments operate. This not only will enable them to understand government better but also will help them see how to target research to public institutions and produce materials that are not overly academic and technical. It will also help researchers gain access to policy makers. Forge professional links between research centers and government policy units, enabling middle -level policy analysts to consult on a regular basis with outside researchers and to farm out research projects when needed. At the same time, researchers will become more aware of the real constraints on policy makers and how these can be incorporated into their research. Develop methodologies for producing materials that present research results in a user friendly way and allow policy makers to use simulation techniques to examine alternative options. Conduct capacity building workshops for researchers that teach them how to produce these materials and present them to policy makers and analysts. 6.7 NETWORKING 6.7.1 FINDINGS One goal of SAGA is to support networking among researchers and research centers. The most successful networking is a product of AERC’s activities. The CMAP, CPP, thematic research workshops, collaborative research projects, and Senior Policy Seminar all serve to provide networking opportunities for participants. SISERA has been less successful in its networking activities, partly because it has not had the resources to maintain close contact with its Emerging Centers. Internet linkages have not worked as well as had been hoped. However, SISERA’s Directors’ meetings and researcher workshops have provided some opportunities for networking. ILEAP workshops have also contributed to networking by bringing together those with interest in or responsibility for international and regional trade negotiations. 6.7.2 CONCLUSIONS Networking is important. It provides for the exchange of ideas and experience. Some of the most important advances in policy have been achieved because of the lessons learned from other countries. The internet is providing new opportunities for exchange of information, but personal contact is still the most important medium for networking. 6.7.3 RECOMMENDATIONS Continue to support networking opportunities between researchers and research centers through workshops, seminars, and collaborative research projects. Consider programming options that would strengthen regional networks either through SISERA, if it continues under a new institutional umbrella, or through ACBF. 6.8 RAISING SAGA’S PROFILE 6.8.1 FINDINGS One of the most commonly noted themes that researchers, policy makers, and others mentioned was that they had never heard of the SAGA project and did not know it was funded through USAID. Among the SAGA MID-TERM EVALUATION REPORT 54 implementing institutions, SISERA has a particularly low profile. Although some of Cornell’s researchers have kept USAID missions informed, this has not uniformly been the case. 6.8.2 CONCLUSION Mission staff is generally interested in the work that is being conducted under the SAGA project. While considerable synergy can come from centrally funded projects such as SAGA, the real issue is less about funding and more about communication and control over implementation. Involving Mission staff in project design and implementation allows them to buy into the project and creates a constituency in Washington. In addition, much more needs to be done to publicize the SAGA project and what it is trying to do. 6.8.3 RECOMMENDATIONS Mission staff should be involved in the design and implementation of SAGA research. Implementers should maintain consistent contact with Mission staff members, updating them on the work they are doing and informing them about activities that are taking place. Mission staff members should receive copies of documents and reports that are produced for the project. Consideration should be given to organizing an annual conference that targets a number of countries and focuses on a specific topic area. One might be the current state of international trade negotiations involving WTO, EPAs with the European Union, and new customs unions such as ECOWAS, and what this means for economic growth and poverty reduction in Sub-Saharan Africa. The goal would be to highlight the work that SAGA is doing and its relevance for the key issues Africa is facing. 6.9 REDUCTION IN USAID FUNDING OF SAGA 6.9.1 FINDINGS Due to budget constraints, USAID has been unable to follow through on the terms of the cooperative agreement with Cornell. Many of Cornell’s research and training activities have had to be curtailed and no new activities have been started. USAID also has been supporting AERC for more than a decade with the expectation that this funding would continue. So, when further funding was disrupted, this was particularly disheartening. USAID has been contributing about 13 percent of AERC’s total budget, and the curtailment of this contribution is likely to severely impact the CMAP, CPP and collaborative research programs. Since many other donors are participating in the financing of AERC and are convinced of the importance of its activities, this put USAID in a very bad light. 6.9.2 CONCLUSIONS USAID’s curtailment of funding for SAGA has severely impacted the attainment of the two components of Strategic Objective 14:5 · Develop strategies, policies, and activities to increase trade and investment, mobilize domestic resources, and liberalize key markets; and · Strengthen African capacity to design, advocate, and manage strategies, policies, and activities for accelerated, sustainable, and equitable growth. 5 The issue of whether this has also jeopardized the Agency’s new Strategic Framework for Africa will be examined in the options paper. SAGA MID-TERM EVALUATION REPORT 55 This Strategic Objective was the cornerstone for the SAGA project, which was developed recognizing the strong comparative advantage of the US in economic research and teaching related to international development and the building of research institutions that support policy reform. AERC is widely recognized as the premier African institution supporting policy-relevant economic research and training. Although other donors also provide support, the US has played a critical role in shaping the directions in which AERC has advanced with respect to lines of research supported, its links with policy making, and soundness of the graduate-level training that is being undertaken. USAID’s ability to influence the directions in which this institution will move in the future, however, will be seriously compromised by lack of financial support. This will also have repercussions at the national level on the quality of researchers and research institutions, which are vital to improving the analytical basis for policy decisions. SAGA’s support for collaborative research between American and African researchers recognizes the fact that the most important research on international development has for some time been undertaken in the United States. This has served as the intellectual basis for market-oriented, trade-based policy reform, with due consideration of the impact of this reform on poverty. Collaborative research not only produces important findings that feed into the policy process but also provides a critical mechanism for transferring knowledge regarding analytical techniques and research methodologies to African researchers. In this way it contributes enormously to the sustainability and African ownership of the policy reform process. Research cannot be undertaken in a vacuum. Experience has shown that research centers, which bring senior and junior researchers together in collaboration, not only ensure that the research has the depth and substance required for sound policy decisions but also provide the most effective vehicle for transmitting research findings to policy makers. Although a number of such centers have attained premier status in the larger, more advanced African countries, there is an urgent need to support the development of similar centers in the smaller, poorer nations. Whether this is to be done through SISERA or another similar organization, the need for such support exists if policy decisions are to be based on sound empirical analysis. 6.9.3. RECOMMENDATIONS While funding by USAID for the types of activities supported by SAGA is likely to be quite limited, it is nevertheless important that some reasonable level of funding be maintained. Only in this way will USAID be able to bring the comparative advantage of the US in economic policy research to bear on the directions in which such research is pursued in Africa and how it is linked with policy decisions. The highest priority for the duration of SAGA is to maintain at least the existing annual level of support to Cornell ($400,000) and to increase the annual level of support to AERC from $200,000 to at least $600,000. Given Cornell’s diverse research areas, including education and HIV/AIDS, it would be appropriate for AID to consider funding sources outside of economic growth funds for this portion of the cooperative agreement. This would enable the collaborative research to continue, albeit at a reduced level, and would contribute something to the CMAP and CPP programs, which are vital to further upgrading of the economics profession. The current annual level of support to ILEAP of about $500,000 should also be continued, given the urgency posed by ongoing negotiations regarding regional integration (e.g., implementation of the common external tariff in ECOWAS) and economic partnership agreements, principally with the EU. Although some reference has been made in this report to the need to strengthen the linkages between research and policy making, this should not be taken to suggest that SAGA has failed in this respect. Both SAGA and its predecessor project, EAGER, have contributed substantially to improving these linkages. The situation is much better than it was ten years ago in the countries in which these projects SAGA MID-TERM EVALUATION REPORT 56 have operated. But Africa is vast and there is an urgent need to extend this success to other countries where sound policies are needed based on solid analysis. It is also important to recognize that this evaluation report only goes part of the way towards indicating directions for future funding. The Options Paper to be produced subsequently will examine these directions in much greater detail. What is recommended here is essentially only bridging until these options can be fully spelled out and a longer term strategy decided upon. SAGA MID-TERM EVALUATION REPORT 57 REFERENCES African Economic Research Consortium. “Annual Report: 2000 – 2001.” Nairobi: AERC, 2001. ________. “Annual Report: 2001 – 2002.” Nairobi: AERC, 2002. ________. “Annual Report: 2002 – 2003.” Nairobi: AERC, 2003. ________. “Annual Report: 2003 – 2004.” Nairobi: AERC, 2004. ________. “Collaborative Masters Programme for Anglophone Africa.” Nairobi: AERC, 2005. ________. “Collaborative Research Project on African Imperatives in the New World Trade Order.” Nairobi: AERC, 2004. ________. “Collaborative Research Proje ct on African Imperatives in the New World Trade Order: Report of Dissemination Workshop.” Nairobi: AERC, 2004. ________. “Collaborative Research Project on Poverty, Income Distribution and Labour Markets in Sub￾Saharan Africa.” Nairobi: AERC, n.d. ________. “Draft Annual Report: 2004 – 2005.” Nairobi: AERC, 2005. ________. “Financial Status Report: January 1 – March 31, 2005.” Nairobi: AERC, n.d. ________. “Financial Report for Grant to ILEAP: January 2004 – November 2004.” Nairobi: AERC, n.d. ________. “Memorandum of Understanding Between African Economic Research Consortium and International Lawyers and Economists Against Poverty.” Nairobi: AERC, March 2005. ________. “Phase V Strategy.” Nairobi: AERC, [2000?] ________. “USAID support to AERC administrative and support services.” Nairobi: AERC, 2005. Ali, Ali Abdel Gadir and Ibrahim A. Elbadawi. “Working Paper 9: Explaining Sudan’s Economic Growth Performance.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Aryeetey, Ernest and Augustin Fosu. “Growth Working Paper No. 7: Explaining African Economic Growth Performance: The Case of Ghana.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Asea, Patrick, Mthuli Ncube, and Jonathan Leape. “Financial Markets and Monetary Policy in South Africa and Zimbabwe.” In Financial Markets and Monetary Policies, n.d. Chapter 8. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Ayogu, Melvin D. and Chidozie Emenuga. “Central Banking Experience and the Conduct of Monetary Policy in Nigeria.” In Financial Markets and Monetary Policies, n.d. Chapter 7. Available from the SAGA MID-TERM EVALUATION REPORT 58 African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Azam, Jean-Paul and Nadjiounom Djimtoingar. “Growth Working Paper No. 6: Cotton, War and Growth in Chad (1960 – 2000).” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Baye, Francis. “Growth and redistribution effects of poverty changes in Cameroon.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Growth_and_Redistribution_Baye. pdf. Bevan, David and Kerfalla Yansane. “Mid Term Review of the AERC’s Phase V Strategy: “Africa into the 21st Century”. Nairobi: AERC, June 2003. Bloom, David E., Jeffrey D. Sachs, Paul Collier, and Christopher Udry. “Geography, Demography, and Economic Growth in Africa,” Brookings Papers on Economic Activity 1998, no. 2 (1998): 207-295 [journal on-line]; available from http://links.jstor.org/sici?sici=0007- 2303%281998%291998%3A2%3C207%3AGDAEGI%3E2.0.CO%3B2-X; Internet; accessed 23 June 2005. Boccanfuso, Dorothée and Tambi Samuel Kobore. “Macroeconomic growth, sectorial quality of growth and poverty in developing countries: measure and application to Burkina Faso.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at. Available from http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Bocconfuso_Macroeconomic_Gro wth.pdf. Buchs, Thierry and Johan Mathisen. “Banking Competition and Efficiency in Ghana.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.isser.org/Banking%20in%20Ghana.pdf. Chipeta, Chinyamata and Mjedo Mkandawire. “AERC Working Paper No. 2: Explaining African Economic Growth Performance: A Case Study of Malawi.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Collier, Paul and Jan Willem Gunning. “Policy commitment arrangements for Africa: Implications for aid, trade and investment flows.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Cornell University. “SAGA Monitoring and Evaluation Activities.” Ithaca, NY: Cornell University, n.d. ________. “SAGA Research Program.” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, 2001. Available from Cornell University at http://www.saga.cornell.edu/saga/proposal.pdf. SAGA MID-TERM EVALUATION REPORT 59 ________. “Strategies and Analyses for Growth and Access (SAGA): Progress Report.” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, April 2003. Available from Cornell University at http://www.saga.cornell.edu/saga/annreport/0403rep.pdf. ________. “Strategies and Analyses for Growth and Access (SAGA): Small Research Grants Program, Questionnaire for the Awardees.” Ithaca, NY: Cornell University, n.d. ________. “Strategies and Analyses for Growth and Access (SAGA): Questionnaire for the Hosting Institutions.” Ithaca, NY: Cornell University, n.d. ________. “Strategies and Analyses for Growth and Access (SAGA): Questionnaire for Research Collaborators.” Ithaca, NY: Cornell University, n.d. ________. “Strategies and Analyses for Growth and Access (SAGA): Questionnaire for Stakeholders and Policymakers.” Ithaca, NY: Cornell University, n.d. ________. “Strategies and Analyses for Growth and Access (SAGA): Questionnaire for USAID Mission Directors.” Ithaca, NY: Cornell University, n.d. Cornell University and Clark Atlanta University. “SAGA Progress Report (12/03 – 12/04) & Upcoming Workplan (1/05 – 12/05).” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, 2004. Available from Cornell University at http://www.saga.cornell.edu/saga/annreport/2004rep.pdf. ________. “Strategies and Analyses for Growth and Access (SAGA): Progress Report.” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, October 2002. Available from Cornell University at http://www.saga.cornell.edu/saga/annreport/annrep1002.pdf. ________. “Strategies and Analyses for Growth and Access (SAGA): Progress Report.” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, October 2003. Available from Cornell University at http://www.saga.cornell.edu/saga/annreport/1003rep.pdf. ________. “Strategies and Analyses for Growth and Access (SAGA): Progress Report.” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, April 2004. Available from Cornell University at http://www.saga.cornell.edu/saga/annreport/prgrep0404.pdf. Dabee, Beealasinghe. “Liberalisation of Trade in Services: The Case of Mauritius.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 11. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Danielson, Anders. “Economic Growth without Poverty Reduction: Identifying the Missing Links in Tanzania during Economic Reform.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Growth_without_Poverty_Reductio n_Danielson.pdf. Dorward, Andrew and Jamie Morrison. “Modelling pro-poor agricultural growth strategies in Malawi: lessons for policy and analysis.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at SAGA MID-TERM EVALUATION REPORT 60 http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Modelling_pro￾poor_Dorwood_Morrison.pdf. Gelb, Stephen. “Inequality in South Africa: Nature, Causes and Responses.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro￾Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Gelb_Inequality_in_SouthAfrica.p df. Ghura, Dhaneshwar. “Macro Policies, External Forces, and Economic Growth in Sub-Saharan Africa,” Economic Development and Cultural Change 43, no. 4 (July 1995): 759-778 [journal on-line]; available from http://links.jstor.org/sici?sici=0013- 0079%28199507%2943%3A4%3C759%3AMPEFAE%3E2.0.CO%3B2-S; Internet; accessed 23 June 2005. Glick, Peter and David E. Sahn. “Changes in HIV/AIDS Knowledge and Testing Behavior in Africa: How Much and for Whom?” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.saga.cornell.edu/images/wp173.pdf. Harris, Laurence. “Regional Integration and Capital Markets in Africa.” In Financial Markets and Monetary Policies, n.d. Chapter 3. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Hassan, Rashid and Henry Rempel. “Building an Economics Profession in Africa: An Evaluation of Phase V of the African Economic Research Consortium.” Nairobi: AERC, February 2005. Hentz, James J. “Economic Stagnation in Sub-Saharan Africa and Breaking the ‘Implicit Bargain’,” Issue: A Journal of Opinion 25, no. 1 (1997): 32-34 [journal on-line]; available from http://links.jstor.org/sici?sici=0047- 1607%281997%2925%3A1%3C32%3AESISAA%3E2.0.CO%3B2-L; Internet; accessed 23 June 2005. Hogset, Heidi. “Economic Transfers Through Social Networks and Financial Trickle-Down in Kenya’s Smallholder Sector.” Paper presented at the SAGA Workshop on Empowering the Rural Poor and Reducing Their Risk and Vulnerability, Nairobi, Kenya, 10 February 2005. Available from Cornell University at http://www.saga.cornell.edu/saga/keny0205/hogset.pdf. Honohan, Patrick and Stephen A. O’Connell. “Contrasting Monetary Regimes in Africa.” In Financial Markets and Monetary Policies, n.d. Chapter 2. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Ikiara, Gerishon K. “Service Sector Issues in Sub-Saharan Africa: A Case Study of Kenya.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 8. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. International Lawyers and Economists Against Poverty. “Expenditure Plan for USAID Funding for Grant Period January 1, 2005 – September 30, 2005.” Toronto: iLeap, 23 February 2005. SAGA MID-TERM EVALUATION REPORT 61 ________. “ILEAP Activity Report: June 1, 2004 – December 3, 2004.” Toronto: iLeap, 17 January 2005. ________. “Post Cancun Support for Enhanced African Participation in International Trade Negotiations and Commercial Policy Formulation and Implementation.” Toronto: iLEAP, n.d. ________. “Post Cancun Support for Enhanced African Participation in International Trade Negotiations and Commercial Policy Formulation and Implementation: Report on Activities, January 1 – May 31, 2004.” Toronto: iLEAP, n.d. ________. “USAID Expenditures: June 1 – November 30, 2004.” Toronto: iLeap, n.d. ISSER-Cornell. “Participant Evaluation.” International Conference on Ghana at the Half Century: July 18 – 20, 2004. Ithaca, NY: Cornell University, n.d. Jebuni, Charles D. “Trade Related Intellectual Property Rights and African Economic Development.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 6. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Kasekende, Louis A. and Michael Atingi-Ego. “Financial Markets and Monetary Policy in Sub-Saharan Africa: The Experience of Uganda.” In Financial Markets and Monetary Policies, n.d. Chapter 10. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Kasekende, Louis A., Michael Atingi Ego, and Rachel Kaggwa Sebudde. “Working Paper 8: The African Growth Experience Uganda Country Case Study.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Kimei, Charles S. “Financial Markets and Operation of Monetary Policy in Africa: The Case of Tanzania.” In Financial Markets and Monetary Policies, n.d. Chapter 6. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Koch, Stephen F. and Albert de Wet. “How responsive is capital formation to its user cost? An exploration of corporate tax effects.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/How_Responsive_is_Capital_Form ation_DeWet_Koch.pdf. Luke, David F. “Trade-Related Capacity Building for Enhanced African Participation in the Global Economy.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 4. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Marouani, Mohamed Ali. “Visit to the African Institute of Applied Economics, Enugu, Nigeria, May 3-5, 2005.” Dakar: SISERA, 11 May 2005. Marouani, Mohamed Ali. “Visit to the Development Policy Research Unit, Cape Town, South Africa, June 2-6, 2005.” Dakar: SISERA, 30 June 2005. SAGA MID-TERM EVALUATION REPORT 62 Mattoo, Aaditya. “Developing Countries in the New Round of GATS Negotiations: Towards a Pro-Active Role.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 3. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Mballa, Godfroy and Dominique Njinkeu. “Services at the Millennium Round of Trade Negotiations: The Case of Cameroon.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 12. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Mugunieri, Godiah L. and John M. Omiti. “Enhancing Access, Accountability and Empowerment for the Poor Through Decentralization for Agricultural Extension Service in Easter Kenya.” Paper presented at the SAGA Workshop on Empowering the Rural Poor and Reducing Their Risk and Vulnerability, Nairobi, Kenya, 10 February 2005. Available from Cornell University at http://www.saga.cornell.edu/saga/keny0205/mugunieri-omiti.pdf. Mwega, Francis M. and Njuguna S. Ndung’u. “AERC Working Paper No. 3: Explaining African Economic Growth Performance: The Case of Kenya.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Nambiro, Elizabeth and John M. Omiti. “Enhancing Access, Accountability and Empowerment Through Decentralization and Participation: Agricultural Extension Service in Western Kenya.” Paper presented at the SAGA Workshop on Empowering the Rural Poor and Reducing Their Risk and Vulnerability, Nairobi, Kenya, 10 February 2005. Available from Cornell University at http://www.saga.cornell.edu/saga/keny0205/nambiro-omiti.pdf. Nath, Shyam and Yeti Nisha Madoo. “Growth Working Paper 1: Explaining African Economic Growth Performance: The Case of Mauritius.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Ncube, Mthuli. “Introduction.” In Financial Markets and Monetary Policies, n.d. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. Ndiaye, Abdoulaye. “GATS 2000: As Seen From Senegal.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 10. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Njinkeu, Dominique and Charles C. Soludo. “Industrialising Africa Using WTO Framework.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 5. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Nkurunziza, Janvier D. and Floribert Ngaruko. “Growth Working Paper No. 5: Explaining Growth in Burundi: 1960-2000.” January 2004. Available from the African Economic Research Consortium website under “Growth Working Papers” at http://www.aercafrica.org/publications/category.asp. Njikam, Ousmanou, Joachim Nymeck Binam, and Simon Tachi. “Understanding Total Factor Productivity Growth in Sub-Saharan African Countries.” Paper, University of Yaounde II, May 2005. SAGA MID-TERM EVALUATION REPORT 63 Noorbakhsh, Farhad and Alberto Paloni. “Structural Adjustment and Growth in Sub-Saharan Africa: The Importance of Complying with Conditionality,” Economic Development and Cultural Change 49, no. 3 (April 2001): 479-509 [journal on-line]; available from http://links.jstor.org/sici?sici=00130079%28200104%2949%3A3%3C479%3ASAAGIS%3E2.0.CO %3B2-T; Internet; accessed 23 June 2005. Noumba, Issidor. “Are wealthier nations healthier nations? A panel data approach to the determination of human development in Africa.” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Are_Wealthier_Nations_Healthier_ Nations_Issodor_Noumba.pdf. Nyoro, James K. and Isaac Komo. “An Analysis of Success, Failure and Demand Factors of Agricultural Cooperatives in Kenya.” Paper presented at the SAGA Workshop on Empowering the Rural Poor and Reducing Their Risk and Vulnerability, Nairobi, Kenya, 10 February 2005. Available from Cornell University at http://www.saga.cornell.edu/saga/keny0205/tegemeo.pdf. N’Zue, F.F. and C.J.F. N’Guessan. “The Causality Between Corruption, Poverty and Growth: A Panel Data Analysis.” Paper, University of Cocody, May 2005. Okidi, John. “Report on the Survey Data Analysis Workshop: August 26 – September 6, 2002.” Kampala: Economic Policy Research Institute, [2002?] Oostuhuizen, Mome. “Report on the Technical Assistance Course: Analysis and Measurement of Poverty and Inequality, June 23 – July 4, 2003.” Cape Town: Development Policy Research Unit, [2003?] Oyejide, Ademola and William Lyakurwa, eds. “Africa and the world trading system; Volume I: Framework papers.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Introduction and Overview. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Oyejide, Ademola T. and Adiodun Bankole. “Liberalisation of the Services Sector in Nigeria: Implications of Unilateral and Multilateral Approaches.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Chapter 9. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Oyejide, Ademola and Samuel M. Wangwe. “Trade, Industry and Technology Development in Sub￾Saharan Africa: Policies, Response and Effects.” Paper presented at the Geneva WTO Trade Negotiation Seminar, Geneva, Switzerland, 9 March 2000. Available from the African Economic Research Consortium website under “Trade Papers” at http://www.aercafrica.org/publications/category.asp. Ravenhill, John. “Adjustment with Growth: A Fragile Consensus,” The Journal of Modern African Studies 26, no.2 (June 1998): 179-210 [journal on-line]; available from http://links.jstor.org/sici?sici=0022- 278X%28198806%2926%3A2%3C179%3AAWGAFC%3E2.0.CO%3B2-W; Internet; accessed 23 June 2005. SAGA MID-TERM EVALUATION REPORT 64 Sahn, David and Diery Seck. “Memorandum of Understanding between Cornell and Clark-Atlanta Universities and SISERA for the Implementation of the SAGA Project.” [Dakar?], 21 December 2001. Sahn, David. “SAGA Briefing Report HFM-A-00-01-00132-00.” Ithaca, NY: Cornell University, Strategies and Analysis for Growth and Access project, November 2003. Available from Cornell University at http://www.saga.cornell.edu/saga/annreport/briefrep.pdf. Secretariat for Institutional Support for Economic Research in Africa. “Annual Report 2002 - 2003.” Dakar: SISERA, 2003. ________. “How to Apply for SISERA for funding.” Dakar: SISERA, n.d. ________. “Progress Report by Activity—SAGA Program: October 1, 2001 – September 30, 2002.” Dakar: SISERA, 2002. ________. “Progress Report by Activity—SAGA Program: October 1, 2002 – September 30, 2003.” Dakar: SISERA, 2003. ________. “Progress Report by Activity—SAGA Program: October 1, 2003 – September 30, 2004.” Dakar: SISERA, 2004. ________. “Progress Report by Activity—SAGA Program: October 1, 2004 – December 31, 2004.” Dakar: SISERA, 2005. ________. “Progress Report by Activity—SAGA Program: January 1, 2005 – March 31, 2005.” Dakar: SISERA, 2005. ________. “SISERA Work Program and Budget April 2002 – March 2003.” Dakar: SISERA, April 2002. ________. “SISERA Work Program and Budget April – December 2003.” Dakar: SISERA, April 2003. ________. “SISERA Work Program and Budget April 1, 2004 – March 31, 2005.” Dakar: SISERA, March 2004. ________. “SISERA Work Program and Budget April 1, 2005 – March 31, 2006.” Dakar: SISERA, March 2005. ________. “Strategies and Analyses for Growth and Access (SAGA) Project: Call for Proposals.” Dakar: SISERA, [2001?] ________. “Support Instruments.” Dakar: SISERA, November 1999. Sowa, Nii Kwaku. “Central Banking and Monetary Management and Ghana.” In Financial Markets and Monetary Policies, n.d. Chapter 4. Available from the African Economic Research Consortium website under “Financial Markets and Monetary Policies” at http://www.aercafrica.org/publications/category.asp. University of Cape Town. “Human Capital Outcomes in South Africa, Grant No.: 065077-104, Technical Progress Report for the period ended 25 March 2005.” Paper, Cape Town: Development Policy Research Unit, 2005. SAGA MID-TERM EVALUATION REPORT 65 U.S. Agency for International Development. “Analytical Support to African Trade Negotiations: ILEAP Scope of Work under USAID Grant for 2004 and 2005.” Washington, DC: USAID, 23 February 2005. ________. “Contract between U.S. Agency for International Development and the African Economic Research Consortium regarding the “Strategies and Analysis for Growth and Access” project.” Washington, DC: USAID, 2001. ________. “Contract between U.S. Agency for International Development and Cornell University regarding the “Strategies and Analysis for Growth and Access” project.” Washington, DC: USAID, 2001. ________. “Contract between U.S. Agency for International Development and the Secretariat for Institutional Support for Economic Research in Africa regarding the “Strategies and Analysis for Growth and Access” project.” Washington, DC: USAID, 2001. ________. “ILEAP 2 Indicator Table: Africa Trade Capacity Building Activity.” Washington, DC: USAID, n.d. ________. “Request for Application No. M/OP-01-049.” Washington, DC: USAID, 2001. ________. “Strategies and Analyses for Growth and Access: Activity Document.” Washington, DC: USAID, n.d. ________. “Support for African Research and Analysis On International Trade and Commercial Policy (Revised).” Washington, DC: USAID, n.d. Valodia, Imraan and Myriam Velia. “Macro-Micro Linkages in Trade: How are Firms Adjusting to Trade Liberalisation, and does Trade Liberalisation lead to improved Productivity in South African Manufacturing Firms?” Paper presented at the DPRU-TIPS-Cornell University Forum on African Development and Poverty Reduction: The Macro-Micro Linkage, Cape Town, South Africa, 13-15 October 2004. Available from Cornell University at http://www.commerce.uct.ac.za/dpru/dpruconference2004/Papers/Micro_evidence_of_Trade_Reform _Valodia.pdf. Weeks, John. “Regional Cooperation and Southern African Development,” Journal of Southern African Studies 22, no. 1 (March 1996): 99-117 [journal on-line]; available from http://links.jstor.org/sici?sici=0305- 7070%28199603%2922%3A1%3C99%3ARCASAD%3E2.0.CO%3B2-0; Internet; accessed 23 June 2005. Wuyts, Marc. “An Evaluation of the AERC Research Programme: 1997 – 2003.” The Hague: Institute of Social Studies, July 2004. SAGA MID-TERM EVALUATION REPORT 66 ANNEX A: EVALUATION SCOPE OF WORK Scope of Work for SAGA Midterm Evaluation & Post SAGA Options Request for Proposals Task Order- In accordance with Section C.3 Task B – USAID Field Missions and USAID Washington Bureaus of the Basic Award I. Objective To conduct a mid-term evaluation of the Strategies and Analyses for Growth and Access (SAGA) program, an economic capacity building program focused on Sub-Saharan Africa, and provide recommendations for improving its implementation in the remaining two years before its completion. An additional task is to develop a concept paper exploring options for a possible follow-on activity to SAGA. II. Background SAGA is the successor to the Equity and Growth through Economic Research (EAGER) program administered by AFR/Sustainable development Office/Strategic Analysis Division (SD/SA) in USAID/W. 6 The SAGA program is authorized as a five year, $15 million program with three objectives: (1) policy oriented research on economic reform issues in Sub-Saharan Africa; (2) training and institutional strengthening of African economic policy institutes; and (3) a set of dissemination and professional exchange activities that will increase the knowledge and understanding of African economies in the U.S. as well as Africa. SAGA began implementation in September 2001 and is scheduled to be completed in September, 2006. It is a centrally managed program that originated in the Africa Bureau and was transferred to the EGAT/Economic Growth Office (EGAT/EG) early in its implementation. Since FY 2003, the Africa Bureau has transferred funds for implementing SAGA to EGAT/EG, and the EGAT/EG office provides management of this activity. SAGA has had three major implementers; the African Economic Research Consortium, (AERC) based in Nairobi, Kenya; the Secretariat for Institutional Support for Economic Research in Africa (SISERA) based in Dakar, Senegal; and Cornell University. USAID has a multi-year grant with AERC to support research and training programs for African economists. AERC coordinates across departments of economics from 20 African universities to manage a uniform M.A. program in Economics and has recently started a Ph.D. Economics program simultaneously being offered at four African Universities. AERC also administers small grants to African researchers to carry out policy related studies, and has a strong peer review process to ensure that the outputs are of high quality. SISERA was created in 1997 through support from USAID, the Canadian International Development Agency (CIDA) and the International Development Research Centre (IDRC). SISERA’s overall goal is to ensure provision of relevant economic policy research that informs and influences policymakers and contributes to civil society debates on economic policy reform options. It strives to do this by working with a network of ten or so national African economic policy institutes and providing them with managerial, research and training support so that they can be more effectively used by the policy community. 6 This section provides only a brief overview of the SAGA program, but much more extensive descriptions can be found in the documents listed in the Annex under “Key Documents and Websites”. SAGA MID-TERM EVALUATION REPORT 67 Cornell University has a cooperative agreement with USAID to administer a program with three components: US-based collaborative research on topics pertaining to economic growth with access in SSA; provision of technical assistance to African research institutes; and a small grants program that places U.S. based economic Ph.D.’s or faculty at African policy institutes for a two to three month period in order to maintain professional engagements between Americans and Africans working on economic reform issues. The SAGA program contributed to AFR/SD/SA’s Strategic Objective “Adoption of Strategies, Programs, and Activities for Accelerated, Sustainable, and Equitable Economic Growth”. This strategic objective had two intermediate results: (1) Develop strategies, policies, and activities to increase trade and investment, mobilize domestic resources, and liberalize key markets; and (2) Strengthen African capacity to design, advocate, and manage strategies, policies, and activities for accelerated, sustainable and equitable growth. However, the AFR/SD office has recently completed and obtained approval for a new strategy, and one of the tasks of the contractor will be to determine how well the SAGA program fits into the new Economic Growth SO in Afr/SD as well as in EGAT/EG. III. Tasks The contractor will: (i) Review the project design documents and work plans for all three implementers and provide a summary of each implementer’s objectives under the SAGA program. The major question to be addressed here is: What did the program indicate it was going to achieve, and how were the individual implementers going to contribute to this overall objective? Each implementer has a number of different activities/functions under its overall program, i.e. research, technical assistance, program management, and each of these sub-components should be included in the summary description. Methodology: Work on this section will involve reviewing secondary documents and will be a desk study. Key documents are referenced in the annex to the SOW. (level of team effort: 15 days) (ii) Determine whether the program(s) of each implementer is meeting the stated objectives. Document the major accomplishments/weaknesses of each program since the inception of SAGA, indicating as well what results are likely to be achieved by the completion of the SAGA program. All of the sub￾components of each implementer should be included in this exercise. The contractor should clearly indicate who the major beneficiaries are of all of the programs and their sub-components and to what extent these clients have benefited from the SAGA programs. Institutional strengthening and capacity building efforts such as those undertaken by SAGA are typically very difficult to capture by quantitative indicators. Accordingly, the contractor will need to rely on a mix of evaluation techniques. Major constraints that prohibited the implementers from meeting stated objectives should also be highlighted. Methodology: This section will require field visits to Africa to conduct interviews with the two Africa based implementers, as well as many of the beneficiaries and stakeholders of the project that are based in Africa. Determination of specific site visits will be made in consultation with the SAGA program manager. In addition, the contractor will mostly likely need to employ a questionnaire to solicit input from a broader range of stakeholders on SAGA impacts. It is highly recommended that the contractor employ local consultants with relevant expertise in institutional strengthening during their in-country work to augment existing team capabilities. (level of team effort: 65 days) (iii) Provide recommendations for increasing performance and the documenting/dissemination of this performance in the remaining two years of the SAGA activity. Currently, one of the SAGA SAGA MID-TERM EVALUATION REPORT 68 implementers, SISERA, is “winding down”, leaving only two implementers under the SAGA program. The contractor should provide recommendations for what AERC and Cornell can do to improve impact in the remaining two years of SAGA. The contractor will also assess the Monitoring and Evaluation system of each implementer and determine whether it adequately captures the results that are being achieved. Such an assessment should address whether these results are pertinent to the EG SO Results framework for EGAT/EG and/or AFR/SD. Recommendations should be provided for improving indicators and data collection as warranted, with a particular emphasis on increasing field mission awareness of these activities. Methodology: Outputs under Task 3 will draw upon the expertise of the contractors and the information collected in Tasks 1 & 2 and will be a desk study. (level of effort: 25 days) (iv) Develop an options paper to assist EGAT/EG in considering possible alternatives to consider in the area of economic capacity building after SAGA expires in 2006. Determine the extent to which the goals and objectives of SAGA are applicable to other developing regions and thus warrant “globalizing” SAGA, i.e. do other regions have similar needs for training and institutional strengthening that SAGA provides? What aspects of the program are still needed for African capacity building and what components need to be modified—is there evidence of sustainability once the program ends? What best practices can be distilled and used for replication in other regions? Does USAID have a unique and important role in this type of work vis-à-vis other donors or is there a significant overlap? How would this new activity fit into the new strategic objectives of STATE/USAID joint strategic planning? Methodology: Outputs under Task (iv) will entail interviews with USAID EG officers in USAID/W regional bureaus and pillar Bureau’s, with field mission officers in each region, and donor representatives engaged in similar capacity building efforts. (level of effort: 55 days) IV. Deliverables The contractor will deliver the following: A detailed work plan within three weeks of effective date of the task order that lays out the schedule of the remaining deliverables and is agreed to by the CTO (Cognizant Technical Officer). The work plan will be updated every two months. Four reports: 1) a SAGA project summary report, 2) an Evaluation report, (3) a Recommendations report, and (4) an Options Paper corresponding to tasks 1-3 outlined above. The contractor is required to submit a draft report for each task and get approval before beginning to work on the subsequent task. All reports shall be provided first in draft and after comment, in final. The final reports shall be provided electronically in a format useable to EGAT/EG and in ten hard copies to the CTO and activity manager, and two copies filed with CDIE. USAID Copies The reports should be submitted to: Yoon Joo Lee, Ph D, Economist & CTO USAID EGAT/EG, Room RRB-2.11-72 Washington, DC 20523 Tel:202-712-4281; Fax: 202-216-3010 EM: ylee@usaid.gov SAGA MID-TERM EVALUATION REPORT 69 Rita Aggarwal Senior Africa Economist USAID/W 1300 Pennsylvania Ave. NW EGAT/EG 2nd floor Room 211-126 Washington DC, 20523-2110 Electronic version to: Yoon Joo Lee, CTO for SAGA ylee@usaid.gov Rita Aggarwal, Activity Manager for SAGA raggarwal@usaid.gov Technical Direction: The contractor shall work under the technical direction of the Team Leader for the Economic Policy and Governance Team in the Office of Economic Growth in EGAT (Bureau for Economic Growth, Agriculture and Trade) or his or her designee. Meetings with USAID: The contractor will meet with the SAGA activity manager and other relevant individuals at the beginning of the evaluation exercise to review the tasks and discuss a course of action that is mutually agreeable. The contractor will also participate in half-day roundtable meetings with USAID/W representatives from EGAT/EG, the Africa Bureau, and other relevant individuals after completion of draft reports #3 and #4. V. Period of Performance The period of performance for this Task Order will be from date of award of the Task Order through one year thereafter. V. Team Composition & Budget: Personne l Requirements: Team Leader: Development Economist with minimum ten years of international development experience, experience with local capacity building of economic institutes highly desirable. Candidate should have familiarity with USAID Economic Growth programs, USAID strategic planning and results reporting. Strong familiarity with Africa, French language proficiency preferred. Ability to communicate effectively with economic researchers, USAID officials and other donor representatives. Mid-level Researcher: Monitoring & Evaluation specialist with expertise in several different evaluation techniques. Prefer prior experience in evaluating economic capacity building programs. Knowledge of Africa helpful. Research Analyst: Strong research and writing skills. Work on international development issues for minimum of 5 years. Familiarity with economic capacity building programs helpful. Local Consultant: Ph.D. economist with strong familiarity of the SAGA institutions in the country(s) or considerable experience with similar institutes. The individual should have experience either in evaluating SAGA MID-TERM EVALUATION REPORT 70 capacity building projects such as SAGA and/or have direct experience with local research institutes and networks working in areas related to economic policy reform. The consultant will be an integral part of the SAGA evaluation team during the field visits; he/she will be involved in assisting the team prepare the evaluation questionnaire prior to meeting with the SAGA institutes, facilitate meetings with these institutes, and gather local knowledge of the institute(s) and the impact of the SAGA program that may not be available to the US based team members. The consultant should be expected to contribute in the compiling and analyzing of data obtained from the field visits, and comment on draft and final reports prepared by the US team on Tasks (2) and (3) of the SOW. SOW Annex: Select References to Key Documents and websites on the Strategies and Analyses for Growth and Access (SAGA) program. Cornell/SAGA website: http://www.saga.cornell.edu/saga/conf.html AERC website: http://www.aercafrica.org/home/index.asp IDRC website: http://network.idrc.ca/en/ev-10212-201-1-DO_TOPIC.html http://www.ileapinitiative.com/pages/capacity_seminar.htm Cable to field missions on the SAGA program: E.O. 12958, sent 1/02/2002 SISERA Evaluation, draft report submitted to SISERA July, 2004 Contact: Elias Ayuk, Elias Ayuk [eayuk@idrc.org.sn] SISERA Annual WorkPlan Documents, contact Elias Ayuk, [eayuk@idrc.org.sn] Cornell Semi-Annual Progress Reports, contact Philip G. Neuwirth [pgn1@cornell.edu] AERC annual workplan and relevant evaluations, contact Executive Director [Exec.Dir@aercafrica.org]. SAGA MID-TERM EVALUATION REPORT 71 ANNEX B: LIST OF PERSONS INTERVIEWED Interviewees - Ghana Ernest Aryeetey Director, ISSER (Institute of Statistical, Social and Economic Research) University of Ghana 233-21-501182/3 aryeetey@ug.edu.gh Dr. Yam Asante Program Director (EPM) Department of Economics University of Ghana 233-21-501485 yasante@econs.ug.edu.gh Ruby Adjaidoo Bentsi Economist DFID 233-21-253243 ra-bentsi@dfid.gov.uk Dr. Charles Jebuni Research Fellow CEPA (Centre for Policy Analysis) 233-21-778035 charles@cepa.org.gh Dr. Frank Nyonator Director Policy, Planning, Monitoring & Evaluation Division Ghana Health Service 233-21-684272 frank.nyonator@ghsmail.org Abena Oduro Research Fellow CEPA (Centre for Policy Analysis) 233-21-778035 abena@cepa.org.gh Eline Okudzeto Economist, Trade Agriculture and Private Sector USAID Ghana 233-21-228-440 eokudzeto@usaid.gov SAGA MID -TERM EVALUATION REPORT 72 Dr. Kofi A. Osei Head, Finance Department University of Ghana Business School 233 -21 -501594 kaosei@ug.edu.gh Bernadin Senadza Economist/Lecturer University of Ghana 233 -21 -501485 bsenadza@yahoo.com, bsenadza@ug.edu.gh Jeremy Strauss Trade and Investment Advisor USAID Ghana 233 -21 780580 jstrauss@usaid.gov Dr. Anthony Tsekpo Research Fellow ISSER University of Ghana 233 -21 -501182 aktsekpo@ug.edu.gh Kwesi Yeboah-Konadu CEPA 233 -21 -778035 konadu@cepa.org.gh Mr. Kofi Larbi Ministry of Trade and Industry No Show. Mr. Afrani –Ministry of Education, Youth and Sports ILEAP workshop attendee Terminated the interview Interviewees – Uganda Peter Elyetu Principal Commercial Officer Ministry of Tourism, Trade and Industry 256 -41 -340 -589 elimu@mtti.go.ug Liz Kiingi Program Officer USAID 256 -31 -387 -387 lkiingi@usaid.gov SAGA MID -TERM EVALUATION REPORT 73 Dr. Adam Mugume Economist Makerere University 256 -41 -530115 amugume@fema.mak.ac.ug Onesmus Mugyenyi Research Fellow and Manager ACODE 256 -41 -530 -798 omugyenyi@acode -u.org Dr. Marios Obwona Principal Research Fellow EPRC -Economic Policy Research Centre 256 -41 -541 -023 obwona@eprc.or.ug Dr. John Okidi Executive Director EPRC – Economic Policy Research Centre Makerere University 256 -41 -541234 okidi@eprc.or.ug Dr. Sarah Ssewanyana Senior Research Fellow EPRC 256 -41 -540159 ssewanyana@eprc.or.ug Dr. Germina Ssemogerere Lecturer Faculty of Economics and Management Makerere University 256 -75 -212 -903 pssemo2@yahoo.com Interviewees – South Africa Dr. Melvin Ayogu Professor of Economics, Head AERC PhD program University of Cape Town 27 -21 -650 -2763 mayogu@commerce.uct.ac.za Dr. Haroon Bhorat Director Development Policy Research Unit (DPRU) University of Cape Town 27 -21 -650 -5698 hbhorat@commerce.uct.ac.za SAGA MID -TERM EVALUATION REPORT 74 Rashad Cassim Head, School of Economic & Business Sciences University of Witwatersrand 27 -11 -717 -8142/12 cassimr@sebs.wits.ac.za Neal Cohen Economist USAID Pretoria 27 -12 -452 -2248 ncohen@usaid.gov Stan du Plessis Macroeconomics Lecturer University of Stellenbosch stan@sun.ac.za Stephen Harnival Executive Director Trade and Industrial Policy Strategies (TIPS) 27 -12 -431 -7900 stephen@tips.org.za Trudi Hartzenberg Executive Director Trade Law Centre for Southern Africa (TRALAC) 27 -21 -883 -2208 trudihartzenberg@tralac.org Joann Feldman Lawrence Director, Economic Growth Office USAID Pretoria 27 -12 -452 -2241 jlawrence@usaid.gov Dr. Mthuli Ncube Head of Business School University of Witswatersand 27 -83 -454 -6597 ncube.m@wbs.ac.za Dr. Charles Simkins Professor of Political Economy University of Witswatersand 27 -21 -726 -4535 sapere@mweb.co.za SAGA MID -TERM EVALUATION REPORT 75 Ben Smit Director Bureau for Economic Research (BER) University of Stellenbosch 27 -21 -887 -2810 bws@bers.sun.ac.za Interviewees – Kenya Prof. Olu Ajakaiye Director of Research African Economic Research Consortium 254 -20 -2734150/3/7 Olu.ajakaiye@aercafrica.org Julius K. Kilungo Program Specialist/Economist USAID 254 -20 -862400/2 jkilungo@usaid.gov Joseph T. Karugia Manager, Research African Economic Research Consortium 254 -20 -2734150/3/7 joseph.karugia@aercafrica.org Dr. T. Nzioki Kibua Executive Director Institute of Policy Analysis & Research 254 -20 -251179/252885 tnkibua@ipar.or.ke Prof. Peter Kimuya Economics Department University of Nairobi pkimunya@africaonline.co.ke Jane Kabubo -Mariara Senior Lecturer & Consultant Researcher Department of Economics University of Nairobi 254 -20 -245566 jmariara@uonbi.ac.ke G. Kioko wa Luka Administrator Institute of Policy Analysis & Research 254 -20 - 251179/252885 kioko@ipar.or.ke SAGA MID-TERM EVALUATION REPORT 76 William Lyakurwa Executive Director African Economic Research Consortium 254-20-2734150/3/7 exec.dir@aercafrica.org Jacqueline Macakiage Executive Assistant African Economic Research Consortium 254-20-2734150/3/7 jacqueline.macakiage@aercafrica.org Elijah B. Manyara Deputy Director Department of External Trade Ministry of Trade & Industry 254-20-315001 kextrade@africaonline.co.ke James K. Nyoro Executive Director Tegemeo Institute of Agriculture Policy & Development Egerton University 254-20-2717818 tegemeo@kenyaweb.org Njuguna Ndung’u Director of Training African Economic Research Consortium 254-20-2734150/3/7 njuguna.ndungu@aercafrica.org Steve Ndele Senior Economist USAID Kenya 254-20-8622218 sndele@usaid.gov SAGA MID-TERM EVALUATION REPORT 77 Hezron O. Nyangito Ag. Executive Director KIPPRA – Kenya Institute for Public Policy Research and Analysis 254-20-2719933 hnyangito@kippra.or.ke Prof. Willis Oluoch-Kosura Ag. Program Director AEEB – Agriculture Economics Education Board 254-20-2734150/3/7/63 willis.kosura@aercafrica.org Interviewees – Senegal Mabousso Thiam Administrator PPIP - Project de Promotion des Investissements Prives 221-869-60-60 mthiam@ppip.sn Prof. Abdoulaye Diaone Director, CRES UCAD 221-842-85-96 cres@ucad.sn Aliou Faye Director CEPOD - Centre d’Etude de Politiques pour le Développement 221-842-64-22 fayealiou@hotmail.com Prof. Ahmadou Aly Mbaye Director Centre de Recherches Economiques Appliqees Universite Cheikh Anta Diop de Dakar 221-824-78-61 ambaye@refer.sn Elias T. Ayuk Senior Program Specialist International Development Research Centre 221-864-00-00 aeyuk@idrc.org.sn SAGA MID-TERM EVALUATION REPORT 78 ANNEX C: INTERVIEW PROTOCOLS SAGA Interview Protocol – Implementing Institutions Name: Date: Position: Institution: Country: Overall Project: 1. Where do you feel your project has had the greatest success? 2. Who have been the main beneficiaries of the project? How has it impacted these beneficiaries? 3. Were there any shortcomings in the design of the SAGA project at the program and implementation levels? If so, please explain? 4. Have there been any substantial changes in project design or implementation resulting from actions taken by USAID or other external events since the project began? How have you responded to these changes? 5. What have been the main impacts of the project? Do you believe the project impacts are sustainable? What factors support sustainability and what factors make sustainability a challenge? 6. Do you have a plan to ensure sustainability? Please provide details. 7. Do you feel that the project is strengthening economic research capacities within Africa? If yes, how? 8. Do you feel that the research the project is conducting/supporting assists or will assist in formulating policies? What is your assessment of the project’s policy outreach efforts? Should these be strengthened? How? 9. Are there additional areas of research you feel SAGA should address and fund in the future? 10. Has the project been able to strengthen the networking capabilities of researchers, research centers and clients? If yes, please provide details. 11. Please describe the relationship that you have with other implementing partners. What has worked well? What has been challenging? What suggestions do you have regarding how these relationships could be strengthened and be made more effective? Performance Management 12. In what ways, if at all, were policy makers and stakeholders included in the project design, implementation, and review processes, including selection and organization of research? 13. How have project management decisions been made at the level of your institution? Among implementing organizations? In relation to USAID? 14. How do you measure the effectiveness of the project activities in contributing toward project goals? Do you have a performance monitoring plan? What is the extent of your data collection efforts regarding performance monitoring? 15. How are findings from ongoing monitoring efforts incorporated into adjusting your program activities? Please provide specific examples of how you have used these findings to alter your program activities. SAGA MID-TERM EVALUATION REPORT 79 QUESTIONS FOR CORNELL: 16. Has the small grants program been able to strengthen linkages between African and U.S. economic researchers? If yes, please describe in what ways. 17. Has the project been able to strengthen the capacity of economic research institutes in Africa? The capacity of individual African researchers? What have been some of the challenges? What are the prospects for additional progress? QUESTIONS FOR SISERA: 18. Has the project been able to strengthen the capacity of economic research institutes? What have been some of the challenges? What are the prospects for additional progress? QUESTIONS FOR AERC: 19. In what ways has the CMAP and CPP training been most effective? Least effective? 20. How well has the project been able to build and sustain local capacity of researchers? QUESTIONS FOR ILEAP: 21. Has the project been able to increase participation and discussions on trade policy issues? a. If yes, in what ways? Who has been involved? b. Do you feel there is increased demand for research on trade policy? If yes, please describe. SAGA MID-TERM EVALUATION REPORT 80 SAGA Interview Protocol – Partner Institutions Name: Date: Position: Institution: Country: 1. What kind of support does your organization and its members receive from the project? Please describe. 2. How has the support provided by SAGA benefited your organization’s efforts? 3. Is this the type of support that you feel would have the greatest impact on your organization? If so, why? If not, why not and what other types of support would be beneficial? 4. Do you feel that research conducted under the project has helped to strengthen your economic research capacities? Please elaborate. 5. Has the research conducted under the project been designed to influence policy? Has it in fact influenced policy? If yes, in what ways? If no, what changes could be made to have a greater influence on policy? In designing the research conducted under the project, were policy makers, business leaders, or other stakeholders consulted? 6. In what specific ways do you feel that the research support provided through SAGA could be improved? 7. Are there additional areas of research you feel SAGA should address and fund in the future? 8. What have been the major impacts/effects of your organization’s work and the support you have received on your organization? On participating researchers? On policy decisions? 9. Do you feel these changes will last, even when support from SAGA ends? Why/why not? What can be done to ensure that the changes will last? 10. Do you feel the project has been instrumental in helping researchers and research centers network and build relationships with one another? If yes, please describe. 11. For AERC partners: Do you feel the project adequately supports the MA/PhD programs? Please elaborate. a. Are there areas that can be improved? What are they? b. Who are the AERC partners for which this is intended? 12. What would happen if USAID funding were to stop? Would something significant be missed? Would other donors pick up the ball? 13. What has USAID’s contribution been to the project? 14. Do you have any other comments or observations related to the project that you would like to share? SAGA MID-TERM EVALUATION REPORT 81 SAGA Interview Protocol – Researcher Name: Date: Position: Institution: Country: Research Support: 1. What kind of support did you receive from SAGA and how was it provided? 2. What was/is the timeframe that you received support through SAGA? 3. Is this the type of support that you feel would have the greatest impact? If so, why? If not, why not, and what other types of support would be beneficial? 4. In what specific ways do you feel the research support could be improved? Practice / Application: 5. What have been the major impacts/effects of your work? What specific changes have occurred resulting from or supported by your work? 6. Do you feel these changes will last, even when your involvement / support ends? Why/why not? What can be done to ensure that the changes will last? 7. Do you feel that research conducted under the project can help to strengthen economic research capacities? Why/why not? 8. Can it influence policy? Why / why not? What changes might be made that would strengthen the linkages between research and policy decisions? 9. Are there additional areas of research you feel SAGA should address and fund in the future? 10. Do you feel the project has been instrumental in helping researchers and research centers network and build relationships with one another? If yes, please describe. 11. What would happen if USAID funding were to stop? Would something significant be missed? Would other donors pick up the ball? 12. What has USAID’s contribution been to the project? 13. Do you have any other comments or observations related to the project that you would like to share? SAGA MID-TERM EVALUATION REPORT 82 Interview Policymakers Name: Date: Program: Institution: Country: 1. In what ways have you interacted with / been involved with SAGA? 2. Have you ever been consulted regarding the choice of SAGA research proposals? The way the research has been organized? How the results have been disseminated or used for policy purposes? If the answer to any of these is yes, please elaborate? 3. Given your experience with SAGA, in what ways do you think the project has been most effective? Least effective? 4. Do you feel that the objectives of the SAGA program (go over them very briefly) are appropriate and offer the kind of support primarily needed? 5. Do you feel the research conducted under the project can help to strengthen economic research capacities? 6. Can it influence economic policy? If yes, in what ways? If no, how might the research be reoriented to have a greater influence on policy? Is there a need for more and better research in support of policy? 7. What do you feel are the key constraints or opportunities for economic -policy researchers to have their work utilized and considered by policymakers? 8. Do you think the project has had/will have a lasting impact? If so, what will the impact be? 9. Given the African context, what factors do you see supporting sustainability of policy-oriented research and what factors make sustainability a challenge? 10. What other forms of support do you think would be most beneficial to strengthen academic institutions, research institutes, and individual researchers? 11. What would happen if USAID funding were to stop? Would something significant be missed? Would other donors pick up the ball? 12. What has USAID’s contribution been to the project? 13. Have you used SAGA related research to inform or design your policies? 14. What advice would you give to researchers/institutes to have their work utilized and considered by policy makers? SAGA MID-TERM EVALUATION REPORT 83 SAGA Interview Protocol – USAID Mission Name: Date: Position: Country: 1. How familiar are you with the SAGA program and its goals and objectives? 2. Do the implementing partners initiate discussions with this Mission regarding their research and other objectives? 3. How involved is the Mission with SAGA activities, i.e. in terms of setting research agendas, participating in collaborations or providing additional financing, etc? Please describe. 4. Do you feel that the project has facilitated the Mission’s efforts to strengthen local economic research institutes and/or provide assistance to African economists and social scientists? Please describe how. 5. To what degree is the Mission interested in participating in SAGA efforts to disseminate research and provide TA through conferences or workshops? 6. Has the project been able to effectively design or adapt research projects to better serve the needs of the Mission and government policy makers? Please explain. 7. Do you feel that the implementers have been responsive to needs of the Mission and worked well with Mission staff to coordinate activities? Please elaborate. 8. To what degree to you feel that the project is able to improve local capacity for conducting quality research and policy analysis? 9. Do you have any other comments or recommendations about the project? SAGA MID-TERM EVALUATION REPORT 84 SAGA Interview Protocol – CMAP/CPP Alumni/Students Name: Date: Program: Institution: Country: 1. (Alumni): When were you in the MA/PhD program? / (Current student): When did you start the MA/PhD program? 2. What aspects of the program did/do you find most useful? Least useful? 3. Do you feel that the program adequately prepared you / is adequately preparing you for a career in this field? Please explain. 4. In what ways do you feel the program could be improved? 5. Do you feel there are any factors or conditions that limit the program’s effectiveness? Please explain. For Alumni: 6. What are you doing now? (Note: if working, ask: Do you believe that this graduate program assisted you in securing a job that otherwise you may not have gotten?) 7. In what specific ways have you actually applied what you have learned? Please describe. For All: 8. Do you have any other comments or observations related to the graduate program that you would like to share? 9. What would happen if USAID funding were to stop? Would something significant be missed? Would other donors pick up the ball? 10. What has USAID’s contribution been to the project? SAGA MID-TERM EVALUATION REPORT 85 ANNEX D: WEB-BASED SURVEYS SAGA Project Partner Institution Survey Please answer the following questions to the best of your ability. 1. What is the name of your institution? 2. Which organization provides you support? Cornell SISERA Both 3. What kind of support does your organization receive? Please check all that apply. · Technical Assistance (please specify) ß Grant proposal preparation and review ß Training courses or workshops ß Communications and outreach strategies ß Other · Research Fellowships · Institutional Strengthening (please specify) ß Core institutional support ß Support for collaborative thematic research ß Assistance in your networking efforts ß Support for the mobility and training of researchers ß Assistance to help improve management ß Other · Collaborative research projects with U.S. scholars · Other (please specify) 4. Our institution has benefited from the support we received from SAGA Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments on how your organization has or has not benefited. 5. I feel that the support we received strengthened linkages between African and US researchers Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. SAGA MID-TERM EVALUATION REPORT 86 6. I feel the support we received strengthened the managerial capabilities of our institute Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 7. I feel the support we received has been instrumental in helping researchers and research centers network and build relationships with one another Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 8. I feel the support we received strengthened researchers economic research capacities Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 9. In what specific ways do you feel the support you receive from the project could be improved? 10. What research areas do you feel the project should address in the future? SAGA MID-TERM EVALUATION REPORT 87 SAGA Project Researcher Survey Please answer the following questions to the best of your ability. 1. Please list the country where you are conducting your research 2. Is the research support you receive for (check all that apply): ß Collaborative research ß Comparative research ß Thematic research ß Special workshops ß Other (please specify) 3. The research support I receive under this project has helped me expand my economic research capabilities. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 4. I feel the research support I receive has expanded my professional opportunities Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 5. I feel the research support I receive has helped me build links with policy makers who are interested in my research Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. SAGA MID-TERM EVALUATION REPORT 88 6. As a result of this project, I have been able to network with other researchers and/or possible collaborators. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 7. I have published research that I have conducted as a result of this project Yes No Please provide any comments here 8. Are there additional areas of research you feel SAGA should address and fund in the future? Yes No If yes, please list these research areas. 9. Overall, the support that I have received through this project has been: Extremely Poor Below Average Average Above Average Excellent 1 2 3 4 5 Please provide comments here. 10. In what specific ways do you feel the research support could be improved? SAGA MID-TERM EVALUATION REPORT 89 SAGA Project Researcher Survey Please answer the following questions to the best of your ability. 1. Please list the country where you are from: Benin Cameroon Congo Cote D’Ivoire Ethiopia Gabon Ghana Guinea Kenya Nigeria Senegal Togo Uganda Other: 2. I attended an ILEAP workshop in: ß Ghana OR Kenya 3. Please check your profession: ß Academic or research center researcher ß Government policy maker ß Government policy analyst ß Business person ß Consultant ß Other: 4. The training / assistance ILEAP provides under this project has helped you enhance your trade negotiation skills. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 5. Do you receive research support from ILEAP? Yes No 6. If you do receive research support, the research support you receive has enhanced your capacity to undertake policy relevant research. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. SAGA MID-TERM EVALUATION REPORT 90 7. Due to the training/assistance from ILEAP, you have expanded your understanding and skills of the strategies and issues necessary to engage in trade/service negotiations. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 8. ILEAP’s support has played a role in ensuring coherence between trade negotiations and other domestic policies Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 9. As a result of this project, you have been able to network with other researchers, policy makers and/or possible collaborators. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here. 10. If USAID stopped funding this project, this work would not be affected. Strongly Disagree Disagree Undecided Agree Strongly Agree na/don’t know 1 2 3 4 5 Please provide comments here of how it would or would not be affected. 11. What specifically does USAID contribute to this project? 12. Are there additional areas of research you feel ILEAP should address and support in the future? Yes No If yes, please list these research areas. 13. Overall, the support that you have received through this project has been: Extremely Poor Below Average Average Above Average Excellent 1 2 3 4 5 Please provide comments here. 14. In what specific ways do you feel the support could be improved? SAGA MID-TERM EVALUATION REPORT 91 SAGA Project CMAP Alumni Survey Please answer the following questions to the best of your ability. 1. When were you a student in the program? 2. Which of the following institutions did you attend? · Addis Ababa University, Ethiopia · University of Zimbabwe, Zimbabwe · University of Nairobi, Kenya · University of Botswana, Botswana · University of Dar es Salaam, Tanzania · University of Malawi, Malawi · University of Ghana, Ghana · Makerere University, Uganda · National University of Lesotho, Lesotho · Moi University, Kenya · University of Sierra Leone, Sierra Leone · University of Namibia, Namibia · University of Cape Coast, Ghana · University of Swaziland, Swaziland · University of Zambia, Zambia · Kwame Nkrumah University of Science & Technology, Ghana · Kenyatta University, Kenya · Egerton University, Kenya · University of Mauritius, Mauritius · University of Eduardo Mondlane, Mozambique · University of Liberia, Liberia · Other: 3. How would you rate the quality of instruction at the institution you attended? Extremely Poor Below Average Average Above Average Excellent 1 2 3 4 5 Please provide comments on quality here. 4. How would you rate the quality of instruction at the JFE? SAGA MID-TERM EVALUATION REPORT 92 Extremely Poor Below Average Average Above Average Excellent 1 2 3 4 5 Please provide comments on quality here. 5. I feel that CMAP adequately prepared me for a career in economics. Strongly Disagree Disagree Undecided Agree Strongly Agree 1 2 3 4 5 Please provide comments on how the program did or did not prepare you here. 6. Are you currently working in a job that is related to the field of economics? Yes No What is the work that you are doing? 7. I feel that I have been able to apply what I have learned in the program in my job. Strongly Disagree Disagree Undecided Agree Strongly Agree 1 2 3 4 5 Please provide comments here. 8. Do you feel there are any factors or conditions that limit the program’s effectiveness? Yes No Please explain. 9. In what ways do you feel the program could be improved?