1 ABSPII MIDTERM REVIEW February – May 2006 Review Team Julian Adams John Howard Jim Lorenzen Paul Teng June 1 2006 2 A. INTRODUCTION The project document for the USAID Leader Award # GDG-A-00-02- 00017-00, entitled Agricultural Biotechnology Support Project II, stipulates that an external technical project review will be conducted at the end of the second year and during year 5 of the project. For each review at least two acknowledged authorities would visit several sites in Africa and Asia. This document reports on the findings, conclusions and recommendations of the mid-term evaluation mission conducted during the specified periods between February and May 2006. Evaluation team members visited with USAID personnel, research personnel and other country stakeholders in Washington D.C., and selected countries in Africa and Asia. A.1 Background and context USAID has been funding efforts in agricultural biotechnology for over fifteen years. Such efforts have included the Agricultural Biotechnology Support Project (ABSPI), led by Michigan State University from 1990-2002, which took an integrated approach to biotechnology, providing support for research as well as the range of policy issues (IPR, technology transfer and biosafety) in developing countries. Recognizing that the issues associated with biotechnology product development and dissemination, namely technology development, policy development, and commercialization, have become increasingly complex, and recognizing the need to draw on experts from various fields to provide appropriate technical assistance, the Agency split its portfolio of biotechnology activities into two major global efforts in 2001– one focused on technology development and the other on biosafety policy development. The resultant Agricultural Biotechnology Support Project II (ABSPII) has a mandate primarily for technology development and for policy development in issues pertaining to intellectual property rights (IPR) and technology transfer. The sister project, the Program for Biosafety Systems (PBS) has a mandate to focus on biosafety policy issues. USAID issued Requests for Applications (RFA) in early 2002 for the two projects, and from the responses, selected the consortium led by Cornell University for ABSPII and a CG center, initially ISNAR and subsequently transferred to IFPRI, for PBS. The cooperative agreements for these projects were effective in 2002/2003. In 2004, biosafety efforts in India and Bangladesh were transferred from PBS to a new project, the South Asia Biosafety Project (SABP), which is implemented in a partnership between IFPRI and AgBios (Merrickville, Ontario Canada). 3 Broadly, the RFA had stipulated four program components, namely  Technology development,  Policy activities on intellectual property rights and technology transfer,  Biotechnology commercialization, and  Biotechnology outreach and communication. ABSPII was tasked to develop partnerships to undertake activities in these under the overall USAID Collaborative Agricultural Biotechnology Initiative (CABIO). The selection criteria in the RFA were systems framework design, strength of the partnership, and potential to create an enabling policy environment. The successful respondent to the RFA, the Cornell University consortium, comprised strong representation from eight U.S. public institutions, nine private sector entities, six international agricultural research centers, six NGOs/foundations, and seventeen national/regional partner organizations. The Cornell University-led proposal also had a transparent process of prioritizing GM technologies into product commercialization packages which formed the core set of activities around which policies, outreach and communication could be leveraged to ensure impact. The ABSPII portfolio of technologies proposed by the Cornell University Consortium and accepted by USAID, included the following: 1. Fruit and shoot borer resistant eggplant 2. Late blight resistant potatoes 3. Drought and salt tolerant rice 4. Tobacco streak virus resistant sunflower and groundnuts 5. Papaya resistant to ringspot virus 6. Multi-virus resistant tomatoes 7. Bananas resistant to Sigatoka and nematodes 8. Pod borer resistant chickpea 9. Cassava resistant to mosaic disease 10. Sweet potatoes resistant to feathery mottle virus ABSPII is a five-year Leader with Associate (LWA) Cooperative Agreement active in Asia and Africa. The country specific programs in Asia include India, Bangladesh, Indonesia and the Philippines, while in Africa the current country￾specific programs are located in Uganda and Mali. ABSPII also has a mandate to develop sub-regional approaches to biotechnology in Africa to complement the Agency’s ongoing investment in sub-regional agricultural organizations to promote greater dissemination of technology and make effective use of scarce donor and African resources. The LWA form of a cooperative agreement was 4 introduced by USAID several years ago to provide additional flexibility in the Agency’s procurement mechanisms. The benefit of the LWA is to provide the opportunity for Missions and regional bureaus to support specific activities within the broader scope of the Leader cooperative agreement through Associate Awards, which require reporting specifically on the scope of the individual associate award. Funding for the Leader component of the ABSPII agreement is provided through a combination of funding from USAID, including the Office of Environment and Science Policy, USAID regional bureaus and field Missions. In addition, the project receives funds through two Associate Awards to support the Uganda and Mali programs. The overall goal of ABSPII is to use biotechnology as a tool to address key constraints to agricultural productivity to promote economic growth, improve environmental quality and achieve food security in developing countries. To achieve this, the project is expected to integrate technology development by creating an enabling policy environment for biotechnology product development. In addition, ABSPII is to work in coordination with other USAID￾funded efforts in providing technical assistance for the development of biosafety policies to ensure a consistent approach to these issues with recipient country governments. While the primary focus of ABSPII has been in specific technology development projects, the program has undertaken other activities, including a range of communication activities, intellectual property rights activities and regional priority setting in East and West Africa. A.2 Purpose of the Mid-term Evaluation The goal of this mid-term evaluation is to examine the impact of ABSPII against USAID objectives. Further, this evaluation will provide external guidance and advice to assist ABSPII in strengthening the program to increase likelihood of success to achieve overall Agency objectives. Specific details of the Statement of Work (SOW) are provided in Appendix 1. A.3 Team composition The Evaluation Team comprised: 5 Julian Adams, Professor of Molecular, Cellular and Developmental Biology, and of Ecology and Evolutionary Biology, University of Michigan, 830 N. University, Ann Arbor MI 48109-1048; John Howard, President, Applied Biotechnology Institute, San Luis Obispo, California. James Lorenzen, Associate Professor, Department of Plant, Soil and Entomological Sciences, University of Idaho, Moscow, ID 83844 Paul S. Teng, Professor and Head, Natural Sciences & Science Education, National Institute of Education, Nanyang Technological University, 1 Nanyang Walk, Singapore 637 616. A.4. Itinerary and Schedule of work The mid-term review took place during the period 8 February to 31 May 2006. A.4.1. Briefing, Washington D.C. The first activity of the Mid-Term Project review team was a briefing in Washington D.C. between February 8-10 2006, attended by three of the four team members – Julian Adams (attended day 1 only), John Howard and James Lorenzen. Meeting Agenda was as follows: Wednesday, February 8 10: 00 am 10:30am Opening remarks by Frank Shotkoski Fruit and shoot borer-resistant eggplant: Vijayaraghvan 12:30pm Review Team meets for lunch with Ray Wu 1:30pm Drought and salt tolerant rice: Ray Wu 2:30pm Virus resistant tomato in West Africa: Molly Jahn and Bob Gilbertson 4:00pm Coffee Break & meeting with Molly Jahn 4:30pm Review Team meets with the ABSPII team Thursday, February 9 9:00am Late blight potato by K.V. Raman and Sandra Austin-Phillips 10:45am Socio-economic assessment presentation by George Norton 12:00pm Lunch with the Review Team 6 1:00pm Tobacco streak virus-resistant groundnut by Roger Beachy and Vijayaraghvan 2:15pm Multiple virus-resistant tomato and papaya ringspot virus￾resistant papaya by Desiree Hautea 4:15pm Communication and outreach by Andrea Besley Friday, February 10 9:00am - Review Team meets with USAID The team then broke into sub-teams to visit different cooperating institutions and sites. A.4.2. Visit to Mali by John Howard and Julian Adams between February 12 – 16 2006 Throughout the visit to Mali, Drs. Howard and Adams were accompanied by Dr. F. Shotkoski. February 12. Informal discussion with Dr. F. Shotkoski February 13. Visit to Mali USAID Mission. Discussions with the Director of the Mission, Alex Newton and Dr. Ram Shetty, Also present were Dr. Harry Bottenberg, Gregory Vaut and Robert Kagbo. February 14. Visit to the Institut d’Economie Rurale in Sotuba near Bamako, met with Madame Gambey, the Director, and with Dr. Dembélé, the Scientific Director. The rest of the day was spent inspecting the tomato variety trials in various locations near Bamako. February 15. Visit to the AVRDC site, located at ICRISAT. Met with the local Director of ICRISAT, and Drs. Virginie Levasseur and Issoufou Abdourhamane. February 15. Met with the Minister for Rural Economy in Bamako. February 16. Visits to the University of Mali in Bamako, and the Laboratoire Central Vétérinaire. Meetings with the Director General of the LCV, Dr. Tembely, and with Dr. Oumar Niangado, a previous Director of the Institut d’Economie Rurale, (now with the Syngenta Foundation). A.4.3. Visit to India by John Howard, Julian Adam, & Paul Teng February 18. Group presentation by ABSPII South Asia team at Hotel Taj President Lands End, Mumbai. February 19. Dinner with Vice Chancellor, Tamil Nadu Agricultural University Coimbatore 7 February 20. Visit Tamil Nadu Agricultural University to meet with scientists and view biosafety greenhouses for transgenic eggplant research. Visit farmers’ fields to view eggplant fruit/shoot borer infestations and damage. February 21. Visit to Directorate of Rice Research, Hyderabad and ICRISAT, Patancheru. February 22: Leave for Aurangabad from Mumbai. Visit MAHYCO research center, Jalna and the field station. February 23. Leave for Delhi in the evening. February 24. Visit with Dr. Lawrence Paulsen, USAID Mission, New Delhi. A.4.4. Visit to Indonesia by J. Lorenzen January 26. Visit to Indonesian Center for Agricultural Biotechnology and Genetic Resources Research and Development (ICABIOGRAD), Bogor; consultations with R. Coffman and ABSPII SE Asia team. January 27. travel to Lembang, visit Indonesian Vegetable Research Institute (IVEGRI). A.4.5 Visit to India by J. Lorenzen February 18. Group presentation by ABSPII South Asia team at Hotel Taj President Lands End, Mumbai (together with Julian Adams, John Howard, and Paul Teng). February 19. Depart for Simla. February 20. Visit CPRI, Shimla (full-day program, facilities tour, consultations). February 21. Travel to Modipuram, visit CPRI Modipuram, return to New Delhi. A.4.6. Visit to the Philippines by J. Adams January 19. Visit to Institute of Plant Breeding, University of the Philippines at Los Baños (UPLB-IPB), review of project activities and facilities. Meet with UPLB Chancellor, Luis Rey I. Velasco. Group Presentation by ABSPII Philippine team. Meeting with Estrella Alabastro, Secretary, Department of Science and Technology, the Philippines, and Chair ABSPII Advisory Board. A.4.7. Visit to the Philippines by John Howard and Paul Teng. February 26. Visit Institute of Plant Breeding, University of the Philippines, Los Baños project activities and facilities. Group presentation by ABSPII Philippine team. 8 February 27. Visit with Philippine Council for Agriculture, Forestry and Natural Resources Research and Development (PCARRD) Executive Director P.S. Faylon; Visit with the UPLB Chancellor, Luis Rey I. Velasco, and UPLB Executive Director, Cecilio R. Arboleda. Visit with Chair of UPLB, Institutional Biosafety Committee, and a member of the National Biosafety Committee, Philippines A.4.8. Discussion meeting in Washington D.C. by J. Adams, J. Howard and J. Lorenzen between 26-28 April to plan preparation of review document. B. EVALUATION OF MANAGEMENT ISSUES B.1 Overview The review team was favorably impressed with several aspects of the management of the ABSPII project. We felt that ABSPII/Cornell University has done an excellent job of introducing the concepts of product development, regulatory, intellectual property production and public relations into the program. The priority setting exercise appeared to have the appropriate amount of rigor for this stage of the project and has been followed up with an active participatory ex ante analysis of potential socioeconomic impact. Although there may inevitably be arguments concerning the initial portfolio of projects selected, the review team was generally satisfied that a good faith effort was made to obtain the appropriate information required to arrive at the initial list. In addition, ABSPII/Cornell University should be commended for establishing linkages with local country groups to determine their priorities, and to assist in the selection of projects. Our impression is that the ABSPII project is favorably regarded in many of the locales we visited and credit for that impression must go to Cornell University and the general high regard which this university enjoys world-wide. This is an important factor that has enabled the project to build up its credibility and achievements to date. It is important that the catalytic role provided by the involvement of Cornell University be substantiated by a focus on substantive progress as well as on buy-in from partners – which appears to be significant in both India and the Philippines. Intellectual Property (IP) issues (Program Component 2) were apparently incorporated during the initial selection of the projects. However, the review team was not provided with enough information to determine if these (IP issues) were adequately and comprehensively addressed. The review team notes that since the beginning of ABSPII, three projects have been discontinued, pod borer resistant chickpea, mosaic disease resistant cassava, and feathery mottle virus-resistant sweet potato. Decisions to discontinue the chickpea and sweet potato project were taken at the 2005 board 9 meeting. It is not clear from the material presented why the cassava project was dropped from the portfolio particularly as the board recommended continuing this project in 2005. The review team considers it important to review the portfolio of projects at regular intervals; to consider additions to the portfolio, as well as removals for those projects where significant progress has not been achieved. Consequently the review team considers that these steps provide evidence that there was an evaluation for these selected early stage projects. The review team has not been provided with sufficient information to evaluate the basis of these decisions. The review team was also favorably impressed with regional coordination of some of the projects in South and South East Asia. For example, coordination of Bt eggplant project in India, Bangladesh and the Philippines, with MAHYCO playing a central role, brings an extra dimension and allows for synergistic interactions between the three country-specific Bt eggplant projects. Management of the projects within India and Bangladesh has been sub￾contracted to Sathguru Management Consultants Pvt. Ltd. with headquarters in Hyderabad, India. The review team was very favorably impressed by Sathguru’s day-to day, and month-to month management of the India and Bangladeshi projects. There appears to be excellent communication between Sathguru and the individual principal investigators and interested parties in India and Bangladesh. Sathguru utilizes the MS-PROJECT project management system, which appears to enable timely tracking of tasks, at several levels of detail, and provide the appropriate information to interested parties. The review team was provided with MS-PROJECT printouts, to illustrate the level and detail of project monitoring that Sathguru provides. We were informed that this system, already used for the India and Bangladesh projects, will also likely be adopted by the S.E. Asia team. The level of management provided by Sathguru does not include scientific leadership, nor was it intended to. Regional management in South East Asia (the Philippines and Indonesia), provided by Dr. Desirée Hautea, is similarly considered to be excellent. In distinction from the management activities provided by Sathguru in South Asia, Dr. Hautea lends scientific leadership to the projects as well as management expertise, such as detailed tracking of tasks and information dissemination. In Mali, local management of the tomato project is provided by Dr. Levasseur. The review team was impressed with Dr. Levasseur’s energy and commitment to the project. 10 We congratulate Cornell University on their achievements to date in leading ABSPII but we do have some concerns, which we describe below. B.2 Project Portfolio Although the criteria used for project selection appeared to be reasonable, we are concerned that the final mix of projects, was weighted heavily towards “proof-of-concept” projects, and not enough towards projects that were in later phases – product development and commercialization. Thus, the review team considered that only two projects, Bt eggplant and PRSVR Papaya are beyond “proof-of-concept ” stage, and are likely to be commercialized either in the lifetime of ABSPII, or shortly thereafter. Since technology development and product commercialization and the associated IP and Technology Transfer activities represented three of the four program components in the RFA, the review committee considers it unfortunate that majority of the projects are in the “proof-of-concept ” stage and in some cases in the very beginning of the “proof￾of-concept” stage (e.g., PYV resistant tomatoes in Mali, and DST Rice), and not likely to be commercialized within the intended time frame1. B.3 Budget Issues The review team was concerned that too large a proportion of the overall budget has been allocated to management, and too little has been allocated to individual projects for research and product development. The review team considers that minimal central management should be required for the majority of projects which are in “proof-of-concept” phase, beyond yearly (or longer) evaluations of progress of the individual projects through this initial stage. The review team was aware that this issue has been the subject of discussion between USAID and ABSPII. However we found it difficult to determine the extent of the reduction of management costs, as requested by USAID in 2005, as accounting practices have apparently changed over the last year. We understand that in earlier years Sathguru management costs were folded into to the overall management budget, whereas now they are budgeted separately in the respective project costs. B.4 Ongoing Project Management Although the review team was satisfied that the process used to arrive at the initial portfolio of projects to be funded was sound, the review team did not 1 The RFA mentions 10 years. 11 see any strong evidence of coherent and overarching scientific leadership beyond the initial startup period for ABSPII. Thus, we could find no information indicating that ABSPII management exercised scientific leadership by engaging in a continuing evaluation of the projects. Rather, the annual meetings of the advisory board appear to have been utilized for the assessment of progress of the individual projects. The review team considers it important to review the portfolio of projects at regular intervals by smaller more focused teams. Furthermore, the review team is concerned that resources and ancillary support appear to have been allocated to all projects regardless of their status or phase. Thus, the review team considers it premature to devote resources and effort to outreach and communication for projects in the “proof-of-concept” stage. We recognize, however, that communication is needed for projects which have clearly gone into a product development and likely commercialization phase. Unrealized public expectations for the efficacy or value of a particular product can have a significant negative impact on the future acceptance of other transgenic crops. The review team is aware that the leadership of ABSPII has undergone three changes since its inception, and that the present director, Dr. Frank Shotkoski has only occupied this position for one year prior to this review. The review team is therefore reluctant to ascribe the lack of evidence of an overarching and coherent scientific leadership to any shortcomings of either the incumbent Director or of either of the two previous directors of ABSPII. Rather, it may simply be a consequence of the frequent changes in leadership over the last three years. B.5 Nature of the Cooperative Agreement between ABSPII and USAID Funds to support ABSPII were awarded to Cornell University to be administered as a “cooperative agreement” between ABSPII and Cornell University. USAID regulations specify that a “cooperative agreement” include provisions for the substantial involvement of USAID in decisions relating to the execution of the work specified in the RFA. However, it has become clear to the review team that the nature of the relationship between the ABSPII leadership and USAID has not approached the spirit or the letter of a cooperative agreement, almost since the inception of ABSPII. Rather, the ABSPII leadership has been operating as if USAID had awarded a grant, rather than a cooperative agreement, to carry out the work. The review team considers that the lack of cooperation between the two entities has in many cases had a significant negative impact on the ability of ABSPII to accomplish its goals. We understand however, that the more complex the scope of a project, the more difficult it may be to operate within USAID regulations for a cooperative agreement. The review committee did not consider that it was part of their charge to determine the genesis or to assign responsibility for this lack of cooperation. Nor do we think it is profitable, or constructive to do so. Rather, in the following section, we 12 suggest a modified management structure which may encourage and support a cooperative arrangement, and bring both entities back into conformity with specifications of a cooperative agreement2, and in so doing, with USAID regulations. B.6. Relations of Cornell University with Partner Institutions Although Cornell University possesses considerable intellectual resources, the global nature and wide ranging scope of ABSPII inevitably requires partnerships with other institutions, both public and private to achieve its goals. The proposal submitted by Cornell University listed a number of such partners. However, with a few notable exceptions (e.g., Sathguru, MAHYCO), the review committee is concerned that implementation of some of the individual ABSPII projects has not taken sufficient advantage of these other resources. For other projects, the review committee is concerned that project design and progress could have been significantly advanced by the inclusion of investigators, groups of investigators or institutions not initially mentioned in the proposal. It is thus somewhat difficult to escape the conclusion that for some projects, the choice of implementing investigators or groups of investigators was determined more by their direct association with Cornell University than by their ability, or by their national or international stature. As a consequence, the review committee questions whether the association with Cornell University was a factor in determining the projects selected for support during the initial phase. B.7. Global and Regional Cooperation Although, as we mention above, the review team was favorably impressed with regional cooperation in Asia, we were disappointed to see that the ABSPII initiatives in Africa are proceeding in apparent complete independence, and so lack any synergy or “value-added” benefits that could be gained. We note that the tomato projects in Mali and the Philippines bear considerable similarity to each other; both involve virus resistance to two different viruses, and the first goal for both projects is to develop geminivirus resistant varieties. The review committee felt that an opportunity for profitable interaction, and South-South cooperation is being lost for these projects. 2 We use the terms “Cooperative Agreement”, “Contract” and “Grant” as defined by USAID. 13 C. EVALUATION OF INDIVIDUAL PROJECTS C.1. Bt Eggplant ABSPII has been supporting the development and commercialization of Bt eggplant. Three countries are involved in this work; India, Bangladesh and the Philippines, with India taking the lead role, as eggplant is a particularly important vegetable crop in India. This project is a cooperative venture involving a commercial enterprise, MAHYCO, as well as University partners, most notably Tamil Nadu Agricultural University. The field trial phase is well￾advanced, and MAHYCO expects that a Bt eggplant variety will be commercialized in India at the latest by June 2007. Bt eggplant is the most advanced of all the product development plans in ABSPII – South Asia and potentially could be the showcase product from this project in the near term. The project has achieved its major milestones in a timely manner since the first contact between partners in 2003. The concept of conferring non-tissue specific resistance to two lepidopteran borers is proven through a high level of resistance and subsequent bioassays using the Cry1Ac gene (for which Monsanto holds the patent rights) transformed into MAHYCO genotypes. Field efficacy trials approved by the Review Committee on Genetic Modification (RCGM) were conducted at 11 locations in India using the Bt-MAHYCO genotypes in 2004. No field trials have yet been carried out in Bangladesh and the Philippines. First backcross seedlings (BC1) using local, farmer-desired varieties (10 from India, 19 from Bangladesh, and three from the Philippines), are in greenhouses of several laboratories as of November 2005. Freedom to Operate (FTO) reviews were reported to have been completed and apparently show no IP issues related to the Public-Private partnership for technology transfer and subsequent commercial and public sector release of the transgenic varieties. (We note that review team was not provided with materials relevant to these issues.) A regulatory framework is in place in India and the Philippines by which final approvals required to release the hybrids and/or open-pollinated varieties (OPVs) may be obtained. Safety studies completed in 2005 in India for regulatory compliance include sub-chronic oral toxicity study, allergenicity studies, goat study, cow feeding study, rabbit study, alkaloids and amino acid profiling, and second year trials conducted by the Indian Council of Agricultural Research (ICAR). The team was very impressed with the high quality containment facilities in all sites visited, and the pragmatic approach to biosafety that has resulted from close liaison with regulatory authorities at all 14 stages. The project appears on target to obtain final regulatory clearances and commercialize Bt hybrids in 2007 and OPVs in 2008 in India. The assumption is that Bangladesh will follow soon after India and use Indian data for approval. It is significant that both Bangladesh and the Philippines will be willing to accept regulatory packages generated in India for their own approval systems. For the Philippines, a small deviation of 1-2 months from the anticipated timeline to obtain BC2 material was due to further approval required by The Convention on Biological Diversity to ship BC1 material from India to Philippines. Ex ante socio-economic studies show potential for value creation in hybrids and OPVs. The review team did not see evidence of a strategy for resistance management or of using Bt eggplant in an integrated pest management (IPM) scheme. This has to be flagged as a potential area of concern since past experience with unilateral pest control methods in vegetables shows that the evolution of resistance is likely to occur sooner without practicing IPM. The project appears well managed by both MAHYCO and by Sathguru. The review team had no criticisms of the management of the execution of this project. However, it should be pointed out that this project was initiated by MAHYCO for India in 1999, well before the beginning of ABSPII. ABSPII is contributing to the funding of the last stages in the commercialization of Bt eggplant - the field trials and the construction of the regulatory package for the Government of India. MAHYCO estimates that the cost of the regulatory package for a new event is $3,000,000 - $5,000,000, and is contributing most of the costs. ABSPII’s support to MAHYCO for the commercialization of Bt eggplant amounts to ~$100,000/year. Given the high costs to put together a regulatory package it is not realistic to expect ABSPII to entirely fund the regulatory package. However the question must be asked, is the ABSPII funding for Bt eggplant resulting in either i) the commercialization of a transgenic crop which would not have otherwise be commercialized, or ii) the expediting of the commercialization process. MAHYCO had told us that they would have proceeded to commercialization with or without ABSPII contributions. Therefore, given the early and continuing interest by MAHYCO in the Bt eggplant, we conclude that the answer to the first question must be “no”. Furthermore, given the considerable resources invested by MAHYCO, it is unlikely that ABSPII’s contribution will expedite the commercialization of Bt eggplant hybrids. Therefore, the primary benefit of ABSPII involvement is less the initial commercialization of the hybrid Bt eggplant in India, and more the broadening of the MAHYCO effort to aid in the introduction of open-pollinated cultivars developed by public institution, and the extension to Bangladesh and 15 the Philippines. These are useful and noteworthy accomplishments, creating model private-public and south-south partnerships. C.2. Virus Resistant Tomatoes, Mali, SE Asia There are two projects that involve multiple virus resistance in tomato, in both cases dealing with natural host-plant resistance to a geminivirus (tomato yellow leaf curl virus; TYLCV) as the primary problem and genetic engineering for a virus of secondary importance, a potyvirus (PVY) in the case of W. Africa and a cucumovirus CMV in SE Asia. Evaluation of these two “separate” projects is combined in this section, consistent with the team recommendation that they be coordinated (see section D. Future Recommendations) ABSPII has been supporting a project to develop a transgenic virus resistant tomato variety for Mali, West Africa. Currently the Mali tomato industry is suffering severe losses due to two similar but recognizably different virus infections caused by Tomato leaf curl Mali virus (TLCMV) and Tomato yellow leaf curl virus (TYLCV). Both these diseases are caused by geminiviruses. Natural host-plant resistance to geminiviruses exists in tomatoes. While we (JA & JH) were in Mali we visited several locations in the vicinity of Bamako where field trials of ~40 varieties of commercial varieties of tomatoes were being carried out in order to evaluate which, if any of the varieties exhibited natural resistance to the geminiviruses present in Mali, and which also would be adapted for growth under Malian conditions. We understand that additional trials are being conducted in neighboring West African countries, for example Benin, Togo and Ghana. Initial indications from the field trials in the vicinity of Bamako are that some established varieties exhibit resistance to the geminiviruses found in Mali. Assembly of the seed of the commercial varieties and the design and execution of the field trials has been, and continues to be under the direction of Professors Molly Jahn (Cornell University until summer 2006) and Robert Gilbertson (University of California at Davis). It is noted that Professor Gilbertson has already been working in Mali for the last three years on geminivirus infections, supported by other USAID funds. The team considers that this is very promising and valuable research, with very reasonable probability of success and useful side-benefits in infrastructure development and personnel training. Identification of one or more pre-existing commercial varieties of tomatoes adapted for growth in West Africa is however an intermediate goal of this project. The next goal will be to develop a variety of tomatoes which is resistant both to geminiviruses and potyvirus in West Africa. No natural host-plant resistance to potyviruses has been identified in tomatoes. However, naturally resistant varieties of peppers (another member of the Solanaceae) are known, as 16 are accessions from several potato species. A future goal of this project is to transfer a cloned resistance factor from pepper to tomatoes using tested transgenic technologies. Although potyvirus resistance is determined by a recessive allele in pepper, this gene is effectively dominant when overexpressed as provided by Dr. Jahn's construct. Professor Jahn has already created transgenic tomato plants in her laboratory at Cornell University which contain potyvirus resistance determinants. This work will need to be repeated for the varieties chosen once the West Africa trials are completed, with subsequent transfer to geminivirus-resistant lines performed via independent transformation or backcrossing. However, the potyviruses represent a large virus family and the review team was not presented with efficacy data for the virus strains likely to be encountered in W. Africa. The review team was satisfied in general from the information presented to us in Washington D.C. and in Mali, in that tomatoes represent a significant crop in West Africa, furthermore that there is extensive trade in tomatoes and tomato products among West African countries, principally the six comprising the former French West African colonies. Furthermore, export of tomatoes from West Africa to the EU is not a factor that would limit acceptability of transgenic tomatoes. Thus EU policies towards transgenic crops should not impact the socio-economic value of transgenic tomatoes in West Africa. The work underway to identify geminivirus resistant commercial varieties of tomato, suitable for commercial growth in West Africa, seems well conceived and executed, and we have confidence in the abilities of both Professors Gilbertson and Jahn to manage this project. Identification of the geminivirus infections requires application of biotechnology tools, but there is no transgenic plant component in this phase of the work. The programs in Indonesia and the Philippines plan to use marker-assisted selection to facilitate gene transfer into adapted cultivars suitable for local markets. The value of the next phase, creation of a transgenic tomato variety resistant to both potyvirus and geminiviruses, although technologically feasible, must be considered to be speculative. The review team was not presented with evidence that potyvirus infections are, at the present time, a significant threat to the tomato crop in Mali or in other West African countries. In contrast the review team saw evidence that attacks of fruit borer, Helicoverpa armigera, a lepidopteran pest of tomatoes, are already significant in Mali. Since Helicoverpa armigera is sensitive to the Bt toxin, it is not clear why the introduction of potyvirus resistance into a geminivirus resistant variety was chosen to be the next phase of this project, rather than creation of a Bt tomato strain resistant to Helicoverpa. The budget for this project appears to be quite modest. The review team learned from Professors Jahn and Gilbertson, that they receive about 17 $120,000/year to support their work in the U.S.A., and while in Mali, we learned that the West African NAR’s receive ~$4,000 to support field trials. However, the total funds available for the tomato project are $1.7 million over 3 years (letter from B. Pathak to F. Shotkoski, dated November 2 2005). We were unable resolve the differences between these figures. More information is needed on past, as well as on future projected expenditures, to fully understand the budgetary allocations. Commercialization of a multiple (geminivirus and potyvirus) virus resistant variety is considered to be several years away, as i) the choice of the parent variety, resistant to geminivirus, into which potyvirus resistance will be introduced, has not yet been made, and ii) “proof-of-concept” for the transgenic plants for the target potyvirus strain(s) in West Africa remains to be demonstrated. Furthermore, the socioeconomic value of the transgenic component of this project will depend on the extent to which potyvirus infections of the tomato crop in West Africa become a realized threat rather than a potential one. It is noted that Mali has not yet passed a Biosafety law, and that even Bt cotton may not be grown in that country. Of the neighboring countries only Burkina Faso has a directive allowing field trials. The SE Asia component of this project is slightly different. Rather than the basket of public and private cultivars being screened for geminivirus resistance in W. Africa, the SE Asia partners are evaluating two advanced resistant lines from AVRDC. These lines are being backcrossed to Philippine and Indonesian cultivars to develop locally-acceptable lines resistant to TYLCV. These programs will use molecular markers developed by AVRDC in an advanced marker￾assisted selection (MAS) scheme. This is important because the multigenic nature of the AVRDC resistant lines complicates the backcrossing scheme and enhances the value of the molecular markers. As in W. Africa, the severe and increasing nature of the geminivirus epidemic makes this an important issue for one of the most important vegetables. As such, this is good use of biotechnology to solve an economically important problem. The combined strategy of using conventional backcross breeding enhanced by MAS, along with using a proven transgenic coat protein technology (CP), gives this program a high probability of success. The CP events will also be provided by AVRDC. As in West Africa, the geminivirus issue is the primary disease threat and this will be addressed by classical breeding. The problem addressed by the transgenic technology is secondary. The importance of the secondary virus issue has not been established for the Philippines, although the project has conducted a field survey to assess the relative abundance of tomato viruses in Indonesia. 18 The relative importance of the secondary virus will determine the importance of the transgenic technology. Some additional “proof-of-concept” work will be needed here to show efficacy for local strains of CMV, and to establish the importance of transgenic plants. As previously mentioned, the Indonesia program has surveyed and archived tomato viruses. This archive is being evaluated using molecular analyses to determine diversity of the target viruses. This aspect of the project is highly commendable. It will be very valuable to have a diverse archive of characterized virus strains once resistant material is available for testing, and selection of strains for challenge tests will be facilitated by knowledge of genetic diversity. Management at the local level appeared very competent for both countries. There did not seem to be much of a role for additional layers of management at this time either from Sathguru or from Cornell University. We would recommend waiting until there is clearer evidence of commercial efficacy before investing funds on anything besides technical progress (see Section D - Future Recommendations). C.3. Drought and Salinity Tolerant Rice, India ABSPII has been supporting work by Professor Wu’s laboratory at Cornell University in developing drought and salinity tolerant (DST) rice. Professor Wu has introduced genes coding for trehalose synthesis into rice. In earlier work he employed antibiotic resistance as a selectable marker to identify transformants, using a rice variety, that was not acceptable to India as a transgenic variety. More recently, using APSPII support, he has repeated these experiments using antibiotic-marker-free technology to transform the genes into a more acceptable variety. Transformants are now being tested in a greenhouse in India at the Directorate for Rice Research (DRR) in Hyderabad. The review team believes that “proof-of-concept” has not been demonstrated and that there are three sets of concerns with this project First, it is not clear what relevance or correlation the greenhouse trials presently being carried out, have to drought and salinity conditions in the field. It is also not clear what constitutes a commercially acceptable level of drought and/or salinity tolerance. This may be different for different geographic areas and the degree to which growers are willing to accept possible yield penalties may vary. Without such information which addresses these practical concerns, it is difficult to predict whether the current laboratory observations can lead to a viably commercial product. 19 Second, no determinations have yet been made of other characteristics of transgenic rice with enhanced trehalose levels - for example taste and palatability tests as well as agronomic characteristics such as yield or germination. It is likely that there may be a yield penalty associated with expression of the genes for drought and salinity tolerance under non-stress conditions. The extent of this penalty will be critical in determining the socio-economic value of this form of DST rice. Germination levels in rice with enhanced trehalose levels may also be significant as it has been shown in other organisms that trehalose plays a key role in germination3. Third, as others have argued (and Professor Wu has indicated that he accepts these arguments), drought and salinity tolerance is a multifactorial trait, and it is therefore unlikely that introduction of gene(s) coding for trehalose biosynthesis alone will provide acceptable tolerance levels. As a consequence the review team was surprised to hear from the Acting Project Director of the DRR in Hyderabad that he expects commercialization of a drought and salinity tolerant rice variety by end of 2007/beginning of 2008. In contrast, the review team feels that the DST rice project is still in the early “proof￾of-concept stage”, and that the expected date of commercialization is unattainable. The review team finds this disconcerting, as the creation of unrealistic expectations, (and the subsequent failure to realize these expectations) does much to fuel negative public perceptions of the value of agricultural biotechnology. Furthermore, given the present knowledge of the multifactorial nature of drought and salinity tolerance, we question whether focusing in one gene for enhanced trehalose biosynthesis is the most appropriate way to proceed. C.4. RSV-Resistant Papaya The Philippines is the lead country for the Papaya ringspot virus (PRSV)- resistant papaya, which is also the most advanced of all the product development plans in ABSPII-SE Asia. It potentially could be the showcase product for this region as it is developed with a majority of local technology and “know-how”. The project has achieved many milestones since the first contact between partners in 2003 but is about a year behind schedule. The concept of coat-protein mediated virus resistance in papaya has been previously validated and commercialized in Hawaii. Unfortunately, the SE Asia strains of PRSV differ enough that new constructs had to be made to use a 3 A recent report (Satoh-Nagasawa, N. et al. 2006. Nature 441: 227-230.) indicating that a trehalose metabolic enzyme controls inflorescence architecture in maize, and appears to be conserved among the gramineae, further argues for the importance of evaluating the pleiotropic effects of increased trehalose levels in rice. 20 similar approach as in Hawaii. Initial efficacy tests with local PRSV strains appear to have validated the new technology through confirmation of gene expression and pathogenicity tests in containment greenhouses in the Philippines (achieved in 2003-04). The review team was told that Freedom to Operate (FTO) reviews in 2003 show no IP issues related to the Public-Private partnership for technology transfer, and subsequent commercial and public sector release of the transgenic varieties. As with the Bt eggplant project, we have not seen these reviews and therefore can not comment on their suitability for the current stage of this project. Confined experiments with T2 materials that have been approved by the Institutional Biosafety Committee (IBC) and the National Committee on Biosafety of the Philippines (NCBP) are ongoing at the University of the Philippines Los Baños - Institute of Plant Breeding (UPLB-IPB). Progression to T3 is being expedited using embryo rescue techniques, in addition to harvested seeds. Open field trials have yet to be conducted in the Philippines and have been delayed due to a regulatory requirement for Southern blots from 3 generations. First backcross seedlings (BC1) using local, farmer-desired varieties will only be done after efficacy trials in the field with the original transformed parents. A regulatory framework exists in the Philippines for final approvals required to release the hybrids and/or OPVs. Project personnel have contributed significantly towards the creation of this framework. The Philippines will be willing to accept regulatory packages generated elsewhere on food safety – this will contribute to considerable savings in time and money. However, because this PRSV resistant papaya product is considered a new event, additional data will still have to be obtained on the molecular characterization of the transgene, on effects on non-target species, on gene flow, on nutritional analysis, as well as on some feeding studies. The team was very impressed with the high quality containment facilities in UPLB-IPB, and the pragmatic approach to biosafety that has resulted from close liaison with regulatory authorities at all stages. There is very strong local commitment to PRSV Papaya by the Philippine Council for Agriculture, Forestry and Natural Resources Research and Development (PCARRD) and UPLB administration. PCARRD views this as a flagship project for public sector crop biotechnology in the Philippines, and has committed funding for it. The project seems on target to conduct multi-location field trials in 2007 and to obtain final regulatory clearance and commercialize PRSV plantlets in 2008. This assumes that the limited confined “field trial” in the BL2 screen-house scheduled for mid- 21 2006 to mid-2007 will result in satisfactory results. UPLB-IPB through its affiliation with the National Seed Foundation, possesses the capacity to scale up production of the PRSV-R papaya hybrids or varieties for distribution. A non-exclusive franchising agreement with East-West Seeds Company is in place for non-transgenics, and has also been proposed as the model for transgenic papaya. Ex ante socio-economic studies show potential for significant value creation. The Review team did not see evidence of a strategy for resistance management for the PRSV-resistant papaya, or plans to use PRSV Papaya in an IPM scheme. C.5. TSVR Groundnut, India Groundnut is an important oil, food, and forage crop in India, covering more than eight million hectares. This important cash crop is threatened by epidemics of Tobacco streak virus (TSV), which caused losses worth $65 M in 2000. In the absence of adequate host-plant resistance, the project has adopted a transgenic coat protein strategy for resistance to TSV. The Danforth Plant Science Center has taken the lead for development of the CP construct, using a visiting scientist from MAHYCO Research Foundation (MRF) to conduct much of the work. Research and project collaborators in India include MRF, the International Crop Research Institute for the Semi-arid Tropics (ICRISAT), MAHYCO, and Sathguru. The project selection appeared to represent a good use of transgenic plant technology. This project appears to have potential to have a significant impact on the crop and on the people in the region if it is successful. The technical progress has gone well to date although it is at a much earlier stage than most other projects because of a later start. While there appeared to be a high probability of overall success, there has been no guarantee of commercial efficacy at this stage. Management at country level appeared very competent and the funds given by ABSPII appeared to be used very efficiently. This project is still at a very early stage, and should be supported at that level, without investing in additional management or commercialization aspects before “proof-of-concept” has been demonstrated and there is clear evidence for a viable commercial product (see section D. Future Recommendations). There is much more work that will need to be funded before commercialization can happen. Although at this stage there 22 is no clear source of funding, the research needs to progress to a more advanced stage before this is demonstrated as a serious problem. One problem noted by the review team was that the public researchers did not appear to have a realistic expectation for the amount of effort and investment required to develop a regulatory package for commercialization. C. 6. Late-blight Resistant Potato The late blight resistant (LBR) potato project targets one of the most important potato pathogens in the world, using a cloned resistance gene from a wild potato species. Partner countries for this project are Indonesia, Bangladesh, and India. The “proof-of-concept” seems to have been taken as a “given” in this case. The RB gene is one of the most promising plant resistance genes to have been cloned to date with regard to commercial potential. The fact that two groups (University of Wisconsin, private Dutch group) independently cloned the same gene (the sequence of RB is nearly identical that of Rpi-blb1) from similar sources indicates that they have the “right gene” and that it has efficacy against late blight. Transgenic lines planted in high pressure plots in Toluca, Mexico appeared to be highly resistant. The resistance is effective in foliar tissue but does not provide tuber resistance. A few field trials have been done in the USA, but the ABSPII personnel were unable to provide results of those trials. Dr. Raman provided data for one field trial of four transgenic RB lines in Toluca. Dr. J. Bradeen of the University of Minnesota kindly allowed the review team to see data from a replicated field trial of another 50 transgenic lines developed from four cultivars. Based on these data, the RB gene provides good early- to mid￾season protection from the late blight strains evaluated in those trials. A couple of issues regarding “proof-of-concept” remain to be settled. Although RB is claimed to be a “durable” and “race-non-specific” resistance gene, it is a single resistance gene, and the history of late blight resistance breeding suggests that it may be unwise to rely on a single gene being as durable or broad spectrum as claimed. A possibly related concern stems from a 2004 trial with RB conducted in Corvallis, Oregon in which RB lines appeared as susceptible as the susceptible control lines. The USDA-ARS scientist who conducted this trial, Dr. C. Brown (Prosser, WA), speculated that it was due to the late onset of disease pressure at a time when the lines were maturing. He surmised that RB may lose its effectiveness in maturing plants. This is apparently the basis for the statement made in the ABSPII potato PCP document, “For reasons still not known, the RB gene does not provide adequate late-season control when plants are senescing” (Potato PCP p. 4). It is impossible to discern 23 from the available data whether the breakdown of resistance was related to appearance of a compatible strain that overcame the resistance or if it was a maturity-related issue that affected the physiology of resistance. However, a similar scenario apparently exists in the Netherlands with a potato cultivar with LB resistance from S. bulbocastanum derived by traditional plant breeding, that apparently has unsatisfactory late-season resistance. Regardless of the cause, this trial raised a red flag indicating the need for further research (see below). The project has assembled an impressive team of potato and biotechnology scientists in both India and Indonesia (the team did not visit Bangladesh). They have outlined a two-track approach to transfer the RB gene into cultivars suitable for local markets in India, Bangladesh, and Indonesia. Track 1 seeks to transfer RB from validated transformed lines into breeding lines via conventional breeding, while track 2 would transform local cultivars using Agrobacterium￾mediated transformation using the Wisconsin construct and protocol. Two resistant transformed lines developed from cultivar Katahdin have been transferred to target countries India and Indonesia; arrangements are being made to transfer these lines to Bangladesh. Imported plants have been multiplied by in vitro culture in both recipient countries, and have also been planted in the greenhouse to multiply via mini-tuber production and to provide source leaves for detached leaf assays. Both India and Indonesia anticipate field trials of these parental RB-lines imported from Wisconsin in the very near future (first half of 2006). Target cultivars for intercrossing have been identified. Unusual winter weather led to flower abortion in the first crossing block at Modipuram, but the program intends to make the same crosses in a summer crossing block at the Central Potato Research Institute (CPRI), Simla. The plasmid used by the Wisconsin group to generate those transgenic lines has also been transferred to CPRI, India and to the Indonesian Center for Agricultural Biotechnology and Genetic Resources Research and Development (ICABIOGRAD), Bogor, Indonesia to be transformed into adapted cultivars. In both places the plasmid has been transformed into E. coli for replication and molecular verification, and transformed into Agrobacterium tumefaciens for plant transformation. Programs in both countries have proven that they have the capability to transform potato and to use PCR to verify transformation. ICABIOGRAD has optimized their plant regeneration protocol for each of the target cultivars, and has commenced transformation experiments, with several putative transformants of two lines identified at the time of the site visit in late January. CPRI (India) has initiated co-cultivation to transform the target cultivars, and had obtained more than 12 putative transformants of each of their target lines at the time of the site visit, with a target of more than 100 events per cultivar to screen for efficacy, type, and agronomic performance. Both India and 24 Indonesia anticipate having their own confirmed transgenic lines within the next 18 months - probably sooner. Although both Indonesia and Bangladesh have chosen European cultivars for transformation (Indonesia is also transforming recently released cultivars developed locally), the chosen cultivars are beyond variety protection, and licensing should not be an issue. The cultivars chosen by CPRI for transformation and crossing, Kufri Jyoti and Kufri Bahar contain resistance genes to late blight that may provide some additional background protection for the RB gene. Apparently decisions on how screening for single￾copy events should happen remain to be made. CPRI and ICABIOGRAD implied that they might be sending DNA to Wisconsin for analysis (Southern blot analysis). Probably, facilities in either country could be contracted to perform this analysis, e.g. the International Center for Genetic Engineering and Biotechnology (ICGEB) in India or the R&D Centre for Biotechnology-Indonesian Institute of Sciences (LIPI) in Bogor, Indonesia. One issue that should be addressed is whether the “late season” failure is due to lack of gene expression, or to failure of the downstream response pathway to function correctly in maturing tissue (unless it is due to appearance of late￾season compatible strains). The project leader should lead efforts to understand this phenomenon, or at least keep abreast of related issues and inform team members. The University of Minnesota has developed an allele-specific real time RT-PCR assay that could be used to investigate expression over time and has initiated studies on this topic. Similarly, the University of Wisconsin has developed a construct in which the native RB promoter is used to drive expression of a fluorescent tag that could be used to investigate gene expression. Since potato is a photoperiod-sensitive crop, and short days lead to early tuberization and maturity, the efficacy of RB under short days such as might be experienced in the winter season in India, or tropical highlands in Indonesia should be examined. Once the relationship of RB efficacy to maturity and photoperiod is understood, management strategies to optimize deployment of RB can be worked out. The project appears to be at least 6 months behind schedule for the target milestones suggested by the tables in the project document for the breeding component, but near target dates for the transformation aspect of the project. Testing for late blight resistance had commenced at both Indonesia (whole inoculated plants) and India (detached leaf assays; CPRI-Simla, CPRI￾Modipuram) at the time of the site visit. Although it was too soon to see definite results (except for Modipuram), transgenic plants appeared to be developing lesions similar to susceptible lines at that time. 25 The timeline to commercialization is difficult to predict. ABSPII does not plan to have a LBR potato product ready to commercialize for several years. The project timeline calls for a decision-making process in 2008 to finalize events to advance for full regulatory package development. This timeline is possibly achievable for strategy II (direct transformation of local cultivars) but not strategy I (gene transfer by breeding). The timeline would allow two years of field testing of new events (2007, 2008), probably being an initial screen for resistance and type (2007) and multi-location trials in the second year. Ex ante economic studies showing the anticipated benefits of late blight resistant potatoes have been conducted in India, Bangladesh, and Indonesia. In each case the economic impact of late blight and anticipated benefits of late blight resistance are strongly favorable for the project. The cost of late blight in each country is high, and the project should be rated a priority project on that basis. In conclusion, the potato LBR project is a very promising project for which efficacy remains to be proven in the target environments in the target countries. Provided RB-transformed potato lines have acceptable levels of resistance, the project should be encouraged to consider other resistance genes (e.g. Rpi-blb2) and IPM strategies to delay development of compatible strains of P. infestans. C.7. Sigatoka and Nematode Resistant Bananas The review team did not evaluate this project. 26 D. FUTURE RECOMMENDATIONS D.1 Overview The most important overall recommendation from the review team is that USAID continue with a biotechnology program for developing countries. We saw ample evidence that the projects (as well as others not yet developed) could greatly benefit the countries concerned, but that private companies and local governments are unable or unwilling to carry them out for a variety of reasons. This program will undoubtedly have great benefits, and we are very encouraged to see the progress to date, realizing that this is just a small sample of the opportunities that exist. The remainder of our recommendations is focused on suggestions for improvements in the implementation of the program, in order to maximize the returns from the funds available. The review team realizes that circumstances at the beginning of the ABSPII program were different than they are today and will continue to evolve. We also recognize the progress that has been made since the inception of ABSPII. Consequently we did not consider it productive to attempt to analyze what could have been a better way to run the program from the beginning. Rather, our efforts were focused on the program as it exists today, and how best to manage ABSPII from this point forward. Our specific recommendations for future implementation are listed below. D.2. ABSPII Internal and External Communication Communication for any program is always a challenge especially when multiple scientific and commercial disciplines are involved. This problem is compounded when the program also involves different countries and various for-profit and government institutions. While the effectiveness in communicating can in some cases largely be attributed to specific individuals, it is the belief of this review team that a system should be in place to facilitate communication as much as possible, without relying on individuals to always bridge the communication gap on their own. To the credit of ABSPII, they have developed a structured system of monthly conference calls between the various stakeholders involved in specific projects. This appeared to be working quite well for some projects. Our recommendation is that this system continues for all projects with the same discipline and regularity. 27 Communication is that between the management group of Cornell University and USAID is another issue. We do not see how this can be effective without at least bimonthly face-to-face meetings between the director of the ABSPII and USAID. There should be a detailed agenda for every meeting to go over future steps as well as the progress and issues of the past. The content of these meetings can be aided largely by the recommendations in the following section on stage-gates. These bimonthly meetings should be the basis to discuss options and make joint decisions with the management of both institutions, rather than one of the institutions giving notice to the other that a decision has been made. One additional recommendation in communication is that the director of the PBS be included at least for some of these bimonthly meetings. There appears to be opportunity for synergy as well as overlap between PBS and ABSPII. Having these groups meet on regular basis may help facilitate better utilization of resources. D.3. Develop a stage–gate product advancement system Commercialization of products by companies always involves some type of formal or informal product advancement system. A structured system is usually used when product candidates require a large investment of resources, multiple disciplines or long product development cycles. In the case of the product candidates in ABSPII, all three conditions are present but no structured product advancement system is in place. Therefore it is our strong recommendation that a stage-gate product advancement system be put in place as soon as practical. There are many examples of these systems used today that can be found in various companies and there is an extensive literature on this process4. It is important to use the basic structure of the system but to also customize this to ABSPII’s specific needs. Using a textbook approach alone will most likely result in a system that has many unnecessary steps and will cause individuals to criticize the system without seeing the benefits. Therefore it is recommended that the system be developed with all the stakeholders to obtain “buy-in” from the start, and to make sure that all steps are appropriate. In addition, (a) consultant(s) may be hired if needed to facilitate the process and help put a relevant system in place. The review team believes that the institution of such a system would be a great tool to help solve many of the management, resource allocation and communication issues described earlier in this report. 4 For example, Cooper, R. 2001. Winning at New Products: Accelerating the Process from Idea to Launch Third Edition, Perseus Books, N.Y, 425 pp. 28 In order to provide this recommendation with more concrete suggestions, the review team further recommends that the product advancement system address several specific concerns. The first of which is to accurately define the current stage of each project. While this has been attempted in ABSPII in the existing tier system, we believe this needs much more rigor and definition. At the very least we would recommend a system that differentiates between “proof￾of-concept”, early product development, late product development, and market introduction. While most commercial companies would divide this last commercialization stage into more areas including production and sales, we do not see this is as an essential or even appropriate part of ABSPII. Thus, market introduction would entail the release of the product so that entities other than ABSPII can assume control of the commercialization process. At each of these stages we recommend that specific activities be detailed for each of the following areas: research, regulatory, intellectual property, product development, marketing, production, and public relations. In this way resources will be allocated that are appropriate for each stage. This should reduce or eliminate premature allocation of resources to early-stage projects, thereby making adequate resources available for more mature projects. This process should greatly help the existing problem of trying to fund too many activities for too many projects simultaneously. As an example, public relations support may be very important during market introduction but of questionable value during the early stages of research, when it can unintentionally create false expectations. At each stage in the system we recommend that specific quantifiable milestones be agreed upon in advance, and a project should not enter the next stage until it meets these milestones. For example, simply demonstrating pest resistance would not be an acceptable specific milestone. Instead, at each stage there would a quantitative determination as to what is an acceptable level of resistance and into what germplasm. For example, a milestone may read; “at least 70% of the transgenic plants must exhibit at least 80% resistance with a yield of at least 95% of the commercial variety when not infected”. The actual milestones may be different for each project and for various stages of the candidate products. Determination of milestone completion should be objective. Discussion would then center on the definition of acceptable milestones rather than whether they were met. We further recommend that at the beginning of each stage all of the resources required to bring a project to the next stage are defined, as well as estimates of their cost. We realize that such estimates may be quite rough at first, but they should acquire more precision as a project proceeds. These estimates 29 will invaluable in making decisions concerning the availability of resources required. Projects should not be allowed to advance to the next stage, unless sufficient resources for continuation have been identified and can be allocated. The last recommendation with regard to the product advancement system is that projects be reviewed by ABSPII management and USAID at their bimonthly meetings. These reviews would focus on progress toward established, and agreed upon milestones, and whether or not the project is ready for advancement. We also recommend that at least one outside reviewer be present for these discussions. This need not be the same person for all projects but could be a recognized expert in the specific area relevant to the project. This individual could participate in the discussions by phone conference to save on costs. The primary role of the outside reviewer would be to provide an additional objective resource for the projects. This individual could also help facilitate any decisions when USAID and ABSPII management have different interpretations of how to proceed. Further, we believe that the annual meetings of the advisory board are not the most appropriate fora at which to take these decisions. Rather they could be used to ratify decisions made by the more focused meetings. D.4. Project Management As noted earlier, the review team considers that the resources spent on management for the respective projects are inordinately high. This is not to say that the management group is not working hard. However, we believe that an inappropriately high level of management activity has been devoted to many of the individual projects, given their current stage of development. Typically, projects in the later stage of development will require more management than those in the early stages. Since we would define most of the projects to be in the early stages of development, we recommend that the overall level of central management decrease with the current portfolio of projects. We recommend that more resources go to individual researchers in local countries to advance the research and technical issues which are lacking, rather than spending resources on management of the product candidates before it is even known whether the project will meet commercial standards. The product advancement system should help identify the level of management required for specific projects. We also recommend that the “proof-of-concept” projects be managed by the principal investigator (PI), much in the same way that grants are administered. Thus, the role for ABSPII central management would be minimal. ABSPII central management may be used to help select projects, but after selection the PI would be responsible for ensuring that a project meets its milestones. Once the first set of milestones has been accomplished, the USAID/ABSPII management group would have the responsibility of deciding if 30 a project was ready to advance to the next stage. The advancement to the next stage of product development would occur, i) when milestones are met, ii) if there is enough money available and, iii) if there is still consistency with USAID priorities. At any given time, there may be only enough money for one or possibly two candidates in the later stages of development. Therefore, even if early stage products meet the milestones, advancement to the next stage may be suspended or the project even terminated, because the product pipeline is too full. A pyramid approach to product pipelines is typical in commercial operations. With many projects in early stages of development, some may therefore not advance for a variety of reasons. The stage gate system provides management with a tool to facilitate decisions as to when it is appropriate to advance a project, to terminate a project and to allow new products into the pipeline. It is not clear to the review team that sufficient funds are available to support all of the projects to completion. Therefore, it may likely be necessary to eliminate some projects or put others on hold until enough resources become available. We feel it is better to support a few projects to completion rather than let several projects languish because they do not receive the needed resources. We have made some comments as to specific products below but we believe the final decision as what the product pipeline should look like should be made jointly with ABSPII management at Cornell University and USAID, after placing the candidates in the stage-gate system. D.5. Definition of roles and responsibilities The review team believes that the roles and responsibilities of the various participants in the project should be more clearly defined. This has been implicit to date, but we recommend that the roles and responsibilities be reexamined and explicitly stated to all stakeholders to avoid confusion. We highlight some of the examples about which we are most concerned. The first of these concerns the position of Director for ABSPII. We recommend that the Director have a similar role and portfolio of responsibilities as a Chief Scientific Officer in a commercial company. This would require examining broad strategies involving similar technology or intellectual property leveraged over several countries. This individual would provide technical oversight, act as the first line of review and ensure that more detailed reviews with consultants occurred when needed. The Director would also be one of the first to identify projects which are progressing exceptionally well or exceptionally poorly, and begin to make approparite adjustments. This person should also be one of the main conduits for information to USAID. 31 We also recommend that the role of the Principal Investigator (PI) be defined more clearly. Currently, the responsibilities and authority of the PI appear vague. We recommend that the PI execute the project and be responsible for all aspects of the management of the specific project. This individual would report directly to the ABSPII Director. We recognize that the expertise required at early stages may be very technical and that at later stages may be much more commercial. It is unlikely that there will be one individual PI who could handle this transition all the way through the system. Therefore, it is may be expected that the PI will change over the life of a project. In the early stages, a research scientist may likely be the PI, and others on the team may be involved in other aspects of commercialization such as regulatory, IP and application testing. As the product moves through the system the research scientist may take a decreasing role. Regulatory or production issues often dominate in the later stages of development, and during these stages another individual may take the lead, while the research scientist steps into a supporting role. The ABSPII Director should ensure that the most appropriate person is the PI, depending on the concerns and stage of the specific project, but that the PI should be the driving force to coordinate activities and make sure progress is being made. We further recommend that the current oversight system be strengthened significantly. This function may be met by an advisory board or some other grouping of consultants or experts. The current advisory board meets annually and while it can provide some oversight, we recommend that much more is needed. We recommend quarterly meetings to review the overall goals of the program, to keep the advisors familiar with the program, and to monitor progress. This group may be further divided up into subcommittees that would be responsible to provide more detailed oversight of specific areas such as IP, regulatory, technical evaluation, and evaluation of the ABSPII Director, much in the same way a board of directors may operate in a commercial company. D.6. Relationship to Cornell University The review team also examined the image and the role filled by Cornell University as the site of the central management of ABSPII. We found evidence that Cornell University specifically added value because of their reputation and contacts with certain countries. We also found that there is a perception that a land grant university would be more credible entity for program management than USAID, even though the funds for ABSPII are derived from USAID. While the review team sees benefits to having Cornell University (or another University) manage the entire program, there is also the disadvantage of having one university manage the wide array of technical projects in diverse geographical areas. In addition, while Cornell University has arguably done as well as any university could for such a global program, no one university 32 possesses the best expertise for every project. Furthermore, universities inherently lack of understanding and experience in commercializing products. With this in mind we recommend that in the future, USAID continue to use land grant institutions, but USAID may want to consider a different format. Thus, for example, the projects could be divided up by specific technologies, or by geographical areas. This would allow specific universities to manage areas for which they possess the best expertise, and avoid one institution being spread too thinly on projects for which another may have better contacts or expertise. In addition, we recommend that as projects approach the later stages of development, Contracts may be better vehicles than Collaborative Agreements with which to fund the projects. In summary, we recommend that future biotechnology projects use Grants, Collaborative Agreements and Contracts5 as mechanisms to effectively accomplish the work. The specific product candidates and stage of the project would dictate which is most appropriate funding mechanism, rather than attempting to have one mechanism address all situations. D.7. Individual Project Recommendations The specifics of the individual projects have been given earlier and the recommendations summarized here. D.7.1 InsectResistant Eggplant. This is the lead product candidate which has made excellent progress. We recommend giving this the highest priority and full support toward commercialization. There is little to change or criticize for this project. D.7.2. Drought and Salinity Tolerant Rice. The review team considers that “proof￾of-concept” is lacking in that there are no specific requirements which define commercial acceptability. The rigor of placing this in the product advancement system and establishing milestones should help determine what technical hurdles need to be addressed to meet the specific market requirements for commercialization in individual countries. We recommend obtaining more specific information as to what will constitute a commercially viable product in specific locales. This will then determine the likelihood of the current technology meeting these requirements before going forward. Once this information is obtained, the project could be then placed in the stage gate system. D.7.3 Virus Resistant Tomatoes. The strategy for TMVLC resistance is very promising but we suggest that there be could be benefits to better coordination 5 As noted earlier, We use the terms “Cooperative Agreement”, “Contract” and “Grant” as defined by USAID. 33 between Africa and Asia. CMV and PVY are good secondary choices but it is unclear as to their importance in the absence of TMVLC resistance. Therefore, we would encourage more effort to be spent obtaining the first product even if at the expense of the secondary virus resistance. We would recommend reexamining who is the most appropriate PI. In addition, we would suggest examining the use of the Bt gene for control of lepidopteran pests on tomatoes in the relevant countries. D.7.4. Papaya Ring Spot Virus Resistance. This project looks very promising and the second best candidate for commercialization. There is little to change for this project. D.7.5. Disease Resistant Potato. We believe that this project has been prematurely advanced to second tier status. More technical data are required to demonstrate “proof-of-concept” for target countries. The rigor of placing this in the product advancement system and establishing milestones should help determine what technical hurdles need to be addressed to meet the specific market requirements for commercialization in individual countries. As outlined in our earlier general recommendations for “proof-of-concept” projects, we recommend reduced effort in management as well as in activities in other non-technical areas. In line with those recommendations, it may be appropriate to consider who would be the best PI to oversee the project during this phase. D.7.6. Virus Resistant Groundnut. The review team considers that excellent progress has been made for the very early stage of this program. We recommend collecting better marketing data as to the potential impact of this program. D.7.7. Sigatoka and Nematode Resistant Bananas. The review team did not evaluate this project. 34 APPENDIX 1 The review team were provided with this following Statement of Work STATEMENT OF WORK (SOW) Following is a list of key areas which USAID has asked this evaluation to address. The list is designed to guide the team’s findings and written report, and is not a comprehensive list. TECHNICAL REVIEW A. PROJECT GOALS, OBJECTIVES, AND WORKPLANS: 1. Are project goals appropriate and relevant in the context of agricultural biotechnology research and USAID’s objectives for international development? Is ABSPII well situated to achieve its goals as evidenced in Product Plans, annual work plans and progress reports? Is the time horizon allowed for achieving project goals and objectives realistic? 2. Are ABSPII’s criteria for choice of projects reasonable to anticipate project success? Are ABSPII’s expectations for projects realistic in the anticipated timeframe? 3. In designing and implementing its activities has ABSPII been able to build upon the lessons learned under the original (ABSPI) project? 4. Has ABSPII effectively implemented the project plans per the original proposal? 5. Is there a mechanism for adapting goals and objectives to the changing context of agricultural biotechnology? How have project goals, work plans, and partnerships been modified to reflect changes in budgetary considerations, new opportunities, or other constraints? B. RESEARCH/PRODUCT DEVELOPMENT: 1 Have the choice of projects (crop traits) been appropriate to the needs of developing country agricultural systems? Has there been sufficient use of market data in the choice of projects? 2. Are the technology development activities considering the appropriate mix of activities to achieve eventual commercialization? Should the 35 activities under the projects be modified during the course of the program, and if so, how? 3. Has the ABSPII collaborative research process been an effective means of facilitating access to technology between US and host country collaborators? What should be the emphasis on the research being conducted by developing country scientists and US/international scientists and what are the implications for resource allocation given the biotechnology research infrastructure in many developing countries? 4. What role has the private sector had in shaping or contributing to the ABSPII research agenda? Has this relationship functioned successfully? Should there be a greater emphasis for involving the private sector? Has ABSPII provided benefits to developing country partners from collaborations with the private sector? C. IMPACT EVALUATION 1. The project has initiated a series of economic impact evaluations, one for each product under development. Is the approach taken for these evaluations appropriate? What additional types of impact assessments would be useful? Are the results of the impact assessments being used appropriately in ABSPII decision making? D. INTELLECTUAL PROPERTY AND TECHNOLOGY TRANSFER: 1. ABSPII has taken a product-based approach to IPR issues. Has this approach been effective? 2. What has been the impact of ABSPII in promoting technology transfer from public research institutions to the private sector in developing countries? Is the approach appropriate, effective and sustainable? 3. What has been the impact of addressing IPR at the institutional level in terms of assisting public research institutions in developing countries to manage IPR issues related to access to proprietary research tools? 4. What has been the impact or benefit of addressing IPR at the national level, e.g. in India, Indonesia, Philippines, Bangladesh and East and West Africa? 5. Was the approach taken by ABSPII at the national level appropriate for the goals of the program? Was the approach an effective strategy? Has there been sufficient commensurate funding allocation for such efforts? E. PUBLIC AWARENESS AND COMMUNICATION: 36 1. Has ABSPII developed an effective public awareness and communication strategy? Is the current level of activity adequate to achieve the goals of the project? 2. Public awareness and communication is also a goal of the Agency funded biosafety efforts. How well have ABSPII efforts been integrated in this area with the efforts of the biosafety projects? MANAGEMENT REVIEW A. PROJECT MANAGEMENT/ORGANIZATION: 1. Have the various components involved in the ABSPII management and oversight (management team at Cornell University, ABSPII Advisory Board of Directors and USAID) performed effectively together and has each group made a positive contribution? 2. Are ABSPII expectations for projects in line with available resources? Have these resources been managed effectively and efficiently? 3. What mechanisms are used to balance in identifying the research agenda or making modifications if required by budget cuts or changing opportunities? Could these mechanisms be improved? 4. Has the ABSPII program management facilitated ownership of the program and made best use of the other participating institutions in the U.S. and in developing countries? 5. What is the working relationship between US institutional members of the ABSPII? How has the consortium functioned in terms of project participation? Have the various institutions been utilized effectively in achieving the project goals? 6. Has ABSPII worked effectively with other efforts, including other USAID funded efforts such as in biosafety, to achieve maximum impact? Are there specific recommendations to the project to improve these interactions? 7. Has ABSPII been encouraged to work synergistically with other USAID programs? How successful have such interactions been in contributing to ABSPII’s overall goals? 8. Has the management provided the necessary liaison between and across countries, principal investigators (PIs), Missions and USAID/Washington through reports, meetings and other information to facilitate participation and support? 9. Has the composition and involvement of the Advisory Board been effective to achieve project goals? Given the complexity and size of ABSPII 37 would it be more effective to have a Technical Advisory Group for each technology project in addition to an overall Advisory Board? This structure was proposed in the original proposal and revised to the current structure by the ABSPII management and USAID. 10. What changes in ABSPII management would improve organizational efficiency and effectiveness? B. TRAINING/INSTITUTION DEVELOPMENT: 1 What impact have ABSPII projects had on U.S. institutions' international research or development activities/capability? Has the ABSPII program been integrated into other university activities or goals? Has ABSPII been recognized by university administration as contributing to university priorities both at the home institution and other institutions? 2. What impact have ABSPII projects had on host country institutions' research activity/capability? 3. PBS and SABP have taken the lead in addressing biosafety policy in the host countries where ABSPII is working. In some countries the efforts of ABSPII have shaped an environment conducive for commercialization of bioengineered crops. Given the mandates and expertise of these projects, has this been an effective approach? Are there ways to more effectively work on biosafety policy issues? 4. What has been the progress in training of students, technicians and/or other research personnel both overseas and in the U.S? Considering the current and past budget resources, has training been adequately addressed in relation to other priorities? Which, if any, areas of specialty need more focus? C. INFORMATION DISSEMINATION, USAGE AND IMPACT: 1. Is ABSPII poised to effectively analyze and disseminate its findings so as to contribute to the body of development knowledge and practices? What mechanism is used to synthesize project results for application to others? D. BUY-INS: 1. What are the benefits and/or detriments to add-on contracts from country Missions or other donors? 38 2. Has the project changed its focus or objectives to accommodate buy-ins? If so, has this represented good opportunities or distractions? E. FINANCIAL: 1. Is the cost of management appropriate for the size of the Cooperative Agreement? Is the present structure of management cost-effective and efficient? Are there examples of modifications that would improve the performance of the Cooperative Agreement? 2. Are the administrative costs competitive with other modes of administering research programs? 3. Have financial reports and vouchers been submitted in accordance with USAID requirements? What has been the track record of the grantee and sub-grantees in using funds in a timely manner? 4. Have financial reports been used to make budget or workplan reallocations in cases where costs are higher/lower than expected or rate of expenditures faster/slower than anticipated? F. SUSTAINABILITY: Institutionalization of USAID supported interventions is critical for long term sustainability. 1. How is sustainability addressed by the ABSPII? Is sustainability addressed directly in project design? Is there verifiable progress on institutionalization from project efforts to date in any of the country/regional programs? 2. How does the project take into account the financial and institutional requirements among its developing country partners to prepare for continued operation of project activities after USAID funding is terminated?